← Back to all posts

We Tested AI App Builders Again. This Time, We Asked for a Whole Marketplace.

We Tested AI App Builders Again. This Time, We Asked for a Whole Marketplace.

We Tested AI App Builders Again. This Time, We Asked for a Whole Marketplace.

September 10, 2026 · Updated September 11, 2026

AI app builders can all make an impressive first screen. We wanted to know what happened after that.

For this test, we asked a group of AI builders to create Casa Nova, a premium multi-seller marketplace for independent furniture and home décor designers. The brief went far beyond a landing page. It called for a populated storefront, search and filters, product and designer pages, favorites, cart and checkout, buyer accounts, seller operations, admin tools, PostgreSQL, secure image storage, role-based permissions and inventory that changes safely when orders are placed.

We first tested Atoms, MeDo, Lovable, ChatGPT Sites and Hercules. We then expanded the comparison with Bolt, Replit, Base44 and Rocket. Kreova was developed through the same kind of conversation available to its users: build something, inspect it, ask a short follow-up and continue.

The result was not close on full-system completeness. Several competitors produced beautiful work. Some generated substantial code and data. But Kreova was the only builder in our sessions that reached the entire connected application-and kept improving it through ordinary conversation.

How we evaluated the results

We looked beyond whether a homepage appeared. We asked:

  • Was there a complete, navigable application?
  • Did application data persist?
  • Were buyer, seller and admin permissions actually enforced?
  • Did checkout create orders and affect inventory safely?
  • Could sellers manage listings and upload real photographs?
  • Did dashboards use connected data rather than decorative numbers?
  • Were APIs, notifications and integrations available to extend the app?
  • Could the builder continue improving the same project through ordinary conversation?
  • What remained when the available credits, tokens or run ended?

This was not a laboratory benchmark. The products use different models, plans, native backends and credit systems, and they change quickly. Later tests also used a stricter version of the brief that made access control, seller isolation, safe inventory and administration explicit.

The scores below describe the outcomes we observed in our own sessions. They are not a claim that every user will always receive the same result.

The side-by-side result

Builder What it delivered in our test Main limitation Observed score
Kreova Complete connected marketplace, PostgreSQL, real roles and policies, storage, checkout, seller/admin operations, analytics, email, API and webhooks Required iterative build turns; some issues were found and corrected along the way 9.3/10
Replit Excellent editorial storefront, populated catalog and substantial seller UI Seller dashboard and listing controls were reachable while signed out, defeating the requested role boundary 7.6/10
ChatGPT Sites Coherent, attractive and reliably preserved marketplace experience Used D1/SQLite and demo-style role switching rather than the requested PostgreSQL identity model 7.4/10
Base44 Strong storefront, rich seed data, filters, product pages, wishlist and server-side checkout Buyer account, seller operations and admin were deferred; authentication was not enabled and sensitive reads remained open for the showcase 6.8/10
Hercules Strong initial multi-page application using Convex and native auth Difficult production follow-up exhausted credits and disappeared before becoming a usable checkpoint 5.5/10 overall
Rocket Attractive frontend structure and several polished commerce screens Explicitly frontend-only at this stage; the requested Supabase/PostgreSQL backend was deferred, and the deployment could not be verified in our browser 4.0/10
Lovable One generated page Credits ended before the marketplace was delivered 3.5/10
Bolt Database-oriented plan, SQL work and asset search Tokens ended before a completed application was available to audit 3.0/10
MeDo Detailed implementation plan Remained on image search for more than 20 minutes without delivering the system 3.0/10
Atoms Furniture and interior assets Exhausted the available credits before delivering an application 2.5/10

The numbers are subjective. The observable outcomes are what matter.

Atoms - assets without the application

Atoms began collecting furniture and interior photography. It exhausted the available credits after pulling images, before delivering the marketplace itself.

The problem was not the photography. The problem was prioritization: the allowance was consumed before the user received a usable product.

Observed score: 2.5/10

MeDo - a detailed plan that stalled before delivery

MeDo produced a comprehensive-looking plan: visual direction, Supabase, migrations, row-level security, seed data, every storefront page, buyer flows, a seller dashboard and authentication.

It then remained on the furniture and interior image-search stage for more than 20 minutes. At that point the plan was ambitious, but there was no completed system to evaluate.

Observed score: 3.0/10

Lovable - one page before the credits ended

Lovable produced one page from the available credits. That demonstrated frontend generation, but not the marketplace requested in the brief.

Observed score: 3.5/10

Bolt - a promising start that ended at the token boundary

Bolt recognized that the project needed a real database, access policies, seed data, routing, image assets and a large collection of storefront and operational screens. Its plan was sensible and it began executing SQL and assembling the application.

Then the session reached its token limit. The upgrade screen appeared before there was a completed marketplace we could inspect. As with Atoms and Lovable, the decisive question was not whether useful work had begun. It was what the customer had when the allowance ended.

In our run, there was no complete deployed system to audit.

Observed score: 3.0/10

Rocket - polished frontend ambition, with the backend deferred

Rocket produced an attractive Next.js application structure with an editorial homepage, product detail experience and multi-step checkout components. Its visual direction was credible.

However, Rocket explicitly described the result as a frontend-only build and said Supabase would be integrated afterward. That meant the central requirements-PostgreSQL persistence, real buyer/seller/admin authentication, row-level access rules, inventory transactions and seller isolation-were not yet part of the delivered system.

We also could not independently inspect the public deployment in our test browser because it returned a certificate-chain error. That may have been specific to the inspection path, so we did not treat it as proof that the app was universally unavailable. The larger issue was already clear from the builder’s own description: the backend remained future work.

Observed score: 4.0/10

Base44 - a substantial storefront that stopped before operations

Base44 delivered far more than a mockup. It generated 17 images and seeded five categories, six designers, 16 products, 32 product images, 47 reviews, 18 orders, 36 order items and thousands of analytics events. It built the homepage, shop filters, product pages, designer profiles, wishlist, cart and a three-step checkout.

It also created and tested a server-side order function that validated stock and reduced inventory. That was meaningful backend work.

But Base44’s own completion message placed the buyer account, order history, seller dashboard, listing management and admin console in a future “phase two.” Authentication was not enabled, and order and analytics reads were left open so the public showcase could display its seeded data. The intended policies were described, but the requested identity and permission boundaries were not actually active.

This was one of the stronger partial results, but it was still a storefront plus checkout rather than the complete multi-role marketplace.

Observed score: 6.8/10

ChatGPT Sites - a strong visual build, but not the same backend depth

ChatGPT Sites produced a coherent, attractive marketplace experience. It was stronger than the incomplete attempts above and preserved its work reliably.

However, the implementation did not match the requested stack or identity model. It used D1/SQLite rather than PostgreSQL and relied on a demo-style role selector rather than equivalent buyer, seller and admin authentication. It looked like the product, but it was not as complete underneath.

Observed score: 7.4/10

Replit - possibly the best first impression, undermined by access control

Replit was the strongest visual challenger in the expanded test. Its Casa Nova homepage had confident editorial art direction, refined typography, a convincing mobile layout and a populated 16-product catalog. Product pages loaded real high-resolution photography, and the seller area included revenue, orders, views, conversion data, recent activity and listing controls.

Visually, it was excellent.

The critical failure appeared when we tested the boundary rather than the screenshot. While signed out, we could open the seller dashboard and seller listing-management pages. Revenue and operational data were visible, and mutation controls such as Unpublish were presented to an anonymous visitor. We did not activate those controls, but the requested seller-only interface was plainly not protected at the navigation and page-access layer.

For a public design demo, Replit’s result was impressive. For a multi-seller marketplace where one seller must never see or control another seller’s business, that access-control failure materially lowered the overall score.

Observed score: 7.6/10

Hercules - the strongest first attempt, followed by an expensive failure

Hercules was the most credible competitor in the first round.

It chose its native Convex backend and Hercules Auth, created a seven-milestone plan, built a multi-page application, corrected compilation and lint errors, and produced a visually strong initial marketplace. That result was impressive, although it substituted Convex for the explicitly requested PostgreSQL and left seller CRUD, review submission, admin functionality and image uploads incomplete.

We then gave it a difficult production follow-up: secure multi-seller checkout, server-validated totals, seller-scoped order lines, cancellation with exactly-once restocking, audit events, notifications and buyer, seller and admin interfaces.

Hercules spent a long time reading and planning the architecture. It partially entered the schema and backend milestone, reached zero remaining AI credits, and the unfinished follow-up disappeared. The original interface remained, but the production upgrade did not become a usable checkpoint.

Its first generation scored much higher than its final benchmark result. Losing unfinished work at the credit boundary was the decisive issue.

Initial application: approximately 8.0/10
Overall observed score after the follow-up: 5.5/10

Kreova - a working system built through conversation

Kreova did not magically finish every detail in one enormous turn, and we do not claim that it completed the entire marketplace within the free allowance. That is not how we designed it to work.

Kreova’s free plan provides 10 build turns. Questions, previews, project browsing and sharing do not consume a build turn. A build turn is used when the agent successfully changes the application.

By the end of the first 10 build turns, Casa Nova was not finished. Cart, checkout, accounts and the operational dashboards still needed more work. But the project already had a real PostgreSQL schema, seeded marketplace data, access policies, authentication, a transactional backend action and working homepage, catalog, product and designer experiences. In other words, even Kreova’s unfinished free build had more connected full-stack substance than most competing runs left behind when they stopped.

From there, the user continued naturally with instructions such as “Next,” “add charts to that,” “make the images fill the box” and “show cancellation on the seller and admin dashboards.” Each successful turn took only a few minutes and became a recoverable checkpoint.

Those short exchanges ultimately produced:

  • An editorial storefront, catalog, filters and search
  • Product galleries, designer profiles, favorites and reviews
  • A PostgreSQL schema with realistic seeded marketplace data
  • Real buyer, seller and admin authentication
  • Row-level access controls for public, buyer, seller and admin data
  • Server-backed favorites and order history
  • Cart, checkout and transactional inventory updates
  • Seller revenue, orders, product views and analytics charts
  • Product management with real multi-image uploads to storage
  • Admin verification, category management and review moderation
  • Buyer cancellation with ownership enforcement
  • Atomic, exactly-once inventory restocking
  • An append-only cancellation event trail
  • Buyer and seller email notifications
  • Public data APIs and API-key access
  • Signed outbound webhooks, including a live Slack test
  • Desktop and mobile verification throughout

Kreova also found and disclosed its own mistakes. Product views initially updated a column that anonymous visitors were not allowed to modify. Kreova replaced that design with a dedicated view log and asked for a real HTTPS verification when its automated preview could not complete the hidden security challenge. The live visit reached the database and exposed a duplicate-view edge case that could then be addressed through another short instruction.

There were other imperfections during the build: some flows took multiple turns, an early access test used the wrong role, and several visual and permission issues needed correction. We include those details because they are part of the test. The meaningful difference was that completed work survived, problems were visible, and the same project could keep becoming more correct.

Observed score: 9.3/10

What the ten free build turns actually demonstrate

“Ten turns” should not be interpreted as a promise that every complex app will be complete after ten messages. Casa Nova is a large marketplace with three user roles, storage, commerce, analytics, notifications and operational workflows.

The important result is value per successful change.

Within the free build allowance, Kreova established the actual system: data model, policies, seed data, authentication, transactional backend logic and the central browsing experiences. The later turns expanded and hardened that foundation instead of replacing a disposable prototype.

That is a more useful promise than “one prompt builds everything.” It means a user can begin for free, see a real application take shape, ask unlimited questions while evaluating it and decide whether the project is worth continuing.

The real difference was not the first screenshot

Replit showed that another builder can produce exceptional visual polish. Hercules demonstrated that another builder can produce a convincing first application. ChatGPT Sites also delivered a respectable and durable visual result. Base44 assembled a substantial storefront and meaningful seed data.

The gap became clearest after the first generation, when the application needed deeper behavior and corrections.

Real software is built through iteration. Requirements change. A customer notices a mobile layout issue. A security rule blocks checkout. A dashboard metric turns out to be seeded rather than live. An order cancellation needs to restore stock exactly once and notify multiple parties. A warehouse or Slack channel needs the order through a signed webhook.

At that point, visual generation is no longer enough. The builder needs durable project memory, backend-aware tools, tests, access controls, integrations and recoverable checkpoints.

Kreova’s advantage in this test was not that it never made mistakes. It was that each successful conversation changed the real system, completed work remained available, and the next short message could continue from there.

What we learned

Credit quantity alone is a poor measure of value. The useful question is what the customer owns when a credit has been spent.

Did the credit produce a plan, an image search or a page? Or did it leave behind a migration, a working policy, a tested transaction, an uploaded file, a notification, an API, a signed webhook or a deployable interface?

Kreova meters one message that successfully changes the application as one build credit. Questions, previews, browsing and sharing do not consume build credits. Completed changes are saved as checkpoints rather than being hidden inside one enormous all-or-nothing run.

That model is intentionally conversational. People should not need to rewrite a software specification every time they want a change. They should be able to say what they mean, inspect the result and continue.

The conclusion

Our expanded testing reinforced the same product thesis:

Anyone can generate a frontend. Kreova builds the whole app.

In the sessions we observed, Kreova beat every AI builder we tested on complete, connected application delivery. That is a specific benchmark result, not a universal claim about every builder, prompt or future version.

Other products produced assets, plans, partial pages, beautiful storefronts or strong initial applications with important backend gaps. Kreova produced the database, authentication, permissions, storage, checkout, seller and admin operations, analytics, email, APIs, webhooks and live application-and continued improving the same system through ordinary conversation.

Explore Kreova or open the live Casa Nova marketplace.