A six-engineer team shipping AI agents at SaaS scale
Durable set out to make business ownership accessible by removing the operational grind that crushes small-business owners. The company’s AI business builder helps entrepreneurs launch websites, manage customer relationships, and automate content and SEO work — but that convenience hides a serious multi-tenant engineering problem.
Under one seamless customer experience, Durable runs millions of individual businesses across multiple products. Those businesses have wildly different traffic profiles, and the power-law spikes mean a small subset of tenants can consume a disproportionate share of compute at any moment. For a team of six engineers with no dedicated DevOps staff, that made self-hosted infrastructure a second product to maintain.
The company chose consolidation over incremental fixes: one codebase, one infrastructure platform. As CPTO Osama Khan put it, the team realized it either had to build its own platform or build on top of an existing one. It chose the latter.
Where self-hosting broke down
Durable’s pain points weren’t the usual “this is slow” complaints. They were the structural problems that appear when you operate multi-tenant, multi-product infrastructure on a small team:
- Custom domains and SSL at scale — including thousands of dollars spent on SSL termination for customer domains.
- Multi-region cluster maintenance — infrastructure engineers had to keep clusters healthy across multiple geographies.
- Security isolation — a DDoS attack aimed at one customer could degrade performance for every other site on the platform.
- Observability and cost attribution — tracking usage and spend across millions of tenants.
Those problems were manageable when Durable was mostly a website platform. But once agents became core to the product, the infrastructure demands changed fundamentally.
Agents introduce a different class of infrastructure problems
Durable’s product is multi-agent, multi-model, and multi-modal. That created three issues self-hosted website infrastructure wasn't designed to solve:
- Model orchestration: The team needed to switch models and providers quickly for reliability and cost reasons — without being locked into contracts or reworking the system with every outage or model release.
- Tenant isolation: In a multi-tenant agent system, context bleeding between customers is existential. Durable required strict guardrails so one business’s data could never leak into another’s workflows.
- Per-customer AI economics: AI workloads vary enormously per tenant. Durable needed customer-level visibility into runs, models, and spend to support usage-based and outcome-based pricing with real fidelity.
A full rewrite, not a slow migration
Rather than incrementally patching the old stack, the team treated the infrastructure change as a single decisive move. Khan described it as skinning the old product inside an iframe, pushing to production on the new platform, then ripping out the self-hosted infrastructure entirely.
The old codebase wasn’t refactored. With a lean team, Durable used coding agents to accelerate the rewrite — burning tokens in exchange for speed as models improved — and emerged with a production-quality codebase. Khan acknowledged the conventional wisdom against rewriting a product, but said it worked in their case.
Cost wasn’t the original motivation, but it became a measurable outcome. Durable’s new infrastructure runs at 3–4x lower cost than the self-hosted setup did.
What the new platform buys them
Today Durable’s entire stack runs on the consolidated platform. The company serves roughly 1.1 billion tokens per day — about 360 billion per year — and ships production agents to customers in a single day.
That scale is attached to more than three million customer businesses, ranging from a 19-year-old selling chocolate-covered bananas at fairs to a newly licensed plumber to a laid-off tech executive building a new venture. All of them are running on infrastructure maintained by six engineers and no DevOps team.
Founder James Clift said the team has achieved roughly 100x leverage from the platform choice, with every engineer, product manager, and designer delivering about 10x the output possible a few years ago. That efficiency, he argues, isn’t a one-time win — it’s the operating model he expects other technology companies to adopt.
