The maintenance bottleneck behind a popular AI library
The AI SDK ecosystem has grown far beyond what a small team can keep up with by hand. The open-source project serves over 20 million npm downloads a week and has more than 26,000 GitHub stars. The scope of maintenance spans model providers, UI framework bindings for React, Next.js, Svelte, and Vue, sandbox execution environments, and harnesses for tools like Codex, Claude Code, and Pi. After years of that compounding growth, the repo reached more than 1,000 open issues and almost 800 pull requests by late June — the same week a notable Anthropic model release drove a surge of new PRs.
That backlog isn't something any individual contributor can close with more hours, no matter how disciplined they are. Since generating code is cheap, the queue only grows faster over time. Rather than scale the human team, the maintainers at Vercel built a software factory they call ai-sdk-factory. Four weeks after launch, it authors between 25 and 35 percent of merged PRs and closes 70 to 80 percent of incoming issues.
Three questions before building
The team started with a few design constraints about where automation fits in a project that ships foundational infrastructure.
Why not just more agents?
There's no shortage of examples of aggressive agent-powered maintenance in open source. Some maintainers run fleets of coding agents and encode every agent failure into project documentation. Others prefer to block AI-generated submissions entirely. All of those approaches still route every change through one human's full attention, which becomes the fundamental bottleneck. Because human accountability remains the core of trust in agentic engineering, the factory needed to maximize reviewer efficiency as a first principle rather than remove humans from the loop.
Where does AI SDK sit on the automation spectrum?
Software automation exists on a spectrum. At one end, agents write and deploy code with no human reading it. In the middle are harnesses where a person steers an agent or fleet. At the careful end are systems so risky they're barely automated, like firmware in critical medical devices. AI SDK operates closer to the careful end because millions of applications depend on it; quality and security are non-negotiable, and humans must control what ships. The factory's job is to automate the lifecycle around the human without replacing them.
Matching effort to risk
The depth of review a change needs scales with its risk. Centralized roadmaps with detailed specs can define risk up front, but open-source projects also get unplanned community issues and PRs that may not align with project goals. That means agents must evaluate full units of work in project context, not just generate code from a request.
The factory produces a comprehensive assessment for each change covering fit and risk, with a documented chain of evidence. That lets reviewers allocate effort appropriately: a quick glance for documentation fixes, focused validation for well-defined provider changes, and deep review for new public APIs.
Architecture of the factory
ai-sdk-factory autonomously processes incoming issues and pull requests. Specialized agents handle discrete, reviewable tasks such as reproducing bugs, implementing features, and creating backports for older SDK versions. A human remains in control throughout — every change is merged only after human approval.
One agent per task
The factory was built incrementally, starting with classification of issues into bugs, features, or docs updates. That first step provided visibility into the shape of the backlog and passed context to the more specialized agents built later. A single agent with skills for every step seemed simpler initially, but the team chose separate agents per task to keep each capability easy to test in isolation, debug, and maintain with its own prompts, context, and evals.
Current dedicated agents cover:
- Bug reproduction
- Bug fixes
- PR reviews
- Backports
- Documentation updates
- Feature analysis
- Feature implementation
Security baked in from the start
Sandboxes are the foundation of the defense architecture. Every agent runs inside an isolated Vercel Sandbox containing its code, runtime, and only the secrets required for its specific task. A shielding layer around each sandbox controls network access, blocking the exfiltration paths an attacker would use to pull secrets from the isolated environment. A public repository must assume every issue, PR, comment, and link is attacker-controlled; the sandbox contains the damage any single task can do. The last line of defense is always the human review required before merge.
From local CLI to managed infrastructure
The first several agents ran through a local CLI, allowing the team to iterate quickly on inaccuracies and friction. Once multiple steps ran reliably, the system moved to managed infrastructure built on Vercel Functions for the API and webhook ingress, Vercel Queues for task execution, Vercel Blob for logs, Vercel Sandbox for agent workspaces, and Neon Postgres to store factory data. GitHub webhooks feed the issue queue; workers pull them and kick off agent runs automatically. A monitoring UI tracks every run in parallel and visualizes the queue for reviewers.
Walking a feature through the factory
On July 24, a community member requested blocked-domain support in OpenAI web search, spawning issue #17898. The factory processed it end to end, ultimately merging the implementation to main and backporting it to two older SDK versions.
Classification
The first agent classified the request as a Feature with high confidence, commenting on the issue with its rationale and applying a label.
Analysis
The analysis agent made no assumptions about the request's technical validity. Instead, it generated and ran a probe script, issue-17898-type-probe.ts, which failed when it failed to find blocked-domain support on main. That failing probe served as documented evidence the feature was missing. The agent then built a spec: add an optional blockedDomains filter to the existing web-search tool and map it to the provider's blocked_domains field. The agent also confirmed the spec fit the provider-adapter architecture, was backward compatible, and scoped the necessary documentation changes.
Implementation
The implementation agent opened PR #18033. It ran a live end-to-end test — executing an OpenAI web search with wikipedia.org blocked and confirming the domain wasn't reachable — and included the test as evidence on the PR.
Automated review
A review agent scored the change and approved it without finding concerns, rating the feature as fully implemented with low side-effect risk, no performance risk, and low backwards-compatibility risk.
Human review and merge
Lars read the chain of agent evidence, reviewed the code, and merged PR #18033 into main.
Backporting
After the merge, the factory automatically opened two backport PRs: #18035 for v6 and #18036 for v5. The v5 backport didn't apply cleanly, so the factory agent labeled and committed the conflicted state, identified and validated a fix, and pushed it seventeen minutes later. Lars reviewed and merged both backport PRs.
Factory Output After Four Weeks
Just over four weeks into production, the software factory is responsible for a significant share of the project's merged work. Between 25–35% of all PRs merged weekly to main are authored by the ai-sdk-factory agents.
The impact is even more pronounced on release branches. Factory PRs account for over 50% of weekly merges to the v6 release line, with v5 showing a similar pattern. Backports to these older branches were previously skipped—the merge-conflict overhead rarely justified the effort. The factory has changed that calculus, and both v5 and v6 now receive considerably better support.
Issue Triage and Backlog Reduction
Issue handling shows a comparable shift. During July, the factory closed over 75% of all issues closed on the repository.
The overall backlog has dropped measurably. Open issues fell from a peak of 1,022 in late June to 844 by early August—a roughly 25% decline in open bugs over the same stretch.
Because the factory operates in the open, every pull request it authors on the repo is publicly visible and auditable.
Failure as a Feedback Loop
The most revealing part of operating the factory is watching how it fails. Each run terminates in one of four states, and only one of them ships:
- Success results in a merge; the other three outcomes are treated as inputs back into the system.
- Flawed runs indicate the agent produced incorrect output; the remedy is improved prompts, richer context, or a new eval case so the same error is caught automatically on the next attempt.
- Blocked runs mean the environment lacked a prerequisite—a credential, a service, or a dependency—and the solution is to provision it.
- Manual runs represent deliberate boundaries, prompting a reassessment of whether factory improvements now make automation safe to expand past them.
Every fix along these lines extends the automation boundary. Work that the factory could not be trusted with one week becomes part of its standard workload the next.
Operating the factory applies the same discipline teams already apply to test suites and CI pipelines, but directed at the system that performs the work rather than the system that validates it. As agents come to define the software development lifecycle, maintaining and improving the factory is positioned to become the core engineering role.
