The failure mode of the all-knowing agent

Meta's data column onboarding pipeline spans three code repositories, three programming languages and as many as six pipeline stages that must remain synchronized. Configurations are written as code: Python definitions drive C++ code generation, which feeds data processing services and, eventually, analytics systems. Adding a single column means coordinated changes across configuration registries, backend services and data pipelines.

The first attempt at automating this followed the common instinct: one agent loaded with every schema, code generation rule, routing rule, validation requirement and population pattern for more than 10 data sources. It passed demos. In production it began to hallucinate — blending subsystems, applying one subsystem's field naming conventions to another, or reaching for a code generation pattern from the wrong data source.

The trouble was context volume, not prompt quality. Beyond some threshold, extra knowledge made the agent worse. A preprint from the Windreich Department of Artificial Intelligence and Human Health at Mount Sinai supports the pattern in the other direction: orchestrated multiagent systems held 65.3% accuracy at 80-task batches where single agents dropped to 16.6%, with token usage reduced up to 65-fold, which the authors credit to task isolation preventing context interference.

A separate single-agent variant was also evaluated. With engineers hand-writing a system prompt per task, it succeeded 70–80% of the time; without that per-task prompt engineering, success fell to 10–20%. That result pointed at architecture rather than prompting.

Two layers: deterministic routing, isolated specialists

The redesign produces a 1,300-line orchestrator that routes more than 20 workflows to 14 specialized agents and performs no domain work of its own. The layers split cleanly:

  • The orchestrator classifies intent, collects required fields, runs reliability checks and delegates. It never reasons about code, generates configurations or touches the codebase.
  • Fourteen subagents each hold isolated context and deep expertise in one area. A schema management agent knows the configuration registries, a population agent knows code generation for each of the 10 data sources, a validation agent knows build systems and quality checks. None of them sees the whole picture.

Every request runs through the same seven steps:

  1. Read the relevant subagent's instruction file.
  2. Check whether the target already exists, to prevent duplicates.
  3. Gather required fields from the user.
  4. Auto-discover what can be inferred, such as repository type or existing configurations.
  5. Run the reliability gate: active warnings or required reviewers.
  6. Delegate to the subagent with full context.
  7. Handle results, surface errors and confirm success.

Only step 6 involves LLM reasoning. Everything else is deterministic.

Workflow templates carry the sequencing

The key separation is between what must happen and how each step is executed. Workflow templates — deterministic state machines that bound the LLM's decision space — fix the phase order:

  • Column Onboarding: Schema → Validation → Population → Testing
  • Event Onboarding: Schema → Population → Validation → Testing → Event Processing
  • Pipeline Migration: Assessment → Planning → Population → Validation → Monitoring

Templates draw from the same 14 subagents in different orders. When pipeline migration arrived as an entirely new use case, it required no new agent code — just a template resequencing existing agents. Production capability, zero greenfield engineering.

Each subagent follows an identical shape: the domain knowledge files to load at runtime (usually a few thousand tokens each), a pre-gathering checklist for the orchestrator, a step-by-step execution plan, discovery patterns for locating code in the monorepo, and error-to-resolution mappings for common failures. Because the structure is uniform, a new subagent is templated work rather than new architecture.

Knowledge is not baked into the orchestrator. More than 25 knowledge files, organized by workflow domain, load on demand when a subagent executes. Editing a file changes behavior from the next session onward with no orchestrator changes, and the same files back multiple platform implementations of the agent.

Measured impact

Preparing configuration for a new column once took three to four days by hand, and about a day under an earlier config-driven system. The thin orchestrator completes it in under an hour. Roughly two-thirds of the system's changes land without significant human edits — the payoff for trading generality for deep domain knowledge, with a general orchestrator over specialist subagents.

Why keeping the router thin pays off

Heavyweight orchestration — making the central component the smartest one — holds until scale, complex multi-step workflows and heterogeneous subsystems expose it. The thin approach works because architectural boundaries match computational ones:

  • Deterministic stays deterministic. Selecting the right subagent is a routing decision, not a reasoning task, and doesn't belong to an LLM.
  • Isolation blocks contamination. The population agent for source A never sees source B's patterns, so it cannot hallucinate them.
  • Failure stays contained. A failed subagent gets retried or escalated by the orchestrator; its failure doesn't corrupt orchestrator state or other agents' contexts.
  • Every layer improves independently. A stronger model helps all 14 subagents at once, a knowledge file edit applies immediately, a new template unlocks a capability without touching an agent.

The failure mode of the alternative is the strongest argument for the pattern. A heavyweight orchestrator that hallucinates spreads the error downstream — wrong agent, wrong fields, skipped reliability checks. A thin one follows a fixed protocol and leaves little surface for hallucination; the creativity stays downstream, scoped to a domain, where risk is lower and failures don't escape.

Transferring the pattern

  1. Separate routing from reasoning. Routing should be deterministic — keyword matching, intent classification, field gathering — with all domain reasoning pushed into isolated subagents.
  2. Make specialists structurally uniform. Give every subagent the same skeleton — knowledge sources, required inputs, execution steps, error handling — so new capabilities are templated additions.
  3. Scale through workflow templates, not new agents. Ask first whether existing agents can be resequenced. Agents you can't reuse across workflows are too coupled to one use case.
  4. Load knowledge at runtime. Knowledge embedded in the orchestrator forces redeployment on every update; on-demand loading takes effect immediately across platforms.
  5. Draw the deterministic/probabilistic line explicitly. For each component, decide whether it makes a routing decision or performs reasoning, and keep it on one side.

What comes next

The architecture is being scaled to handle more data columns across new pipeline domains. The compositional thesis says scaling means adding workflow templates and knowledge files, not rebuilding agents. Meta is also exploring automated feedback loops that identify failure patterns from production usage and propose targeted updates to subagent knowledge, with human review gating every change.

The lesson worth carrying: the most important component in the system does the least work.