Harness engineering for coding agent users

A trust problem at the token level

Software engineers have a natural trust barrier with AI-generated code. Language models are non-deterministic, they lack our project's context, and they operate on tokens rather than genuine code understanding. To let coding agents work with less supervision, we need a systematic way to increase confidence in their output. Emerging work in context and harness engineering offers a practical mental model for building that trust.

The term harness has become shorthand for everything in an AI agent except the model itself — often expressed as Agent = Model + Harness. In the bounded context of coding agents, part of the harness is built in (system prompts, code retrieval mechanisms, orchestration systems), but agents also expose features that let users build an outer harness tailored to their specific codebase and workflow. A well-built outer harness has two goals: increasing the probability that the agent gets it right on the first attempt, and providing a feedback loop that self-corrects issues before they reach human eyes. Done well, it reduces review toil and improves system quality — with the added benefit of fewer wasted tokens.

Guides and sensors

Harnessing a coding agent means both anticipating unwanted outputs and putting mechanisms in place for self-correction. These controls fall into two categories:

• Guides (feedforward controls) — anticipate the agent's behaviour and steer it before it acts. They increase the probability of good results on the first attempt.
• Sensors (feedback controls) — observe after the agent acts and help it self-correct. They are particularly powerful when their output is optimised for LLM consumption, such as custom linter messages that include instructions for fixing the problem — a positive form of prompt injection.

Using only one type creates predictable failure: feedback-only agents repeat the same mistakes; feedforward-only agents encode rules but never learn whether they worked.

Computational vs inferential controls

Guides and sensors also differ in how they execute:

• Computational — deterministic and fast, run by the CPU. Tests, linters, type checkers and structural analysis complete in milliseconds to seconds with reliable results.
• Inferential — semantic analysis, AI code review, and LLM-as-judge approaches. These run on GPU or NPU, are slower and costlier, and produce more non-deterministic results.

Computational controls are cheap enough to run on every change alongside the agent. Inferential controls are more expensive, but they enable rich guidance and semantic judgment that computational tooling cannot provide. Despite their non-determinism, inferential sensors can significantly increase trust when paired with a strong model suited to the task.

Examples

The steering loop

The human role in this system is to steer the agent by iterating on its harness. Whenever an issue occurs more than once, the feedforward and feedback controls should be improved to make that issue less probable — or prevent it entirely.

AI itself can help build a better harness. Modern coding agents make it cheaper to create custom controls and static analysis. Agents can help write structural tests, generate draft rules from observed patterns, scaffold custom linters, or produce how-to guides from codebase archaeology.

Quality left, drift monitored

Teams that continuously integrate already face the challenge of spreading tests, checks and human reviews across the development timeline based on cost, speed and criticality. Aspiring to continuous delivery means ideally every commit state should be deployable. The earlier an issue is found, the cheaper it is to fix, so checks belong as far left in the path to production as possible. The newer inferential feedback sensors need to be distributed across the lifecycle on the same principle.

This creates two distinct questions about where controls belong:

• Pre-integration: What is fast enough to run before a commit is created? Linters, quick test suites and basic code review agents fit here.
• Post-integration: What is too expensive for the commit loop but valuable in the pipeline? Mutation testing and broad code review that considers the bigger picture repeat the fast controls and add depth.

Beyond the change lifecycle, another class of controls addresses gradual degradation:

• Continuous drift sensors: Monitors running against the codebase outside of change cycles — dead code detection, test coverage quality analysis, dependency scanners.
• Runtime feedback agents: Systems that watch for degrading SLOs and suggest improvements, or AI judges that continuously sample response quality and flag log anomalies.

What the harness regulates

An agent harness functions as a cybernetic governor, using feedforward and feedback controls to regulate the codebase toward a desired state. That desired state has multiple dimensions, and distinguishing between them matters because harnessability and complexity vary significantly across categories. Three categories are currently useful.

Maintainability harness

Regulating internal code quality is the easiest harness type today, since extensive pre-existing tooling is available. Computational sensors reliably catch structural issues: duplicate code, cyclomatic complexity, missing test coverage, architectural drift and style violations. These are cheap, proven and deterministic.

LLMs can partially address problems that require semantic judgment — semantically duplicate code, redundant tests, brute-force fixes and over-engineered solutions — but at higher cost, with probabilistic results. That makes them unsuitable for every commit.

Neither approach reliably catches the higher-impact problems: misdiagnosis of issues, overengineering, unnecessary features and misunderstood instructions. They may occasionally catch these, but not reliably enough to reduce supervision meaningfully. If the human did not define correctness clearly in the first place, no sensor can fill that gap.

Architecture fitness harness

This category groups the guides and sensors that define and verify architectural characteristics — essentially fitness functions applied to agent work. Practical examples include:

• Skills that feed forward performance requirements, paired with performance tests that give the agent feedback on whether its changes improved or degraded them.
• Skills describing logging standards and other observability conventions, combined with debugging instructions that ask the agent to reflect on the quality of the logs it had available.

Behaviour harness

The hardest problem is guiding and verifying functional behaviour. Current practice among teams granting high autonomy to coding agents generally looks like this:

• Feed-forward: a functional specification at whatever level of detail the team chooses, from a short prompt to multi-file descriptions.
• Feedback: verifying the agent-generated test suite runs green, has reasonably high coverage, potentially monitoring its quality with mutation testing — then supplementing with manual testing.

This approach places considerable faith in AI-generated tests, which is not yet justified. The approved fixtures pattern has shown promising results for some teams, but it works better in some contexts than others and is used selectively rather than as a general solution. Designing harnesses for functional behaviour that meaningfully reduce supervision remains open work.

Harnessability varies

Not every codebase is equally amenable to being harnessed. A strongly typed language provides type-checking as a natural sensor. Clearly defined module boundaries make architectural constraint rules possible. Frameworks that abstract away incidental complexity implicitly raise the agent's chances of success. Without these properties, the corresponding controls are simply unavailable.

Greenfield teams can design for harnessability from day one, since technology and architecture choices determine how governable the codebase will be. Legacy teams face a harder problem: technical debt makes the harness most necessary where it is most difficult to build.

Templates for common topologies

Most enterprises have a few service topologies covering most of their needs: business services exposing data via APIs, event processing services, data dashboards. Mature engineering organisations often codify these in service templates. Those templates may evolve into harness templates: a bundle of guides and sensors that leash a coding agent to a topology's structure, conventions and tech stack. Teams might begin selecting tech stacks partly based on what harnesses already exist.

This would inherit the familiar challenges of service templates: the moment teams instantiate a template, they start drifting out of sync with upstream improvements. Harness templates would face the same versioning and contribution problems — potentially worse, since non-deterministic guides and sensors are harder to test.

The irreplaceable human layer

Human developers bring an implicit harness to every codebase: absorbed conventions, an innate sense of complexity, accountability for what goes into a commit, and awareness of organisational alignment — which technical debt is tolerated, what the team is prioritising and what "good" means in this specific context. Human pace itself matters; working in small steps creates space for experience to trigger and apply.

A coding agent has none of this. No social accountability. No aesthetic objection to a 300-line function. No instinct for "we don't do it that way here." No organisational memory for which conventions are load-bearing versus mere habit. Harnesses are an attempt to externalise human developer experience and make it explicit — but that effort has limits. Building a coherent system of guides, sensors and self-correction loops is expensive, so prioritisation must be guided by a clear objective: a good harness should not aim to eliminate human input, but to direct it toward where human judgment matters most.

From Patterns to Practice

The framing above reflects what teams are already doing in production, and it gives us a vocabulary for the harder problems ahead. Instead of debating individual features like skills or MCP servers, we can talk about how a system of controls should be designed to make agent output trustworthy.

Recent engineering write-ups illustrate this shift:

• OpenAI's team describes its harness as a layered architecture enforced by custom linters and structural tests, with recurring "garbage collection" scans that detect drift and have agents propose fixes. Their conclusion is telling: the hardest problems now are designing environments, feedback loops, and control systems.
• Stripe's work on "minions" emphasizes pre-push hooks that run relevant linters based on a heuristic. Their focus on shifting feedback left, combined with their "blueprints," shows how feedback sensors are being integrated directly into agent workflows.
• Mutation and structural testing, both computational feedback sensors that were historically underused, are seeing renewed interest.
• Developers are increasingly discussing how coding agents integrate LSPs and code intelligence as computational feedforward guides.
• At Thoughtworks, teams are tackling architecture drift with a mix of sensors: improving API quality through agents plus custom linters, and raising code quality with what is colloquially called a "janitor army."

What a Harness Still Needs to Solve

The open questions go beyond the behavioral harness discussed earlier. Keeping a harness coherent as it grows is non-trivial: guides and sensors must stay synchronized so they do not contradict each other. Even when they agree, we need to know how much discretion an agent should have when instructions and feedback point in different directions. A sensor that never fires could mean quality is genuinely high, or it could mean the detection mechanism itself is too weak to notice problems.

That last point is why harness coverage and quality need their own evaluation tools, similar to what code coverage and mutation testing provide for test suites. Right now, feedforward and feedback controls are scattered across delivery steps with no easy way to reason about them as a whole. Tooling that helps configure, sync, and analyze these controls as a unified system is an untapped opportunity. Building the outer harness is becoming a continuous engineering discipline rather than a one-time setup exercise.