Why sensor feedback matters for maintainability

Maintainability problems tend to surface in predictable ways when coding agents are doing the work. The first sign is usually a small change that touches an outsized number of files. Another is a modification that breaks something that was previously stable. Agents working in tangled codebases also suffer in ways familiar from human developers: they look in the wrong place for existing implementation, duplicate code because they miss an earlier version, or pull in more context than the task justifies.

The experiment described here treats those warning signs as measurable signals. The setup is an internal analytics dashboard for community managers, built in TypeScript with NextJS and React, that reads chat activity, engagement, and demographic data from several external APIs. The application was rebuilt from scratch with AI assistance, using Cursor, Claude Code, and OpenCode, with models varying by task type. Markdown guides for the agent were deliberately kept sparse to test how much maintainability could be maintained through sensor feedback alone.

Sensor placement across the delivery path

Sensors ran at different points, depending on what kind of drift they were meant to catch.

During the coding session, computational checks ran continuously: the type checker, ESLint, Semgrep (the SAST tool prescribed by the internal AppSec team), dependency-cruiser for structural module rules, test results and coverage, incremental mutation testing, and GitLeaks in the pre-commit hook.

In the pipeline, the same computational sensors ran again in CI, confirming results on clean infrastructure after integration.

On a slower cadence, additional reviews detected accumulated drift rather than moment-to-moment errors. These included a security review based on an AppSec checklist prompt, a data handling review targeting rules like "no user names should ever reach the web frontend," a dependency freshness report that scripted age and activity checks before having AI produce upgrade recommendations, and a modularity and coupling review.

Linting rules tuned for agent failure modes

Linters like ESLint mostly address maintainability risk at the level of individual files and functions. But the obvious rules for catching common AI shortcomings were not active in the default preset. Maximums needed to be configured for the number of function arguments, file length, function length, and cyclomatic complexity.

Work is emerging elsewhere on rule sets aimed specifically at agent failure modes, such as an ESLint plugin by Factory with rules around requiring test files or structured logging. But for this experiment, the more interesting work was customizing the guidance agents receive when a rule fires.

For self-correction to work meaningfully, the agent needs context, not just a violation message. A custom ESLint formatter was built to override default rule messages with guidance about how to handle the situation. For the no-explicit-any warning, for example, the agent was given the judgment that a warning may be suppressed when the use is genuinely justified, but that the suppression should be reported visibly so a human can review it later.

We want things to be typed to make it easier to avoid errors, especially for key concepts.
But we also want to avoid cluttering our codebase with unnecessary types. Make a judgment
call about this. If you choose to not introduce a type, suppress it with:
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- (give reason why)`,

This approach changes the economics of lint warnings. Historically, achieving a clean baseline with tools like ESLint has meant choosing between fixing everything or suppressing noise one line at a time, both of which feel burdensome. With agents, the balance shifts: the tool message can instruct the agent to make a judgment call, suppress when appropriate, and keep suppressions reviewable.

Thresholds received similar treatment rather than point-in-time suppressions. Agents were told they could raise a line count or complexity limit slightly when a refactoring was genuinely unnecessary or impossible, rather than having to suppress or comply. The threshold stays elevated, so the rule fires again if the metric worsens further. Constraints are preserved without forcing a binary outcome.

What the linting observations showed

  • Reviewing the exceptions the agent created — warnings suppressed, thresholds increased — provided a useful entry point for human code review.
  • Agents frequently raised the cyclomatic complexity threshold, but suggested good refactorings when prodded further. It was the only rule where this happened, and the likely reason was missing self-correction guidance for that rule: no explicit instruction was present to treat threshold increases as the exception rather than the norm.
  • Different code regions can want different treatment of the same rule. For no-console, the backend should use a logger component, while the frontend has different logging requirements. Self-correction guidance is where that nuance can live.
  • Only one rule conflict surfaced so far. max-lines and max-lines-per-function prompted useful component breakdowns, but in the React frontend this led to chains of smaller components with many props passed down. Whether agents can balance such tradeoffs consistently is still an open question.

Cost-benefit shift in static analysis

The cost-benefit balance that traditionally made linting inconsistent is shifting on both sides. Creating custom rules and scripts is much cheaper with AI, so the upfront cost drops. The benefit side grows because agents produce habits a human developer might not, and this tooling surfaces hygiene factors early.

There is a caveat: linters also risk giving a false sense of security. They cannot catch the more semantic aspects of quality, and the remedy remains unclear. Activating new rule sets always produced a mix of irrelevant findings and important ones. The concern is feedback overload: too many signals could send an agent into a spiral of unnecessary refactorings, and that risk needs to be managed alongside the benefits.

Dependency rules for module boundaries

Code organization across files and modules is a separate concern from in-file complexity, and tools that police those boundaries have historically seen even less use than basic linting. The experiment covered three angles: deterministic dependency rules via dependency-cruiser, coupling analysis that combined deterministic and inferential methods, and a purely inferential modularity review.

A layered module structure was defined with the agent midway through implementation, and rules were written in dependency-cruiser to enforce those layers. One rule, for example, forbids code in clients from importing anything from services:

{
  name: “clients-no-services”,
  comment:
    “API clients must not depend on the orchestration layer above them. “ + LAYERS,
  severity: “error”,
  from: { path: “^server/clients/”, pathNot: “/__tests__/” },
  to: { path: “^server/services/” },
},

As with ESLint, the error messages were expanded to serve as self-correction guidance, recapping the layering concept rather than just stating the violation:

ERROR  clients-no-services
  API clients must not depend on the orchestration layer above them. 
  [Layers: routes -> services -> clients + domain; Services orchestrate: fetch data via clients, compute via domain -- no I/O, no SDKs, no knowledge of data fetching.]

Rules were applied to other structural conventions as well, such as the structure of React hooks in the frontend. A further rule required every new file to land somewhere in the predefined folder structure, to catch the agent creating new directories outside the agreed layout.

What the dependency rule observations showed

  • The configuration syntax for dependency-cruiser has a steep learning curve, and the agent absorbed most of that cost. Without AI, getting these rules in place would have taken considerably longer.
  • After rule introduction, the agent violated them a handful of times and then corrected course on the basis of the dependency-cruiser feedback. The rules helped clean up already haphazard folder organization and kept new code aligned.
  • Static structural rules are a useful complement or alternative to describing code structure in a markdown guide, with the limitation that they can only express what is visible through imports, file names, and folder paths.

Coupling metrics through custom tooling

To avoid the limitations of off-the-shelf static analysis, I had a coding agent build a custom analysis tool on top of the TypeScript compiler. The tool computes per-file coupling metrics and exposes them through two interfaces: a web UI for human review and a CLI named coupling-analyser designed for consumption by coding agents.

For my own use, the web visualizations drew on established concepts like dependency structure matrices (DSM). Even so, I found them tedious to work through. The data is granular and demands considerable context to interpret and translate back into high-level design principles, which suggests such visualizations won't meaningfully reduce a human's cognitive load during review of AI-modified code.

For the agent-facing path, I gave the model the CLI and a prompt requiring it to produce a modularity report grounded strictly in the CLI's output. The instructions were light on any definition of good or bad design, delegating interpretation to the LLM: "Produce a markdown report on modularity and coupling quality... grounded in actual CLI output from npx coupling-analyser, not guesswork from static browsing alone."

With the custom CLI at its disposal, the model (Claude Opus 4.7) was told to run the report subcommands and to structure its findings around module IDs, numerical evidence, and future change risk. It successfully pointed out the same hotspots I'd identify by inspecting the diagrams myself, in a more digestible format. Grounding the LLM in deterministic tool output also raised my confidence in the result — and likely spent less time and tokens than letting the agent scan the codebase for coupling patterns directly.

Where the coupling analysis falls short

The findings themselves were lackluster. The LLM flagged a factory that initializes all components as a top issue — despite the factory being an intentional, lightweight dependency injection layer. It declared a shared zod schema between frontend and backend a "god module," though this explicit contract pattern is reasonable when both sides live in the same repository and evolve together.

Two patterns stand out from the experiment:

  • Legitimate high-coupling hubs need a suppression mechanism, or they generate persistent noise in future analyses.
  • One genuinely useful catch: an index.ts in the domain folder indiscriminately re-exports every file under ./domain, and it is imported widely. Though a barrel export can be a deliberate layer contract, it merits investigation in this codebase.

The core lesson: what counts as good or bad coupling depends on design context, not just the raw import graph. On its own, this metric layer lacks the contextual depth to be useful to an AI. A more viable role is risk triage during code review — knowing that a changed file has ten or more callers could signal an AI reviewer to spend more tokens there, or a human to pay closer attention.

LLM-led modularity review

The mediocre coupling results could stem from an underspecified prompt, useless data, or shallow data lacking code context. To test the last possibility, I went fully inferential with Vlad Khononov's "Modularity Skills". This proved far more fruitful, surfacing several valid refactoring leads that would meaningfully reduce future-change risk. Running the analysis a second time (without the first run's context) uncovered yet another distinct issue, and when the second run had CLI access, the tool mostly confirmed the findings rather than contributing new ones.

Concrete failure modes identified in this pass:

  • Duplicate backend route code — All three endpoints each carried their own near-identical route file. Any cross-cutting API change (request IDs, error handling, logging) would require edits in three places; the model noted that agents tend to copy-paste on repetition three or four times rather than refactor unprompted.
  • Semantic duplication in backend calls — Two pages used a shared hook for data fetching, but the third page deviated with its own reimplementation. This produces divergent error handling and duplicated change surfaces.
  • Repeated core argument passing — Out of several pages, all pass a chat space ID and date range individually to the backend. A change to the date-range mechanism previously required edits to more than 40 files. The review echoed my suspicion: request parameters are repeated at every level, and a wrapping object existed but was never applied consistently — an "inconsistent mess."
  • Misplaced responsibility — Authentication fallback logic (mock data for unauthenticated users) had ended up inside the factory responsible for wiring modules, risking oversight as new routes are added.

On the high-import-count hubs previously flagged as "god classes," the modularity review noticed both but correctly identified each hub as justified in context — likely a result of the review actually reading code semantics rather than relying exclusively on data-only coupling metrics.

What holds up for monitoring

  • Tools like dependency-cruiser can effectively enforce basic dependency direction rules as live sensors, but their utility tops out there.
  • LLM-based modularity review worked well when paired with strong prompts, serving as effective "garbage collection" for a codebase. Grounding it in coupling data made little marginal difference.
  • Running this review after most of the code had already built revealed concerning, valid findings — evidence that absent a capable human review and such analyses, agents were compounding inadvertent technical debt.

For codebase design and modularity overall, deterministic sensors alone fall short. AI is required to interpet semantic meaning, evaluate trade-offs, and separate intentional design patterns from genuine violations.

Regression detection: what the test suite can tell us

Tests serve multiple roles in a codebase: they help shape design, document intended behavior, and act as a safety net against regressions. When a pre-existing test fails, the developer (or the AI agent) must determine whether the failure signals an accidental break or an intentional spec change. A strong test suite makes that judgment call much safer for AI-driven development.

In the chat analytics application, the agent wrote the entire test suite over time, with oversight limited to manual testing and coverage checks. The goal was to examine, in hindsight, how effective an AI-generated test suite really is at catching regressions.

Two key risks emerge when AI generates tests without human review:

  • Coverage is not a reliable proxy for test effectiveness. High coverage can coexist with weak assertions.
  • Tests may validate faulty behavior. This article focuses on test effectiveness, assuming the code under test behaves correctly and asking whether the tests would catch breaking changes.

Available tools for measuring test quality

  • Coverage ($) — shows which lines and branches tests execute, but not what they verify.
  • Property-based testing ($) — generates many input combinations from defined properties to expose missing logical cases.
  • Fuzz testing ($$) — throws malformed or unexpected inputs at the system to find robustness gaps.
  • Mutation testing ($$) — makes small deliberate code changes and checks whether tests catch them.

In this project, coverage and mutation testing were the two techniques applied; property-based and fuzz testing were less suited to the application's needs.

Mutation testing in practice

The mappers.ts file illustrated the limits of coverage-based thinking. It showed 100% statement coverage and 75% branch coverage, yet had no dedicated unit tests. Stryker, the mutation testing tool used, reported 13 survivors — meaning 13 introduced mutations went undetected by the entire test suite. Coverage was inflated because a large acceptance test happened to execute these mapper functions without ever asserting on their individual output. A future change to dvpToSchema, for instance, could break the data graph display in the UI while keeping all tests green.

Key observations

  • AI analysis is strong at identifying mutation hot spots and producing a prioritized roadmap for improving test quality.
  • Stryker generates an enormous JSON results file; a custom script was written to let the agent query these results efficiently without overflowing the context window — a good example of augmenting AI with tooling.
"""Query a Stryker mutation-testing JSON report from the command line.

Usage:
python query_stryker.py <report.json>; <command> [options]  

Commands:
   summary Overall status totals, mutation scores, thresholds.
   files Per-file breakdown, default sorted by mutation score asc.
   hotspots Lines with the most survivors / no-coverage mutants.
   tests Test effectiveness: weak, unused, or top-killer tests.

Examples

# 1. Overall health — mutation score, status breakdown, threshold pass/fail
python ./query_stryker.py reports/mutation/mutation.json summary

# 2. Worst files first, with an action hint (strengthen assertions vs add tests)
python ./query_stryker.py reports/mutation/mutation.json files --top 10 -v

# 3. Same, but only for files you've changed in git (auto-detects the repo)
python ./query_stryker.py reports/mutation/mutation.json files --changed -v

# 4. Zoom into one file: every (line, actionable counts, sample mutators)
python ./query_stryker.py reports/mutation/mutation.json hotspots --file server/services/ai-summaries.ts --top 30

"""

Broader implications

There is a clear trend toward more end-to-end acceptance tests, especially now that AI generates tests so fluently. Reviewing large volumes of AI-written unit tests is tedious, and expecting sustained human attention for all of it is unrealistic. Techniques like approved scenarios are gaining ground as a response. The caveat is that acceptance tests often boost coverage without adding granular assertions, giving a false sense of security. Mutation testing provides a way to monitor exactly that blind spot.

The practical constraint is cost: running mutation tests continuously is resource-intensive. In this project, they were run manually in incremental batches rather than on every change.

Closing observations

Computational sensors performed best when targeted at the file and function level. Cross-file concerns such as modularity and coupling produced noisy raw data that only became useful once interpreted semantically by an LLM — at which point the accuracy and utility depended heavily on the quality of the prompt and the model's ability to match findings to different audiences.

Interactions between sensors may become a growing concern. The tension between max-lines and max-lines-per-function appeared across refactorings: breaking functions into smaller pieces shifted complexity into chain-heavy component properties. It is plausible that other, subtler conflicts between sensor rules will surface as the tooling matures.

The question of how guides and sensors coexist remains open. In this project, guides were intentionally omitted to isolate the sensors' effect. This raises follow-up questions about how the balance will settle in large-scale AI-assisted development: which guides become redundant once the right set of sensors is in place, whether sensors enable lighter models with less comprehensive reasoning, and how to bundle and maintain cohesive guide-sensor pairs.

The regulatory role of mutation testing becomes significantly more important when the majority of test authorship is delegated to an AI. It cannot, nonetheless, address the separate question of whether tests assert behavior that is in fact correct.

Sensors, whether computational pattern checks or LLM-based inferential analysis, are not a substitute for human oversight — but this workflow did noticeably improve the review experience and professional trust in the resulting code quality.

Acknowledgements

Thanks to Chris Ford and Matteo Vaccari for the conversations about sensors, and the Thoughtworks colleagues who offered feedback and ideas. GenAI was used for research and polishing the language.

A Practical Sidecar for Sensor Data

To actually feed this sensor data to an agent, I built a small “sidecar” CLI application that runs all the computational sensors described in these experiments. It reads a config file defining each sensor command and runs them continuously at intervals—similar in spirit to running test suites in watch mode while coding.

The CLI serves two purposes: it provides situational awareness for the human during supervised sessions, and it produces a token-efficient, guidance-enriched summary for the agent. Each check generates both a visual status for the user and a compact report that the agent can pull up on demand.

Key elements in the agent-facing summary include:

  • A configurable “scouting rule” at the top—a global guidance prompt I can inject from the sensors config file.
  • Trend indicators like “Worse than / Same as snapshot,” based on a persisted baseline the CLI captures.
  • Directional hints—e.g., “higher / lower is better”—though for obvious metrics like test failures or coverage, these are often redundant given the model's training data.
  • Target thresholds for metrics where “good” isn't simply 0 or 100, shown as “(below target threshold of...)” annotations.

This gives us the chance to shape the agent's self-correction far better than using the underlying tools directly. The summary could even be trimmed further to report only failures, like an “evergreen” CI monitor that stays quiet when everything is fine.

Getting the Agent to Check In

There's no single reliable way to make the agent query the sensors regularly. I tried several integration points:

  • Via a guide: A skill or an AGENTS.md section asking the agent to check regularly. This is the least effort and what I mostly used, but it proved unreliable—agents frequently skipped checks or called tests and linters directly instead.
  • Via hooks: Forcing checks through harness hooks after each file edit is plausible, though too much distraction could hinder the agent's flow.
  • Via git hooks: A pre-commit hook works well for agents that commit often, ensuring sensors run before any code lands in a commit.
  • Custom extension: Building a dedicated tool into the harness (I tested this with Pi) is promising. It can prompt the user at startup to begin a session and show sensor state in the status line, but I haven't used it long enough to confirm it triggers more reliably than a markdown instruction.

Adding a New Sensor

Initially, every new sensor meant writing a parser to extract results from tool output, such as reading coverage-final.json or parsing semgrep's output. I created a skill to make this quicker, but it still required code changes to the CLI for each addition.

I eventually shifted to a default parser expecting a unified JSON schema from each sensor. Instead of the CLI running ruff directly, it executes python ./ruff_sensor.py -- check sensors/, where the wrapper script runs the real tool and converts its output into the schema the CLI can render. This approach suggests how customization might shift: rather than building a plugin system for every conceivable sensor type, some tools can stay simple and assume users will write small adapters themselves.

Performance and Effectiveness

Running more sensors costs more resources, and the performance gap between build tools across stacks is significant. My Python setup caused no issues—most of those tools are Rust-based and fast. For TypeScript codebases, the sensors consistently spun up my M3's fan, though it wasn't purely JavaScript's fault: my virus scanner was also reacting to the sensors constantly rewriting files.

Measuring whether the sensor setup actually helps is similar to asking how effective a CD pipeline is. A perpetually green pipeline catches nothing; a perpetually red one is too strict or flaky. To get to an answer, I logged a history of sensor states on every check. This data can reveal:

  • Whether sensors are failing less often over time—a signal of improving guides or models.
  • Which sensors never fail—candidates for removal.
  • Which rules trip frequently—places where better guidance could pay off.

Where It Falls Short

I found the setup valuable both as my own “head-up display” in supervised sessions and as a source of information for the agent. Two gaps remain before it's more broadly useful:

  • Making agents reliably invoke the CLI in unsupervised sessions needs deeper harness integration—simply telling the agent to check isn't dependable.
  • I haven't made the CLI sandbox-ready. For an agent operating in a constrained environment, the tool should be available via a registry or similar packaging so that sandboxing isn't an extra hurdle developers skip.

You can find the Sensors CLI source code on GitHub.