When better tools made code review worse

Swap in better tools and you expect better results. That was the assumption when GitHub's Copilot code review team migrated from its own exploration tools to the shared Unix-inspired utilities—grep, glob, and view—that power the Copilot CLI and cloud agent. The goal was sensible: consolidate tool implementations into one maintained harness that multiple products could share.

The benchmarks told a different story. Review costs went up and fewer real issues were caught. The tools weren't at fault. The instructions were.

After rewriting the tool guidance to match how a reviewer actually works, the team saw roughly 20% lower average review cost with no quality regression compared with the control. The fix wasn't in the tools themselves but in the workflow encoded around them.

Why a simple swap failed

The original Copilot code review tools were designed for earlier agentic models. Those models made fewer tool calls and struggled to pull in context autonomously, so the tools returned matched lines plus extra surrounding code. That padding added tokens, but it compensated for model limitations at the time.

The Copilot CLI harness, by contrast, was built for interactive coding tasks where exploration is part of the job—mapping a codebase, planning a change, editing across multiple turns. Its tools return precise results without the extra context, which is fine when a developer is driving the session.

Review is a different job. It starts from a diff, asks targeted questions about whether the change introduces a real problem, and should gather only the narrowest evidence needed to answer. The generic CLI-style instructions gave the agent the wrong instincts for that task.

Trace analysis exposed a browsing loop

Internal benchmarks were key because they show the agent's full path—which tools it called, what output came back, where errors occurred, and whether it was narrowing toward evidence or expanding the search.

With the shared tools, the agent behaved like it was exploring a repository rather than investigating a pull request. It would search broadly, guess likely paths, read whatever turned up, discover new search targets, and carry all that extra context forward. Each tool result stayed in the context window, inflating cost and diluting focus. A tool result isn't a disposable printout for an agent; it's persistent context that shapes subsequent reasoning.

A coding assistant might map a whole area before editing to avoid breaking something. A reviewer doesn't. A reviewer starts from the diff and asks: Where is this function called? Is this config key used elsewhere? Is there a test with the same pattern? The smallest relevant code range that explains the behavior is the goal, not a broad tour of the repository.

The tools worked as designed. The instructions implied they should be used like a broad coding assistant, not a reviewer.

Rewriting guidance around a reviewer's workflow

The revision made the instructions review-shaped:

  1. Start from the diff and form specific review questions.
  2. Use glob when paths are uncertain and grep to locate candidates, symbols, and call sites.
  3. Batch cheap discovery before reading files.
  4. Use view only once the agent knows which file or range it needs.
  5. Batch focused reads instead of alternating search and read.

Generic posture told the agent to "inspect repository context that may be relevant." The review-shaped guidance said: narrow first with grep and glob, read exact evidence with view, and recover from failures with a single simpler correction rather than expanding the search.

If a diff changes an authorization helper, the right question isn't "show me every file that calls this helper." It's "are any request-handling callers relying on the old behavior?" That distinction keeps the investigation short and focused.

The wording change was small but shifted the agent's rhythm from "browse, read, search again" to "ask, narrow, read, decide."

Errors as a tuning signal

The revised instructions also changed failure recovery. A failed grep now led to one simpler, corrected search. A wrong path led to glob, not guessing neighboring paths and hoping something useful existed there. Small tool failures no longer cascaded into wider exploration loops.

Benchmarks let the team debug behavior rather than chase scores. Running the same examples, comparing tool traces, updating instructions, and rerunning made it possible to answer concrete questions: Did the agent narrow before reading? Did it batch independent searches? Did view calls have a clear reason? Did tool errors move elsewhere or actually drop? The most telling signal was that tool-call counts stayed similar, but more calls targeted relevant evidence instead of repeatedly widening the search.

Instructions are product surface

This case is a reminder that for agents, tools are not swappable implementation details. Tool descriptions and system instructions function like API documentation. Unclear docs lead developers to inefficient or wrong choices; unclear tool prompts do the same to a model.

The lesson holds a counterexample. Applying the same review-focused tool instructions to the Copilot CLI did not produce a similar win. That product handles broader, interactive tasks where a user may change direction mid-session and the right context isn't known upfront. The same tools can support both products, but only when the instructions match the job. Shared infrastructure scales when the workflow around it is tuned per use case, not when it's treated as one-size-fits-all.