Smarter token use in long Copilot sessions

As GitHub Copilot takes on more agentic work—planning, editing, debugging, reviewing, and tool orchestration across long sessions—efficiency is less about raw token counts and more about how tokens are spent. The GitHub team is working on two fronts: reducing the repetitive context the harness sends to the model each turn, and expanding automatic model selection so developers don't have to manually tune which model handles which task.

Tightening the harness in VS Code

In long VS Code sessions, the Copilot harness assembles a great deal of recurring information for the model: instructions, repository context, conversation history, tool definitions, and task state. Not all of it is needed at every step. Two changes in GitHub Copilot for VS Code address this overhead directly:

  • Prompt caching lets Copilot reuse model state for repeated prompt prefixes instead of recomputing the same prefix on every request.
  • Tool search lets the model load tool definitions on demand, rather than sending every full tool schema into context on every turn.

This matters more as agents gain access to more tools—MCP tools, terminal commands, file operations, workspace search, and product-specific actions all carry their own schema. Sending all of them up front adds fixed cost to every turn, even when few are actually relevant. Tool search keeps the available toolset broad while reducing the schema sent to the model.

For implementation details on prompt caching, cache-control breakpoints, provider-specific tool search, and behavior across long-running agentic sessions, refer to the VS Code technical deep dive.

Why Auto model selection exists

Auto answers a practical question: which model is best for this task right now? After the first prompt, Copilot uses task intent and current model health to pick a model. A quick explanation, a focused edit, and a complex multi-file change do not all benefit from the same level of reasoning, and Auto makes that call without requiring manual model settings.

GitHub's evaluations found that no single model consistently performed best across tasks. A more efficient model often reached the same outcome, while stronger models mattered most when the task required deeper reasoning. Auto is designed to route up when the task demands it and stay more efficient when it does not—not as a trade of quality for cost, but as a fit between model capability and work difficulty.

Two signals drive routing

Auto combines real-time model health with task-aware routing:

  • Real-time model health is tracked by a dynamic engine that monitors availability, utilization, speed, error rates, and cost. A model can be capable of a task yet not be the best choice at a given moment, so routing takes current system conditions into account.
  • Task-aware routing with HyDRA considers reasoning depth, code complexity, debugging difficulty, and tool orchestration needs. HyDRA identifies models that meet the quality bar for the task, then selects the best fit among them.

The goal is not to push every task to the largest model or every task to the cheapest one, but to pick the model that fits the work.

Cache-aware routing

Switching models every turn can be counterproductive. When a conversation stays on the same model, the prompt prefix is cached and reused across turns. Mid-conversation model changes break that cache, which can cost more than the routing change saves. Auto routes at natural cache boundaries: on the first turn when there is no cache to lose, and after compaction when older turns are summarized and the prompt prefix resets. Between those points, the selected model stays put so caching continues to build.

Routing across languages

Because Copilot serves developers worldwide, the routing model was trained on conversations spanning 16 language families, including CJK and European languages. In evaluations, routing accuracy stayed within four points of the English baseline across language groups, with no statistically significant quality gap.

Learning where escalation helps

Rather than labeling tasks simply "easy" or "hard," the router is trained on where models actually diverge. For each training query, responses from a less capable and a more capable model are scored across quality dimensions. The router learns when the stronger model adds real value and when a more efficient model produces an equally good result. For context-dependent messages in longer agentic sessions, training uses complete multi-turn conversations, including original user intent, recent assistant responses, and conversation metadata.

Where Auto is headed

Auto with task intent is already available in Visual Studio Code, github.com, and mobile. GitHub is expanding it to additional surfaces in the coming releases:

  • Auto with task intent is coming to Copilot CLI, GitHub App, and additional IDEs.
  • Copilot Free and Student plans will be simplified to use Auto as the only model selection option.
  • Admin controls will let organizations set Auto as the default or enforce it as the only option.

Getting more from each credit

Several user-side habits can stretch Copilot credits further:

  • Start with Auto. It selects models based on task intent without requiring manual picks each time.
  • Keep context focused. Start a new session when switching tasks, compact long sessions when needed, and mention relevant files when you already know where the code lives.
  • Avoid changing models or settings mid-session. Switching models, reasoning levels, context size, or tool configuration breaks cache reuse and forces context rebuilds.
  • Plan before parallelizing. Ask Copilot to plan first for larger tasks. Parallel agents can help when work is truly splitable, but they consume credits in parallel and should be used deliberately.
  • Use only needed tools. Broad toolsets add extra context. Enable what is relevant and disable the rest. The agent finder in GitHub Copilot can help streamline tool usage.
  • Check usage. The AI usage page shows where credits go across features and models. In Copilot CLI, session-level usage helps spot expensive patterns while working.

Detailed spending guidance is available in the documentation on getting more out of your AI credits.