The problem with predicting edits, not tokens
Editing code is rarely a single action. A typical session involves a sequence of small changes: refactors, fixes, edge-case handling, cleanup. In February, GitHub launched next edit suggestions (NES), a custom Copilot model designed to predict that sequence—to infer the logical next edit from the code you've already written. The model has since received several major updates, the newest shipping earlier this month.
Predicting the next edit is a fundamentally harder problem than predicting the next token. The model has to understand what you're doing and why, then respond quickly enough to stay in your flow. It also has to know when to stay silent, since noisy suggestions break focus. NES must infer intent purely from local context—there are no explicit prompts—and it must integrate tightly with VS Code so suggestions appear exactly where expected.
Frontier models didn't meet the bar on quality and latency. Smaller models were fast but produced weak suggestions; larger ones were accurate but too slow for an in-editor experience. NES isn't a general-purpose chat model. It's a low-latency, task-specific model that runs alongside the editor, responding in real time. That required co-designing model training, prompting, and UX around a single goal, with tight coordination between the model training, prompt design, UX design, and VS Code teams.
Building the right training data
The architecture wasn't the hard part—the data was. No existing dataset captured real-time editing behavior, and the first attempt made that painfully clear. Using internal pull request data seemed reasonable: diffs look like edits. But internal testing revealed a model that was overly cautious, reluctant to touch unfinished code, hesitant to suggest changes to the line being typed, and often choosing to do nothing. In practice, it performed worse than a vanilla LLM.
Pull request data failed because it reflects a reviewed final state, not the editing process:
- It shows only the final state, not the intermediate edits developers make along the way
- It lacks temporal ordering, so the model can't learn when changes happen
- It contains almost no negative samples—cases where the correct action is "don't edit"
- It misses abandoned edits, in-progress rewrites, and other common editing behavior
The team reset its approach and collected a richer dataset by capturing editing sessions from internal volunteers at scale. Data quality proved critical: a smaller volume of high-quality edit data produced better models than a larger volume of less-curated data. Supervised fine-tuning (SFT) on this custom dataset produced the first model to outperform vanilla models, providing a foundation for subsequent releases.
Refining with reinforcement learning
After several successful SFT-based models, two limitations surfaced. SFT can teach the model what a good edit suggestion looks like, but not what makes one bad. And SFT relies on labeled suggestions, leaving the much larger pool of unlabeled code samples untapped.
The team turned to reinforcement learning (RL) to address both issues. Starting from a well-trained SFT model, they optimized it against a broader set of unlabeled data using a grader capable of judging the quality of the model's edit suggestions. The grader design relies on a few key ideas:
- It uses a large reasoning model with specific grading criteria
- Its criteria are routinely updated based on new analysis of model outputs, seeking qualities that indicate unhelpful edits
- It evaluates not just suggestion correctness, but whether the resulting code diff is easy to read in the UI
- Prompt optimization: NES runs many times per minute as you edit, so context size directly affects latency. Trimming prompts, reusing cached tokens between calls, and removing unneeded markup made suggestions appear faster without reducing quality.
- Data quality filtering: LLM-based graders filter out ambiguous or low-signal samples, reducing unhelpful or distracting suggestions.
- Synthetic data: Distilling data from larger models trains a smaller model without losing quality.
- Hyperparameter tuning: Hyperparameters were tuned for the new base architecture to optimize suggestion quality.
- Offline testing: Models are tested against targeted cases to gauge performance in specific scenarios.
- Internal dogfooding: Engineers across GitHub and Microsoft use each candidate in daily workflows, providing qualitative feedback.
- A/B experiments: The most promising candidates serve a small percentage of real-world NES requests, tracking acceptance, hide, and latency metrics before a ship decision.
- April: Improved model quality and restructured the response format to require fewer tokens, yielding faster, higher-quality suggestions.
- May: Addressed feedback that NES was suggesting too often by improving suggestion quality and reducing the model's eagerness, leading to fewer workflow disruptions.
- November: After nearly thirty summer candidate models failed to beat the May release in A/B testing, this version finally cleared the bar, delivering higher quality at lower latency via shorter prompts, reduced response length, and increased token caching.
- Reducing eagerness: Adding more "no-edit" samples and tuning suggestion thresholds so the model intervenes only when likely useful, not distracting.
- Increasing speed: Since NES runs multiple times per minute, latency is reduced at the model, prompt, and infrastructure levels.
- Improving developer experience: Refining how edits are displayed so suggestions feel visible but not intrusive, with expanded settings for customizing NES behavior.
- Edits at a distance: Suggestions across multiple files, not just where you're typing
- Faster responses: Continued latency improvements across model and infrastructure
- Smarter edits: Better anticipation of context and cross-file dependencies
Continued post-training with RL has improved generalization. Because it extends training to unsupervised data, RL expands the volume and diversity of available data without requiring known ground-truth edits. This pushes the training process toward harder cases and prevents collapse into simple scenarios. RL also makes preferences explicit through the grader, letting the model better avoid bad suggestions on out-of-distribution cases.
Improving the latest model
The most recent NES release builds on that foundation with changes across four areas:
How candidates are evaluated
The team trains dozens of model candidates per month. Each modifies training data, adapts training approaches, experiments with base models, or targets specific developer feedback. Every candidate passes through three evaluation stages:
Since the initial launch, three major model updates have shipped, each balancing speed and precision:
| Release | Shown rate | Acceptance rate | Hide rate |
|---|---|---|---|
| April | +17.9% | +10.0% | -17.5% |
| May | -18.8% | +23.2% | -20.0% |
| November | -24.5% | +26.5% | -25.6% |
Balancing eagerness and speed
Developer feedback has guided almost every NES change. Some developers found early versions too eager, suggesting edits prematurely. Others wanted the opposite—a more assertive experience. There's no universal preference, so the team focused on a default that works for most, shifting that balance over time with real usage data:
Future work includes adaptive behavior: NES adjusting to each developer's editing style over time, becoming more aggressive or restrained based on patterns of accepting, dismissing, or ignoring suggestions. That work is ongoing, informed by current feedback.
What's next
To try the newest model, update VS Code and the Copilot Chat extension, then confirm NES is enabled in your VS Code settings.
