A guide to deciding what AI model to use in GitHub Copilot

Model Selection Is a Moving Target

GitHub Copilot continues to add support for new AI models, which raises a practical question for developers: how do you pick the right one? There's no single "best" model—your needs depend on the task, your tolerance for latency, and the kind of output you want. Some developers prefer verbose chat responses; others want terse autocomplete suggestions.

Rather than chase every release, it helps to have a framework for evaluating any model that comes along. The strategies below, drawn from how several developers approach model selection, can help you judge whether a model fits your workflow—regardless of what lands next.

Why You Might Not Need a Single Model

You can switch between models for both chat and code completion in Copilot, so there's no reason to lock yourself into one. Mixing models is common, and often the right call.

[[BLOOK_0]]

Chat vs. Autocomplete

One widely used pattern is pairing a fast model for autocomplete with a more capable model for chat. Autocomplete needs to be quick and responsive, since you're reading and typing in real time. Chat, on the other hand, tolerates more latency—you're often in an exploratory mode, like mapping out a complex refactor. Developers are generally more patient when they're actively thinking through a problem.

Bringing in Reasoning Models

Reasoning models such as OpenAI o1 behave differently from traditional LLMs like GPT-4o or Claude Sonnet 3.5. They take longer to respond because they break a prompt into parts and consider multiple approaches before answering. That deliberative process makes them stronger for complex tasks, even if it costs time.

Several developers deliberately reach for reasoning models in specific situations. Fatih Kadir Akın, a developer relations manager, uses o1 when starting new projects from scratch: "Reasoning models better 'understand' my vision and create more structured projects than non-reasoning models." FirstQuadrant CTO Anand Chowdhary favors them for large-scale refactors: "A model that rewrites complex backend code without careful reasoning is rarely accurate the first time. Seeing the thought process also helps me understand the changes." Cassidy Williams, GitHub Senior Director of Developer Advocacy, splits work between models—GPT-4o for refining prose and Claude 3.7 Sonnet Thinking for verifying code accuracy. She notes that "reasoning models help ensure technical correctness because of their multi-step process. If they initially get something wrong, they often correct themselves in later steps so the final answer is more accurate."

[[BLOOK_1]]

What Matters When Evaluating a Model

When a new model appears and you want to test it, these are the qualities developers actually check.

How Recent Is the Training Data?

Models are trained on different snapshots of the world. That matters if you use newer versions of programming languages, frameworks, or libraries. One developer starts by asking for autocomplete on a project manifest file to see what versions get suggested. "If the versions are quite old, I'll move on," says Xavier Portilla Edo, a cloud infrastructure engineering lead.

Speed and Responsiveness

Latency tolerance is higher in chat than in autocomplete, but responsiveness still shapes the experience. Rishab Kumar, a staff developer evangelist at Twilio, values fast responses in conversational interactions: "I enjoy bouncing ideas off a model and getting feedback. For that type of interaction, I need fast responses so I can stay in the flow."

Accuracy and Code Quality

Correctness goes beyond whether code runs. Portilla Edo compares models based on code structure, patterns, comments, and adherence to best practices. He also looks at maintainability: "Does it follow naming conventions? Is it modular? Are the comments helpful or just restating what the code does? These are all signals of quality that go beyond whether the code simply runs."

Testing a Model in Your Own Workflow

You can read all the benchmarks you want, but the real test is using the model in your normal routine.

Start Small, Then Scale

Akın typically starts with a simple todo app in vanilla JavaScript to check code structure. Kumar begins with a Python websocket server. The point is to start with something you understand well enough to judge, then add complexity. Akın eventually pushes models further: "Eventually I'll see if it can build something in 3D using 3js."

Portilla Edo takes a similar approach with Copilot Chat. "I usually ask it for simple things, like a function in Go, or a simple HTML file," he says, then moves on to autocompletion to see how the model performs there.

Live With It for a While

Chowdhary's approach is direct—when a new model ships, he puts it in his daily workflow and lives with it. "Available benchmarks and tests only tell you part of the story. I think the real test is seeing if it actually improves your day to day."

He watches whether it speeds up debugging or produces cleaner refactors. "It's kind of like dogfooding your own stack: You won't know if it really fits your workflow until you've shipped some real code with it. After evaluating it for a bit, I decide whether to stick with the new model or revert to my previous choice."

The Bottom Line

There's no substitute for hands-on evaluation. Developers who switch models regularly agree that available benchmarks and documentation only get you so far—actual usage reveals how a model fits your specific tasks and preferences.

The landscape is moving fast, though. As Chowdhary puts it: "You don't need to be switching models all the time, but it's important to know what's going on. The state of the art is moving quickly. It's easy to get left behind."