Frontier model, same app: a Copilot comparison in ask, edit and agent modes
GitHub Copilot now supports Claude 3.7 Sonnet, Gemini 2.5 Pro, GPT-4 and other frontier models, but choosing between them isn't always obvious. That becomes trickier as Copilot Chat expands into ask, edit and agent modes, each with different strengths. In a recent video, GitHub Developer Advocate Kedasha Kerr and I built the same travel-reservation web app three times, swapping foundation models mid-session to see how each behaves in realistic coding workflows.
We deliberately kept the stack lean to focus on the comparison: a Flask REST API backend, a Vue.js frontend styled with Tailwind, and a local data.json file standing in for a real database. That gave us enough room to scaffold, wire endpoints and add tests and docs while keeping the demos quick.
Three modes, one Copilot
Before getting into results, it's worth restating the division of labor between Copilot's three modes. Ask mode answers questions about your code and project. Edit mode is a focused rewriting tool for precise changes. Agent mode goes further, driving a task from prompt to committed code across the repository.
Context beats model choice
The first lesson from the demo is that the model is only part of the equation. Prompt shape and the additional files you feed Copilot often dominate output quality. Toggling between modes and switching models mid-build showed how much behavior changes when Copilot is given the right amount of detail or prompted to think several steps ahead.
If you aren't deliberately structuring prompts and selecting context, you're likely leaving accuracy and relevance on the table. For a deeper look, GitHub's guide "Which AI model should I use with GitHub Copilot?" breaks down model tradeoffs more systematically.
Agent mode earns its keep
Agent mode is still maturing, but it already handles multi-file tasks with minimal guidance. It navigates your repository, applies changes and can complete whole workflows that previously required substantial manual orchestration. That power requires a different level of trust and supervision, but two examples from our build show where it fits:
- One-shot project scaffolding: After pointing Copilot at the project README, Kerr invoked agent mode with a simple "implement this" instruction. Running Gemini 2.5 Pro, agent mode generated the complete Flask and Vue repository, including directories, boilerplate code, unit tests and seeded data.
- Documentation on demand: Switching to Claude 3.5 and prompting "make documentation for this app … include workflow diagrams in Mermaid," agent mode produced a polished README, an API reference and two sequence or flow diagrams, then opened a preview to render the charts before I committed.
Custom instructions keep models on script
Custom instructions emerged as a major lever in the demo. They let you set ground rules before Copilot makes any suggestion, covering API call patterns, naming conventions and style standards.
Kerr highlighted how they tune tone, code style and task focus for individual or team workflows. In our case, we used the same custom instructions for every model so that swapping between Claude, Gemini and GPT produced consistent, secure code without re-explaining expectations each time. If you haven't set up custom instructions yet, they're worth exploring when your output needs to follow a stable standard.
Speed versus quality is a real tradeoff
The sessions also underscored the constant tradeoff between responsiveness and output quality. A larger model may take longer to suggest edits but deliver better refactoring ideas; a smaller one is quicker but may miss subtleties in a design.
The practical takeaway is not to hunt for a single "best" model but to know which to switch to and why. A default model may serve you well most of the time, yet having alternatives lined up can cover edge cases faster and more effectively. The code repository from the video is available for forking if you want to test that mix yourself.
