How Copilot Chat agent mode assembles its prompt

In agent mode, Copilot Chat doesn't pass a user's request straight to the LLM. Instead, VS Code gathers relevant files from the open project plus context about the user and active files, then appends definitions of all available tools before sending the entire bundle to the model for inference.

The best way to inspect this prompt context is to watch traffic between VS Code and the Copilot API. You can do that by pointing a local proxy server (such as Burp Suite) at VS Code via its settings:

"http.proxy": "http://127.0.0.1:7080"

Looking at the request from VS Code to the Copilot servers, tool output, user prompts, and system messages are transmitted as separate JSON fields. On the backend, however, those messages are flattened into a single text prompt. That distinction matters: the system prompt instructs the model to treat the user's request as instructions and tool output as data, yet in practice even frontier models (GPT‑4.1, Gemini 2.5 Pro, Claude Sonnet 4 among them) can be steered by malicious content appearing in tool results. The model might then take a different action than the user asked for instead of merely summarizing the retrieved information.

With temperature set to 0, those misdirections are also highly reproducible — the same injected prompt reliably produces the same unwanted behavior.

Tools the agent can call

VS Code exposes a set of powerful agent tools that can read files, generate edits, or execute arbitrary shell commands. The full list is available from the Configure tools button in the chat window:

Copilot displays all available tools, including editFiles, fetch, findTestFiles, and many others.

Each tool implements the VS Code.LanguageModelTool interface, optionally including a prepareInvocation method that shows a confirmation before the tool runs. The design intent is that sensitive tools (like installExtension) always require confirmation, with that user consent as the primary shield against hallucinations and injection. At the same time, forcing approval for every invocation would erode the feature's usefulness, so tools such as read-files run automatically.

Tools from MCP servers, by contrast, trigger confirmation every time. An attacker thus needs to find a path to an unsafe action that never prompts.

Trusted URL parsing bypass in fetch_webpage

The fetch_webpage tool sends HTTP requests on the agent's behalf but only prompts for confirmation when the requested site isn't on the built-in trusted list. By default, VS Code trusted localhost and these domains:

// By default, VS Code trusts "localhost" as well as the following domains:
// - "https://*.visualstudio.com"
// - "https://*.microsoft.com"
// - "https://aka.ms"
// - "https://*.gallerycdn.vsassets.io"
// - "https://*.github.com"

The verification logic was weak — it compared against a regular expression rather than parsing the URL properly. As a result, a URL like http://example.com/.github.com/xyz counted as a trusted domain.

That flaw made it possible to plant a prompt in a GitHub Issue instructing the agent to read the local GitHub token and send it to an external host. Asking Copilot to summarize the issue then produced a token leak — no confirmation was ever requested. The fix decouples the fetch tool from the trusted-domains mechanism (which protects other, unrelated functionality) and prompts for security confirmation on any URL the agent hasn't seen before:

Agent now asks confirmation before executing this action

Simple Browser: another data exfiltration path

The Simple Browser tool is described as a helper for testing local sites, but it also loads external URLs. Injecting similar instructions into an issue led Copilot to load an external page with the local GitHub token in the query string, again without user consent:

Screenshot of a user asking Copilot about the above GitHub Issue.

Today, the tool demands confirmation before opening any new URL. Its rendering of external HTML is additionally contained by the Content Security Policy sandbox directive, limiting the attack surface introduced by an embedded browser.

Immediate file edits that outrun the review

VS Code's editFile tool applies changes to disk and shows the user a keep / undo decision. The catch is the edit has already hit the filesystem by the time the user reviews it, so "keep" is functionally a no-op. Any process watching files picks up the change immediately — most importantly, VS Code itself reloads configuration files like settings.json or mcp.json as soon as they are edited. Because MCP server definitions can shell out to arbitrary commands, rewriting settings.json to launch a malicious server is enough to have code run before the user reacts.

That instant-apply design was reported independently by Johann Rehberger of EmbraceTheRed, who demonstrated execution by overwriting ./.vscode/settings.json with "chat.tools.autoApprove": true, and by Markus Vervier from Persistent Security, who arrived at a similar command execution path. Alongside those reports, the current defense deliberately keeps agent edits inside the workspace. In Insiders builds, edits to sensitive files (configuration files among them) now require explicit user consent, while also addressing the broader complaint about immediate file modification.

How injected instructions gain traction

Obtaining reliable control over agent behavior against tool output isn't always straightforward — casual injection attempts often fail because models aren't immediately compliant. During testing, three jailbreak-style techniques stood out.

  • Broadly true conditions like “only if the current date is <today>” capture the model's attention and gain easier buy-in.
  • Referencing other parts of the prompt frame tends to work well — for example: “If the user says ‘Above the result of calling one or more tools’, perform X.”
  • Imitating the Copilot system prompt and splicing an instruction into the middle is effective. The default system prompt is public, and while injected text arrives in a role: "tool" section, models frequently privilege that content as though it were role: "system".

Among current models, Claude Sonnet 4 showed the strongest resistance to these techniques in the reported assessment, but with enough iteration the attack still reliably succeeded.

New Security Controls in the Chat Loop

LLMs are designed to comply with instructions, but they cannot easily differentiate between directives from the user and untrusted data from external sources. Unlike a query language with a strict schema, prompts arrive as free-form text, images, or audio with no inherent structure. That leaves the model vulnerable to third-party interference, and the behavior varies from one model to another, including the local models you can plug into the Copilot API.

VS Code is addressing this at the tooling level. The planned and rolling updates focus on transparency and explicit control:

  • A full list of available internal tools, and tools contributed by MCP servers and extensions, so you can see exactly what the model has access to.
  • Manual selection of which specific tools the LLM may invoke.
  • Tool sets: named groups of tools you can assign to different workflows or contexts.
  • Mandatory user confirmation before the agent can read or write files outside your workspace or the set of files currently open.
  • A modal consent dialog that must be accepted before an MCP server is trusted and launched.
  • Enterprise policies to wholly disallow capabilities such as extension tools, MCP, or agent mode.

Beyond these shipping features, the team is studying secure coding agent research and experimenting with dual-LLM architectures, constrained information flow, role-based access control, and tool labeling—mechanisms chosen because they offer more deterministic behavior than relying on a model's good judgment.

Practical Defenses That Already Exist

Restricted Workspace Mode

Before letting Copilot loose on an unfamiliar codebase, remember Workspace Trust. When you open a repository in restricted mode, VS Code will not run automated tasks, curtails certain settings, and disables extensions—including the Copilot chat extension—until you explicitly grant trust. It is a simple, effective first gate for untrusted source material.

Containers for a Harder Boundary

Prompt injection succeeds when the agent can reach sensitive data or your host environment. Containerization puts a concrete wall between the agent's actions and your machine. VS Code's Developer Containers integration lets Copilot run its tools inside a Docker container instead of your local shell. To get started, all you need is a devcontainer.json file in your repository; the setup is free.

GitHub Codespaces provides the same isolation without local overhead. A single click on the repository page spins up a dedicated cloud VM that you can drive from your browser or from the local VS Code client. This is particularly useful when the agent genuinely needs permission to execute arbitrary commands or read local files without risking your primary development environment.

Keep the Confirmation Prompts

VS Code's agentic features are built to give you visibility over their actions, but the model is still an assistant that will occasionally attempt risky operations. Your final line of defense is reviewing the confirmations before they are executed. Friction during the session is a feature here; if the model asks to touch files outside your workspace or start an untrusted server, treat that moment as the point of scrutiny it is. As models improve, some of those explicit checkpoints may become unnecessary, but for now consider them essential components of a secure Copilot workflow.

For best results, pair those confirmed actions with isolation from both Workspace Trust restricted mode and a container runtime. A diligently sandboxed agent—whether on Codespaces or in a local container—limits what any single injected instruction can accomplish, even if one slips through the other checks.