How LLMs process your requests

Large language models are trained on vast amounts of text data to understand and generate human-like language. They work by predicting the next word in a sentence based on the words that came before it, making them essentially an ultra-smart autocomplete.

When working with LLMs, there are three concepts worth understanding:

  • Context: The surrounding information that helps an LLM understand what you’re asking. The more context you provide, the more likely the model will grasp your intent.
  • Tokens: Text is broken down into units called tokens, which can be a word, part of a word, or even a single letter. Too few tokens may leave the model without enough context, while too many can overwhelm it or hit built-in token limits.
  • Limitations: LLMs rely on patterns and probabilities from training data rather than true language understanding. They can hallucinate, provide incorrect answers, or produce nonsensical responses — no model is perfect.

A prompt is simply a natural language request that asks an LLM to perform a specific task. The model processes the prompt via tokens and draws on its training data to generate a response. Because these models are nondeterministic, you can prompt the same model three times and get three different outputs — which is why you’ll see varying results across tools like OpenAI’s GPT, Anthropic’s Claude, and Google’s Gemini.

What effective prompt engineering looks like

Prompt engineering is the practice of crafting requests so an LLM understands exactly what you want. Well-crafted prompts dramatically improve the quality and relevance of outputs. The key components are clarity, sufficient (but not excessive) context, and a willingness to iterate when the result isn’t what you expected.

Consider a simple example. If you tell GitHub Copilot Write a function that will square numbers in a list in an empty file, several things remain unclear: what language should the code be in? Should negative numbers be included? Will the input ever contain non-numbers? Should the function modify the original list or return a new one?

A more effective prompt would be: Write a Python function that takes a list of integers and returns a new list where each number is squared, excluding any negative numbers. Now the model knows the language, the input type, the constraints, and the expected output — so the generated code will align far better with your intent.

Fixing common prompt problems

Occasionally you won’t get the output you want. In most cases, the root cause is an underspecified prompt. Here are the scenarios you’re most likely to encounter.

Prompt confusion

Mixing multiple requests in a single prompt can confuse the model. If you highlight code and tell Copilot fix the errors in this code and optimize it, the model has to guess whether fixing or optimizing comes first — and what “optimize” even means (speed? memory? readability?).

The fix is to break the request into concrete, ordered steps: First, fix the errors in the code snippet. Then, optimize the fixed code for better performance. Clear sequential instructions make it much more likely you’ll get the result you’re after.

Token limits

Every model has a limit on how many tokens it can process at once (and GitHub Copilot offers several model options, each with different capabilities). If your prompt is too long or your expected output is extensive, the LLM may hallucinate, return a partial response, or fail outright.

Keep prompts concise and only include necessary context. Ask yourself whether the model really needs an entire code file or just a few relevant lines from one function. Instead of requesting an entire application at once, break it down and generate each component piece by piece.

Assumption errors

It’s easy to assume the model knows more than it does. Telling Copilot add authentication to my app skips over critical details: what does your app do, and which authentication technologies do you prefer?

When crafting prompts, state your requirements explicitly. Outline specific needs, mention any best practices you want followed, and refine your prompt with edge cases and constraints. This helps ensure the LLM doesn’t overlook important aspects of your request.

Prompt engineering best practices

  • Give the model enough context while staying within its limitations.
  • Make prompts clear, concise, and precise.
  • Break multi-step tasks into smaller prompts and iterate.
  • Be explicit about requirements and constraints so the model understands the boundaries of your request.

Prompt engineering takes practice, but the more you refine your prompts, the more productive you’ll be with GitHub Copilot and other LLM tools. As with coding, the key is effective communication — and building requests iteratively to steer the model toward the output you want.