What is context engineering?
Context engineering is the discipline of deliberately deciding what information goes into a language model’s context window for a request — which instructions, which retrieved documents, which memory, which prior conversation turns, and in what order — rather than treating the context window as a convenient place to dump everything potentially relevant and simply hoping the model sorts out what matters on its own. It grew directly out of the recognition that a model’s output quality depends enormously, and quite predictably, on what it’s given to work with, and that simply having a bigger context window doesn’t solve the underlying problem on its own, because more available space doesn’t automatically mean every piece of content placed somewhere within it gets used equally well by the model reasoning over it. This page covers why context engineering emerged and became its distinct discipline separate from prompt engineering, what counts as “context” in a modern AI system, why a limited context budget forces trade-offs even when the technical window is large, the core techniques — selecting, compressing, ordering, and routing context — that make up the practice, how to measure whether a context strategy is working, how the discipline plays out differently in RAG systems versus agentic systems, the failure modes that show up when context is handled carelessly, and how teams practice this iteratively over time rather than treating it as a one-time setup decision made once and never revisited.
Why context engineering became its discipline
In the earliest days of working with large language models, the dominant skill was prompt engineering: crafting the wording of a single instruction carefully enough to get a model to behave the way you wanted. That skill still matters a great deal, but it assumes a relatively simple situation — a short instruction, maybe a handful of examples, and one direct question — where the entire challenge is essentially choosing the right words for that one instruction. As AI applications grew more sophisticated, incorporating retrieved documents, conversation history, tool outputs, and persistent memory, the problem shifted: it was no longer just about how a single instruction happened to be worded, but about which of many competing, available pieces of information should even be included in the request at all, and how each of them should be arranged relative to all the others once they were.
This shift is what context engineering names directly. Where prompt engineering optimizes the wording of a relatively fixed, known set of content, context engineering optimizes which content makes it into the request in the first place, out of a much larger pool of things that could potentially be included — retrieved passages, relevant memories, recent conversation turns, tool results, system instructions — each competing for a limited amount of space and a limited amount of the model’s attention. The distinction matters practically because the two disciplines call for different skills and different tooling: prompt engineering is largely a writing and iteration problem, while context engineering is largely a retrieval, ranking, and systems-design problem, closer in spirit to information retrieval and data engineering than to the craft of writing one single, well-worded effective instruction.
What counts as “context”
Given that context engineering is about deciding what goes into the context window, it’s worth being precise about everything that competes for that space, because the term “context” covers considerably more than just a user’s literal question. System instructions establish the model’s overall role and behavior — these are usually stable across many requests but still consume space in every single one. Retrieved content, in a RAG-style system, is the material pulled from a knowledge base specifically to help answer the current question, and it’s often the largest and most variable component of what goes into a request.
Conversation history carries prior turns in an ongoing exchange, letting a model maintain continuity rather than treating every message as if it arrived out of nowhere, but it grows without bound if left unmanaged, which is exactly the problem the memory and compression techniques discussed later in this page exist to address. Memory, distinct from raw conversation history, contributes durable facts or preferences deliberately retained and surfaced when relevant, rather than replayed from a raw transcript. And in agentic systems, tool outputs — the results of a search, a calculation, an API call made partway through a multi-step task — become part of the context for every subsequent step, adding yet another category of content competing for the same limited space. Context engineering is the practice of managing all of these categories together, deciding how much of the available budget each one deserves for a request, rather than treating any single category — most commonly, retrieved content — as the only thing worth thinking carefully about.
Why a limited budget matters even with a large context window
Modern language models support context windows large enough to hold what would once have seemed like an enormous amount of text, which raises a reasonable question: if the window is that large, why does careful context engineering still matter, rather than simply including everything that might conceivably be relevant. The answer rests on a few compounding realities that don’t go away just because the technical size limit is generous.
The first is cost and latency: every token included in a request is a token the model has to process, and processing cost and response time both scale with how much context is sent, which means padding a request with marginally relevant material has an ongoing cost even when it technically fits within the window. The second, and more consequential, is a well-documented effect often called “lost in the middle”: models don’t attend to every part of a long context equally well, tending to be considerably more reliable at using information positioned near the beginning or end of a context than information buried in the middle of a long one, which means an important fact, if it’s surrounded by enough less important material, can effectively become invisible to the model’s actual reasoning even though it remains technically present and fully available somewhere within the request. The third is simpler but easy to underweight in practice: including irrelevant or only tangentially relevant material doesn’t just passively fail to help, it actively dilutes the signal the model has to work with, making it measurably harder for the model to identify what’s load-bearing among everything it was handed to work with. Taken together, these realities mean that “the context window is big enough” is never sufficient justification for skipping careful selection — a context window’s technical capacity and its actual, practical effective capacity for reliable use are two different things, and context engineering exists specifically to work within the smaller, practical one.
Selection: deciding what belongs in the context at all
Given that including everything possible isn’t a viable strategy, the first and most fundamental technique in context engineering is selection: deliberately choosing which pieces of available information make it into a request, out of everything that could potentially be included. In a RAG-style system, this is what retrieval quality is really about — not merely “does the system have access to a large knowledge base,” but “does it correctly identify which pieces of that knowledge base are relevant to this particular question,” since a retrieval system that returns twenty loosely related chunks when only three are relevant has technically retrieved information, but has done a poor job of the deeper selection task context engineering is concerned with.
Selection extends beyond retrieved content, too. For conversation history, selection means deciding how many recent turns need to be included versus how much can be summarized or dropped without losing anything the current request needs. For memory, it means retrieving only the stored facts relevant to the current situation, rather than surfacing every piece of information ever recorded about a user regardless of whether it bears on the question at hand. Getting selection right is less about any single algorithm and more about a discipline applied consistently across every category of content competing for space in the context — asking, for each candidate piece of information, whether it earns its place everything else that also wants to be included, rather than defaulting to including it simply because it’s available and there’s technically still room left in the budget for it. Treating this as a decision, made deliberately for every category of content rather than only for the retrieved-documents category most teams instinctively focus on first, is what separates disciplined context engineering from a system that merely has good retrieval and calls that sufficient.
Compression: fitting more signal into less space
Once selection has narrowed down what’s relevant, compression addresses a related but distinct problem: even relevant content is often more verbose than it needs to be for a model to use it well, and reducing that verbosity — without losing the substance — frees up space for other relevant content that would otherwise have to be cut. This isn’t about truncating information carelessly; a compressed version of a conversation history, for instance, should preserve the actual decisions, facts, and open questions from that history while dropping the conversational scaffolding — the pleasantries, the back-and-forth clarifications that were already resolved — that doesn’t carry information a model needs going forward.
Summarization is the most common compression technique, condensing a longer piece of content into a shorter one that preserves its essential meaning, and it can be applied at several different points: summarizing older conversation turns while keeping recent ones verbatim, summarizing a long retrieved document down to its most relevant sections rather than including it in full, or summarizing accumulated progress in a long agentic task rather than replaying every intermediate step in full detail. Structured extraction is a related but more aggressive technique, pulling out only well-defined fields from a longer source rather than a general-purpose summary — useful specifically when a system already knows in advance exactly what shape of information it needs, and can afford to discard everything else in that source that doesn’t fit that shape. Both techniques trade some amount of detail for the ability to include more relevant material within the same limited space, and choosing which one fits a situation is itself part of the context engineering discipline, not a decision that can be made once, up front, and then applied uniformly regardless of what’s being compressed in any case.
Ordering and structuring: where things go matters, not just what’s included
Selection and compression both address what makes it into the context; ordering and structuring address the separate question of how that included content is arranged, and this matters more than it might seem given the lost-in-the-middle effect discussed earlier. If a model attends less reliably to content buried in the middle of a long context, then the most important information — the retrieved passage most likely to answer the question, the most recent and most relevant piece of conversation history — deliberately deserves a privileged position near the beginning or the very end of the assembled context, rather than being placed wherever it simply happened to fall in whatever order the underlying content was originally gathered from its various sources.
Deliberate structure matters just as much alongside pure ordering, and shouldn’t be an afterthought: clearly delineating where one type of content ends and another begins — separating retrieved passages from conversation history from system instructions with clear formatting or explicit labels — helps a model correctly attribute information to its actual source and purpose, rather than blurring several different categories of content together into one long, undifferentiated block of text that’s considerably harder to reason over correctly. This is an underrated, often overlooked lever within context engineering as a whole: two requests containing the exact same underlying information, but assembled with different ordering and structure, can produce measurably different quality outputs, purely because one arrangement made it measurably easier for the model to locate and correctly use what mattered most and the other simply didn’t.
Routing: not every request needs the same context strategy
Selection, compression, and ordering all assume a single context-assembly strategy applied to a request, but a further, higher-level technique recognizes that different kinds of requests benefit from different context strategies entirely, which is what routing addresses. A simple factual question might need only a small amount of tightly relevant retrieved content and little else. A complex, multi-part question might benefit from decomposition into several sub-queries, each retrieving its relevant context before being combined. A question that’s about clarifying something from earlier in the conversation might need conversation history weighted much more heavily than fresh retrieval, since the answer isn’t really “out there” in a knowledge base at all — it’s already sitting in what was previously discussed.
Building a system that recognizes these differences and routes each request through the context-assembly strategy that fits it, rather than applying one fixed strategy uniformly regardless of what a request needs, is a more sophisticated but often meaningfully more effective approach than treating context engineering as a single, one-size-fits-all pipeline. This routing decision doesn’t need to be exotic — it can be as simple as a lightweight classification step that decides, before the more expensive retrieval and assembly work happens, roughly what kind of request this is and which of a small number of established context strategies best fits it.
How context engineering and prompt engineering relate
Having covered selection, compression, ordering, and routing as the core techniques, it’s worth returning briefly to the relationship between context engineering and prompt engineering, since the two are easily conflated but address different layers of the same overall problem. Prompt engineering operates on the instructional layer: given that a certain set of content is going into a request, how should the instructions around it be worded to get the model to use that content well — how explicitly to ask for citations, how to phrase a request for a particular output format, what tone or reasoning style to encourage. Context engineering operates one layer below that: deciding what that content is in the first place, before any instructional wording gets applied to it at all.
The two disciplines are complementary rather than competing, and a system that’s strong in one but weak in the other tends to hit a ceiling. Excellent prompt wording applied to poorly selected, badly ordered context will reliably produce a mediocre result, because no amount of careful instruction can make a model reason well over content that was never the right content, or that was buried where the model can’t reliably attend to it. Conversely, excellently engineered context — precisely selected, well compressed, clearly ordered — handed to the model with vague, poorly considered instructions will also underperform, because the model still needs to be told clearly what to do with the good material it’s been given. Mature AI system development tends to treat these as two distinct workstreams, both necessary, with context engineering typically demanding more systems-level, retrieval-oriented thinking, and prompt engineering demanding more careful, iterative attention to language and instruction design — different skills, applied together, rather than one discipline that happens to go by two different names.
Measuring whether a context strategy is working
Everything covered so far describes how to build a deliberate context strategy; it’s worth being equally concrete about how to tell whether that strategy is working, since context engineering decisions can look reasonable in isolation while still producing a context that serves the model poorly in practice. A useful starting measurement is context precision: of everything included in the assembled context for a request, how much of it was relevant and used, versus included but ultimately unnecessary — a low precision score is a direct signal that selection is including more than it should, diluting the useful material with content that didn’t need to be there.
Context recall complements this from the other direction: of everything that would have been useful for answering a request well, how much of it made it into the assembled context — a low recall score points to selection being too conservative or retrieval missing relevant material, leaving the model to work with an incomplete picture even when what it was given was, as far as it went, accurate. Beyond these two, position-sensitivity testing — deliberately checking whether a model’s output quality changes measurably when the same underlying content is reordered within the context — gives a direct, empirical read on whether an ordering strategy is helping or whether the lost-in-the-middle effect is degrading results regardless of what ordering choices were made. And tracking these metrics not just in aggregate but broken down by request type is what reveals the kind of pattern discussed in the earlier section on iterative practice — a context strategy performing well on average while quietly underserving one identifiable category of request, a gap that only becomes visible once measurement is granular enough to separate that category out from the overall average it would otherwise be hidden inside.
Context engineering in RAG systems specifically
In a retrieval-augmented system, context engineering is largely, though not entirely, about the retrieval and assembly of external content: how documents get chunked in the first place shapes what units of content are even available to select from later, hybrid search and reranking determine which chunks get selected out of everything retrieved, and assembly determines how those selected chunks get combined with the user’s question and any relevant conversation history into the final context sent to the model. Every one of the techniques discussed throughout this page shows up here in a fairly direct form: selection is retrieval and reranking, compression might mean summarizing a long retrieved document down to its most relevant passages, and ordering means placing the most relevant retrieved content where the model is most likely to use it well.
What’s specific to RAG is the tight coupling between context engineering and retrieval quality — a context engineering strategy in a RAG system is only as good as the retrieval system feeding it, since no amount of careful compression or ordering can compensate for content that was never the right content to include in the first place. This is why context engineering and retrieval system design tend to be treated as closely related, mutually dependent concerns in a well-built RAG system, rather than as separate problems solved independently of each other by different parts of a team working without much coordination between them. A team that carefully tunes context assembly — ordering, structuring, compression — while leaving retrieval quality unexamined is optimizing the part of the pipeline that can only ever polish what retrieval already handed it, and will eventually run into a ceiling that no amount of downstream context engineering can push past on its own, however carefully that downstream work is done.
Context engineering in agentic systems specifically
Agentic systems introduce a context engineering challenge that doesn’t exist in a single-shot RAG system: context accumulates across many steps within one task, and managing what carries forward from one step to the next is an ongoing, continuous problem rather than a single decision made once per request. Every action an agent takes and every result it receives becomes a candidate for inclusion in the context of its next reasoning step, and without deliberate management, this accumulated history grows unboundedly across a long task, running directly into the cost, latency, and lost-in-the-middle problems discussed earlier, just compounded across many steps instead of appearing once in a single request.
Well-designed agentic context engineering typically maintains a compact, actively updated working summary of task progress — what’s been tried, what’s been learned, what’s still outstanding — rather than replaying an ever-growing raw transcript of every single step at every subsequent step. This connects directly to the compression techniques discussed earlier, applied continuously rather than once: as a task progresses, older steps get summarized or dropped in favor of a current, compact state that captures what still matters, freeing up space for the more immediately relevant recent steps and the next decision the agent needs to make. An agent that fails to manage its accumulating context well is one of the more common, and more avoidable, reasons a long-running agentic task degrades in quality the longer it runs — not because the underlying model got any less capable partway through, but because the context it was working from became progressively noisier and less well-organized as the task went on, drowning out the useful signal from earlier steps under an ever-thickening layer of raw, unmanaged history nobody actively curated.
Failure modes: what happens when context engineering is neglected
Understanding the techniques covered so far is easier when grounded in what goes wrong without them. Context overflow is the most direct failure: content that should have been selected or compressed instead gets included in full, past the point where it fits comfortably within a useful, well-attended context, either hitting a hard technical limit or, more insidiously, staying technically within the limit while still triggering the lost-in-the-middle degradation discussed earlier. Irrelevant context dilution is a subtler version of the same underlying problem: nothing technically overflows, but enough marginally relevant material gets included that the important content is harder for the model to locate and weight correctly among everything else competing for its attention.
Stale or contradictory context is a failure specific to systems that carry memory or accumulated state forward: outdated information, never properly reconciled or removed, sits alongside current information, and a model given both has no reliable way to know which one to trust, sometimes producing an answer that blends the two in a way that’s wrong regardless of which individual piece of context it happened to weight more heavily. And poorly structured context — technically containing all the right information, but presented as an undifferentiated block without clear boundaries between different content types — makes it harder for a model to correctly attribute and use what it’s been given, even when every individual piece of that context was, in isolation, exactly the right thing to include. Each of these failure modes traces back to skipping one of the deliberate techniques covered earlier — selection, compression, ordering, or routing — which is precisely why context engineering treats each of them as a distinct, necessary discipline rather than a single problem solvable with one general-purpose fix.
Practicing context engineering as an ongoing, iterative discipline
Everything covered so far might suggest context engineering is a design decision made once, early in a system’s life, and left alone afterward, but in practice it’s considerably more iterative than that because what constitutes the right context strategy tends to become clearer only once a system is handling varied traffic rather than the narrower range of scenarios anticipated during initial design. Teams that treat context engineering well typically build the ability to inspect exactly what context was assembled for a request — which connects directly to the broader observability practices that matter across AI systems generally — so that when an answer is wrong or a task degrades, the actual assembled context for that case can be reviewed directly, rather than guessed at.
This visibility is what turns context engineering from a one-time design exercise into an ongoing practice: patterns discovered through reviewing problematic cases — a category of question where retrieval consistently over-includes irrelevant material, an agentic task type where working-summary compression is losing something that turns out to matter — feed back into refined selection, compression, ordering, or routing logic, which then gets evaluated against a representative set of test cases before being rolled out more broadly. Treating context engineering this way, as a discipline that keeps improving based on observed evidence rather than a set of decisions finalized once during initial development, is what separates a context strategy that stays effective as a system’s usage grows and diversifies from one that was reasonably well-tuned for its original, narrower set of anticipated use cases and quietly degrades as usage inevitably extends beyond them.
A worked example: context engineering for a technical support assistant
These principles are easier to see clearly against a concrete case, so consider a technical support assistant fielding a wide range of questions about a complex software product, drawing on documentation, past support tickets, and an ongoing conversation with the current user. Selection here means retrieval scoped specifically to the product area the current question is about, rather than searching the entire documentation set indiscriminately, combined with a lightweight routing step that recognizes whether a question is a fresh technical query, a follow-up to something already discussed, or a request that needs a past ticket looked up directly rather than searched for semantically.
Compression shows up in how conversation history is handled: the current and immediately preceding turn are kept verbatim, since they’re most likely to be directly relevant to interpreting the current message correctly, while anything older in a long support conversation gets compressed into a compact summary capturing the actual issue being worked through and what’s already been tried, rather than replayed in full as the conversation grows. Ordering places the most relevant retrieved documentation immediately adjacent to the current question, with system instructions and the compressed conversation summary positioned so the model isn’t forced to hunt through a long, undifferentiated block of text to find what matters for answering the question just asked. And the whole assembled context for any interaction can be inspected directly by the team maintaining the assistant, which is what let them notice, in an earlier version of the system, that a category of question was consistently retrieving outdated documentation for a feature that had since been redesigned — a problem invisible from the final answers alone, but immediately obvious once the actual assembled context for those cases was reviewed directly.
Measurement closes the loop on this example the same way it’s meant to in any well-run context engineering practice: after the team fixed the stale-documentation issue by adjusting how retrieval weighted recency for that product area, they tracked context precision and recall specifically for that category of question over the following weeks, confirming the fix improved the assembled context rather than simply assuming a plausible-looking change had worked and moving on. Position-sensitivity testing on this same assistant had earlier revealed something the team hadn’t anticipated: when a support ticket citation and a documentation citation were both relevant to a question, placing the documentation excerpt first consistently produced more accurate answers than the reverse ordering, a pattern that wasn’t obvious from reasoning about the content alone and only became visible through deliberate, empirical testing of ordering choices rather than intuition about which arrangement seemed more sensible on paper.
Common mistakes in how teams approach context engineering
A consistent set of mistakes shows up across teams building AI systems, worth naming directly because each one traces back to skipping one of the deliberate practices discussed throughout this page. The most common is treating a large context window as a substitute for careful selection, including as much potentially relevant material as technically fits rather than deliberately choosing what belongs, which runs directly into the lost-in-the-middle and signal-dilution problems discussed earlier regardless of how generous the underlying model’s context limit happens to be.
A second common mistake is optimizing retrieval and selection carefully while completely neglecting ordering and structure, assuming that once the right content has been selected, however it happens to be arranged afterward doesn’t meaningfully matter — which overlooks a measurable lever for improving output quality using content the system already correctly identified as relevant. A third is applying one fixed context-assembly strategy uniformly to every request, without any routing logic to recognize that different kinds of questions benefit from different strategies, treating a simple factual lookup and a complex, multi-part question as if they call for exactly the same approach to what and how much gets included. A fourth, specific to agentic systems, is letting working context accumulate across a long task without active compression or summarization, which degrades an agent’s reasoning quality more the longer a task runs, precisely when reliable reasoning matters most. And a fifth, tying every one of these together, is treating context engineering as a one-time setup decision rather than the ongoing, evidence-driven practice described in the previous section — building an initial strategy, never revisiting it once usage reveals its actual gaps, and being surprised, months later, that a system which performed well in early testing has quietly grown less reliable as varied traffic exposed exactly the context-handling weaknesses that early testing was never broad enough to surface in the first place.
What runs underneath every one of these mistakes, and every technique discussed before them, is a single reframing worth carrying forward: a language model’s output is only ever as good as the context it was given to reason over, and a bigger context window changes what’s technically possible to include without changing what’s wise to include. Selection, compression, ordering, and routing are the deliberate, learnable practices that turn “technically possible to include” into “useful once included” — and the gap between those two things, more than any single choice of model or retrieval algorithm, is what tends to separate an AI system that reliably gives good answers from one that merely has access to good information somewhere within reach of it.