The hardest part of building reliable AI systems is increasingly not what you ask the model, but what the model is allowed to know when it decides what to do next.
For the first few years of the modern large language model era, prompt engineering became one of the defining disciplines of applied AI. Developers learned that relatively small changes in wording, examples, formatting, and instruction hierarchy could produce substantial differences in model behavior. System prompts became more sophisticated, prompt templates evolved into application infrastructure, and techniques such as few-shot prompting, structured outputs, role assignment, and explicit reasoning instructions became part of the standard AI engineering toolkit. This approach worked particularly well when interaction with a model followed a simple pattern: a user submitted an input, the application constructed a prompt, the model produced an answer, and the process ended.
Modern AI systems increasingly operate under a very different architecture. Agents can inspect files, search repositories, execute code, query databases, call external APIs, interact with Model Context Protocol servers, maintain state, retrieve memories, delegate work to subagents, and continue operating across long trajectories involving many model calls. In those systems, the wording of the prompt still matters, but it represents only one component of a much larger information environment.
That broader discipline is increasingly described as context engineering.
In Anthropic's guide to context engineering, context engineering is described as the process of curating and maintaining the optimal set of tokens available to a model during inference. That set may contain system instructions, conversation history, tool definitions, retrieved documents, memory, Model Context Protocol resources, application state, previous tool results, and information generated by other agents. The central problem is therefore no longer limited to how an instruction should be phrased, but extends to deciding what information should enter the model's working context, how it should be represented, how long it should remain there, and what should be deliberately excluded.
Prompt engineering asks how to communicate an instruction effectively. Context engineering asks how to construct the information environment in which that instruction will be interpreted.
The second problem is considerably larger.
The Prompt Is Only One Part of the Context
In a production AI agent, the model rarely receives only a user message and a system prompt. Its effective input may also include developer policies, retrieved documents, tool schemas, application state, repository files, previous execution errors, structured memory, intermediate reasoning artifacts, and outputs produced by external systems. Each of these elements consumes context capacity, but more importantly, each can influence what the model believes is relevant and how it decides what to do next.
This means that even an excellent system prompt can be undermined by poor surrounding context. Irrelevant retrieval results may distract the model, ambiguous tool descriptions may lead to incorrect tool selection, outdated memory may conflict with current state, and verbose tool output may bury the information that actually matters. The problem is not merely that unnecessary context costs more tokens. Excess information can alter the quality of the model's reasoning itself.
This is why context engineering is better understood as information architecture for model inference. The objective is not to maximize the amount of information placed inside the context window, but to maximize the amount of useful information available at the moment a decision is made.
Anthropic expresses a similar principle in its context engineering guidance, describing effective context as the smallest set of high-signal tokens that maximizes the likelihood of the desired behavior. That framing matters because it shifts the optimization target away from context capacity and toward context quality.
A bigger context window does not automatically mean a better context.
Bigger Context Windows Do Not Eliminate the Problem
The rapid expansion of model context windows created an intuitive assumption that context management might eventually become unnecessary. If a model can accept hundreds of thousands of tokens, it seems reasonable to simply provide an entire repository, document collection, or conversation history and allow the model to decide what matters.
Research suggests that this assumption is unreliable.
One of the foundational studies in this area, Lost in the Middle: How Language Models Use Long Contexts, examined how language models use relevant information placed at different positions within long inputs. The researchers found that performance could vary significantly depending on where the required information appeared. Models frequently performed better when relevant evidence was positioned near the beginning or end of the context, while information located in the middle was often used less reliably.
The result exposed an important distinction between context capacity and context utilization. A model may technically accept information without being equally effective at reasoning over every piece of that information. The fact that something fits inside the context window does not guarantee that it will receive equal attention or contribute reliably to the final result.
More recent research has explored the problem from a broader perspective. Chroma's technical report, Context Rot: How Increasing Input Tokens Impacts LLM Performance, evaluated 18 language models across controlled long-context tasks and observed that reliability generally became less consistent as input length increased. Even when tasks remained relatively simple, increasing the amount of surrounding context could make it harder for models to consistently identify and use the information required to produce the correct answer.
The practical implication is straightforward. A model with a 200,000-token context window does not necessarily possess 200,000 tokens of equally effective working memory. Developers still need to decide which information deserves to occupy that space.
Context therefore has an opportunity cost. Every irrelevant token consumes attention that could have been spent on something more useful.
From Static Context to Dynamic Context
One of the clearest architectural changes in modern AI agents is the transition from static context loading toward dynamic context discovery. Earlier agent architectures often attempted to prepare everything the model might need before inference began. A coding agent, for example, might receive the repository structure, semantically related files, project documentation, dependency metadata, tool definitions, previous conversation history, and coding conventions all at once.
The approach is convenient, but it assumes that developers can accurately predict which information the model will need before the task unfolds. In complex environments, that prediction becomes increasingly difficult. Most of the initial information may never become relevant, while important details that were not preloaded may only become apparent after the agent investigates the problem.
A context-engineered system can instead begin with a relatively small amount of information and allow the agent to discover additional context as required. The execution loop changes from a static process of collecting everything before inference to a dynamic process in which the model observes its environment, identifies information gaps, retrieves what it needs, and continues reasoning with a newly constructed context.
This pattern is visible in modern coding agents. Cursor described the transition in Dynamic Context Discovery, explaining how its agent harness increasingly moved away from providing large amounts of information up front and toward allowing models to retrieve relevant information themselves. Files, terminal output, conversation history, and tool definitions can become resources that the agent inspects when necessary rather than permanent context that must remain visible throughout the entire task.
This architectural shift has consequences beyond token efficiency. Dynamic discovery can reduce contradictory information, lower context noise, and allow the model's working state to remain focused on the current objective. Cursor reported that dynamic MCP tool loading reduced total token usage by 46.9 percent in runs that invoked MCP tools, although the exact impact naturally depends on the number and complexity of the available integrations.
The broader principle is more important than the percentage.
The agent is increasingly responsible for discovering its own context.
Tools Are Also Context
Tools are usually described as capabilities. A browser tool gives an agent access to the web, a database tool gives it access to structured data, and a filesystem tool allows it to inspect local resources. From the perspective of context engineering, however, every tool also introduces information that the model must understand before it can use that capability correctly.
A tool usually requires a name, description, argument schema, return format, and sometimes additional usage instructions. When an application exposes only a few tools, the resulting context overhead may be negligible. When an agent has access to dozens or hundreds of integrations, the cost can become substantial.
Anthropic documented this problem while introducing advanced tool use and dynamic tool discovery. In one configuration containing tools from GitHub, Slack, Sentry, Grafana, and Splunk, the full tool definitions consumed roughly 55,000 tokens before the agent had performed meaningful work. Anthropic also reported encountering configurations where tool definitions consumed approximately 134,000 tokens.
Its Tool Search architecture approaches the problem differently. Instead of loading every complete tool definition at the beginning of a task, the model receives a lightweight mechanism for discovering tools and can request detailed definitions only when they become relevant. In one example published by Anthropic, initial context consumption decreased from roughly 77,000 tokens to around 8,700 while preserving access to the larger tool ecosystem.
That example illustrates how context engineering extends beyond prompts and retrieval into API design itself. Tool names affect discoverability, descriptions affect selection, schemas affect usability, outputs affect context size, and the number of simultaneously visible tools changes the complexity of the model's decision space.
In other words, tool design is becoming part of reasoning architecture.
Anthropic develops this idea further in Writing Effective Tools for AI Agents, where tools are evaluated according to how effectively agents can understand and use them rather than according to conventional API design principles alone. A tool interface optimized for a human developer is not automatically an optimal interface for an autonomous model.
RAG Is Becoming Part of a Larger Context Pipeline
Retrieval-Augmented Generation was one of the earliest architectural responses to limited model context. Instead of inserting an entire knowledge base into a prompt, a retrieval system identifies relevant documents or fragments and places only selected information into the model's working context.
That architecture remains useful, but retrieval by itself does not solve the context problem. Selecting the correct document is only the beginning. A production system must determine which fragments should be retrieved, how large they should be, how competing results should be ranked, whether surrounding information is necessary, how duplicated information should be handled, and when previously retrieved material should leave the active context.
Anthropic investigated one part of this problem through Contextual Retrieval. Traditional chunking can remove information that explains how a passage relates to the larger document, making an otherwise relevant fragment difficult to interpret in isolation. Contextual Retrieval augments each chunk with information about its location and meaning within the original document before indexing it.
In Anthropic's experiments, contextualized embeddings combined with contextual BM25 reduced top-20 retrieval failure rates by 49 percent relative to the baseline configuration. Adding reranking reduced the failure rate by 67 percent. Those numbers are specific to the experimental setup, but the broader lesson is more general.
Retrieval is not simply search. Retrieval determines what information becomes part of the model's working reality.
A modern RAG architecture is therefore better understood as one stage of a broader context pipeline:
- Search
- Retrieval
- Filtering
- Ranking
- Compression
- Context Assembly
- Inference
The quality of the final output depends not only on the model and its prompt, but also on everything that happened before the selected information entered the context window.
Memory Is Not the Same as Context
Memory and context are often discussed as though they refer to the same concept, but the distinction becomes important as agents become more persistent. Context is the information currently available to the model during inference. Memory is information stored outside that active state that can potentially be retrieved later.
A useful system might maintain thousands of persistent observations about projects, previous decisions, user preferences, execution history, failed approaches, and external events. Injecting all of those memories into every request would defeat the purpose of storing them externally. Persistent memory therefore only becomes useful when the system can decide which memories deserve to become active context again.
The architecture is based on selective recall. Information remains outside the context window until a task creates a reason to retrieve it. Relevant memories are then surfaced, filtered, and inserted into the model's active working state, while unrelated memories remain external.
This becomes especially important for long-running agents. Anthropic's work on effective harnesses for long-running agents explores how agents can maintain continuity across separate context windows through structured artifacts, incremental state, compaction, and explicit handoffs between sessions. The agent does not need every historical token to remain permanently available. It needs a reliable mechanism for preserving what matters and reconstructing sufficient context when work resumes.
The distinction between memory and context therefore becomes architectural rather than semantic. Memory stores potentially useful state. Context determines what the model is allowed to think with right now.
Long-Running Agents Make Context Engineering Unavoidable
Context management becomes substantially more difficult when an agent operates for hours rather than seconds. A simple generation request may fit comfortably inside one model call, but a long-running coding agent can inspect hundreds of files, generate patches, run tests, encounter failures, analyze logs, modify its plan, and repeat that loop many times.
If every intermediate observation remained inside active context indefinitely, the working set would eventually become dominated by historical details. Old errors, outdated plans, redundant terminal output, superseded code, and resolved hypotheses would continue competing with current information.
Long-running agents therefore need mechanisms for deciding what survives. Some information must remain active, some can be summarized, some should be externalized, and some should disappear entirely.
Anthropic explored these challenges further in Harness Design for Long-Running Application Development. Its experiments used structures involving planner, generator, and evaluator agents to build complete applications over extended autonomous runs. Structured artifacts and controlled information transfer helped preserve relevant state without forcing the complete execution history into every subsequent model call.
At this point, context management begins to resemble state management in conventional systems engineering. Information must be retained, compressed, reconstructed, isolated, and sometimes deliberately discarded.
That is far beyond prompt wording.
Subagents Can Act as Context Isolation
Multi-agent systems are commonly justified through specialization and parallelism, but context management provides another reason to use them. Separate agents can create isolated working contexts that prevent unrelated information from contaminating a shared reasoning environment.
Imagine an agent that must investigate five independent technical questions. A single agent could research them sequentially, but every query, search result, failed hypothesis, and intermediate conclusion would accumulate inside one growing context. Alternatively, an orchestrator can delegate each investigation to a separate subagent and receive only the conclusions that matter.
Each subagent operates inside a bounded information environment. The exploratory history of one task does not automatically consume the working context of another task, and the orchestrator can combine structured outputs without inheriting every intermediate detail that produced them.
The pattern resembles process isolation in traditional computing. Separate processes maintain independent internal state and communicate through defined interfaces rather than sharing every piece of memory.
For complex agent architectures, context windows themselves become computational resources that need to be allocated deliberately.
Prompt Engineering Is Not Dead
The rise of context engineering does not make prompt engineering irrelevant. System instructions still need to communicate goals clearly, tool descriptions still require precise language, output constraints still need to be explicit, examples can still shape behavior, and ambiguous prompts can still produce unreliable results.
The difference is that prompts now operate inside a much larger system.
A useful conceptual relationship is:
Prompt Engineering ⊂ Context Engineering
Prompt engineering optimizes instructions. Context engineering optimizes the complete information state from which the model reasons.
That state includes the prompt, but it also determines which tools are visible, which documents are retrieved, which memories are recalled, which observations remain active, which outputs are compressed, which tasks are delegated, and which information disappears before the next inference step.
Prompt engineering remains necessary.
It is simply no longer sufficient.
The Optimization Target Is Signal
The central principle behind context engineering can be expressed simply: more context is not automatically better context. The goal should not be to fill the context window, but to construct the highest-value information environment for the next model decision.
Achieving that goal can require dynamic retrieval, selective memory, context compression, carefully designed tool schemas, isolated subagents, removal of stale observations, structured handoffs, and deliberate control over what enters and leaves the active working set. These mechanisms all address the same underlying problem: the model has finite attention, even when its nominal context capacity is extremely large.
The optimal architecture can also change as models improve. Cursor describes this evolution in Continually Improving Our Agent Harness, explaining how earlier versions of its agent provided substantial static context, including repository structures and semantically matched snippets, while later models became capable enough to discover more of that information independently.
This creates an unusual property for AI infrastructure. Some scaffolding exists primarily because current models are not yet capable of reliably managing a task without it. As model capabilities change, the surrounding context architecture must be reevaluated rather than treated as permanent.
Context engineering is therefore not a fixed recipe. It is an ongoing optimization discipline.
From Better Prompts to Better Information Environments
Prompt engineering taught developers that natural language could function as an interface to computation. Context engineering extends that idea by recognizing that language models do not reason from instructions alone. They reason from the complete information environment presented to them during inference.
For simple generation tasks, a carefully designed prompt may still be sufficient. For agents operating across repositories, databases, APIs, browsers, tools, memories, and long-running workflows, reliability increasingly depends on whether the right information appears at the right time, whether irrelevant information stays out, and whether useful state can survive without permanently occupying the active context window.
The shift is therefore not from prompts to something entirely different. It is a change in abstraction.
Prompt engineering optimizes the message.
Context engineering optimizes the model's working reality.
As AI systems become more autonomous, persistent, and deeply integrated with software environments, designing that reality is becoming one of the defining problems of modern AI engineering.