What Is Context Engineering? What the Model Receives Before It Answers
Context engineering is the design of what information a language model receives at inference time, in what form, in what order and for how long. It is broader than prompt engineering because the model context can include system instructions, user messages, retrieved documents, tool results, memory, current application state, examples, structured data and intermediate artifacts. The goal is not to maximize the number of tokens, but to construct the smallest useful context that preserves the information, constraints and evidence needed for the current task.
What context engineering really means
Every model call is made under a temporary working environment: the current instructions, messages, retrieved evidence, tool outputs and state that fit into the active context window. Context engineering is the discipline of constructing that environment deliberately.
The key word is deliberately. A naive system simply concatenates everything it has: full history, all retrieved documents, every tool response and large system prompts. A context-engineered system decides which information is required for the current decision and which information should remain outside the window until needed.
This makes context engineering partly an information-architecture problem, partly a runtime problem and partly an evaluation problem. The design must decide what can enter context, where it comes from, which version is current, how conflicts are resolved, how much detail is retained and how the result is tested.
The simplest example
Imagine an internal support assistant. A user asks: “Can this customer cancel without a fee?”
The model might need five things: the current cancellation policy, the customer's current contract type, the effective contract date, the relevant exception rules and the user's authorization scope.
It does not necessarily need the entire customer database, the full policy archive, every previous conversation or every support ticket. Context engineering is the process that selects and assembles the five useful pieces while excluding unrelated information.
From application state to model context
Where the simple example stops
Real systems are more difficult because the information needed for one step may not be known before execution begins. An agent can discover new facts through tools, create intermediate files, receive changing external state or span a task longer than one context window.
Context engineering therefore becomes dynamic. The context for step 12 should not simply be step 1 context plus eleven layers of accumulated output. It should reflect the current task state, the decisions that still matter and the evidence required for the next action.
What can enter a model context?
| Context component | Purpose | Typical risk |
|---|---|---|
| System / developer instructions | Define role, constraints, policies and behavior | Too vague, contradictory or overloaded with brittle logic |
| Current user request | Defines immediate task and intent | Ambiguity or conflict with prior history |
| Conversation history | Preserves continuity across turns | Stale assumptions, repetition and token growth |
| Retrieved documents | Provide external knowledge/evidence | Irrelevance, stale versions, weak authority or duplication |
| Current application state | Supplies volatile business/system facts | Using cached or remembered state instead of current authority |
| Tool definitions | Tell the model what capabilities exist and how to call them | Too many overlapping tools or verbose schemas |
| Tool results | Bring observations from the environment into the loop | Large noisy outputs, untrusted content or obsolete observations |
| Memory | Reintroduces selected information from previous interactions | Staleness, incorrect generalization or over-personalization |
| Examples | Demonstrate desired behavior | Too many edge cases can crowd out the current task |
| Intermediate artifacts | Carry plans, summaries, code, calculations or notes | Old intermediate state may be mistaken for final truth |
| Policies / guardrails | Define prohibited or constrained behavior | Conflict with business logic or hidden enforcement gaps |
Context engineering vs prompt engineering
Prompt engineering and context engineering solve different layers
| Prompt engineering | Context engineering | |
|---|---|---|
| Primary focus | ||
| Typical scope | ||
| When it changes | ||
| Typical failure | ||
| Relationship |
Anthropic explicitly describes context engineering as the natural progression of prompt engineering for systems in which the model must work with tools, external data, message history and long-running agent state. The practical distinction is useful because a perfectly written prompt cannot compensate for missing authoritative data or a context polluted by contradictory state.
Context engineering vs retrieval
Retrieval selects candidate information from an external corpus or source. Context engineering decides what happens after and around that retrieval.
The retriever may return 30 passages. A reranker may reduce them to 10. The context layer may select four passages, remove duplicates, attach source/version metadata, combine them with current application state and place them after the system instructions.
This is why a RAG system can retrieve the correct passage and still answer badly: the failure may occur during context assembly rather than retrieval.
Context engineering vs memory
Memory is information preserved outside the immediate model invocation so it can be used again later. Context is the information actually loaded into the current invocation.
A memory system may contain thousands of facts, notes or prior decisions. Context engineering selects which of those should be reintroduced for the current task. Loading all memory on every turn defeats the purpose of having an external memory layer.
The distinction becomes crucial for volatile state. A remembered project status or user preference can be useful, but current authoritative state may need to be re-read before a consequential decision.
Context engineering vs application state
Application state is the current condition of the outside system: account balance, ticket status, file version, workflow stage, deployment state or task progress.
State can be summarized into context, but the summary is not the state itself. For consequential operations, the runtime may need to re-read the authoritative system immediately before the action rather than trust an earlier model-visible snapshot.
Tool design is part of context engineering
Tools do more than give agents capabilities. Tool names, descriptions, schemas and results become model-visible information that shapes decisions.
Anthropic's current context-engineering guidance emphasizes token-efficient tools and warns against bloated tool sets with overlapping functionality. A tool catalog that is difficult for a human to distinguish is also difficult for a model to route reliably.
Tool outputs also need context discipline. Returning an entire 20,000-line log when the agent requested one error condition consumes attention and can bury the decisive evidence.
Just-in-time context vs preloaded context
Two ways to supply information
| Preloaded context | Just-in-time context | |
|---|---|---|
| Method | ||
| Strength | ||
| Risk | ||
| Useful when |
Anthropic describes a hybrid pattern in which some stable context is preloaded while agents retrieve additional information at runtime. This is a useful architecture pattern because not every important fact deserves permanent residency in the context window.
Context is a budget, not a storage system
A context window defines capacity. It does not guarantee that every token will be used equally well. The model must distribute attention across instructions, history, evidence, tools and intermediate state.
The practical objective is therefore not “fill the window.” It is to maximize the utility of the limited attention budget.
Anthropic formulates a similar principle as finding the smallest high-signal set of tokens that maximizes the probability of the desired behavior. OpenAI's context-management guidance likewise warns that uncurated history, redundant tool results and noisy retrieval can overwhelm even large windows.
Why more context can be worse
Additional context can introduce irrelevant information, stale state, duplicate evidence, contradictory instructions or positional competition. It can also cause compaction systems to discard details that later become important.
The classic Lost in the Middle study demonstrated that long-context models can use information differently depending on where relevant content appears, with performance often degrading when decisive information is placed in the middle of long inputs.
This does not mean long context is inherently bad. It means availability inside the window is not the same as reliable utilization.
Context ordering should be intentional
Context construction is also an ordering problem. Critical instructions, current state, decisive evidence and task-specific constraints should not be concatenated arbitrarily.
There is no universal perfect ordering for every model and task. The architecture should therefore test whether reordering evidence changes correctness and whether important information remains robust across realistic context variations.
A stable answer that changes dramatically when two equally valid passages swap positions indicates context sensitivity that should be measured rather than ignored.
Conflicting context needs explicit precedence
A model may receive an old policy and a new policy, a remembered preference and a current explicit instruction, or a cached status and a live API result. The system should not expect the model to infer precedence from prose style.
Context engineering should encode precedence through source selection, metadata, ordering or explicit instructions: current authoritative state overrides stale copies; explicit current user instruction overrides older inferred preference; approved policy supersedes obsolete drafts.
| Conflict | Preferred context rule |
|---|---|
| Current state vs remembered state | Refresh and prefer the authoritative current source. |
| Current policy vs superseded policy | Include current version; keep old version only when historical comparison is required. |
| Explicit user instruction vs old inferred preference | Prefer the current explicit instruction. |
| Primary source vs secondary summary | Use primary source for claims that require authority; summary may support explanation. |
| Tool observation vs model prior | Prefer current observed state when the tool is authoritative for that fact. |
| Two unresolved authoritative sources | Expose the conflict rather than fabricating one consistent answer. |
Compaction is context transformation, not lossless storage
Long-running systems eventually need to trim, summarize or compact history. Compaction creates a new representation of prior context so the agent can continue without replaying every token.
OpenAI's context-management examples use trimming and compression for long-running sessions. Anthropic describes compaction as a primary technique for maintaining coherence when an interaction approaches the context limit.
The difficult part is deciding what cannot be safely removed: unresolved tasks, identifiers, user constraints, security boundaries, architecture decisions, exceptions, source provenance and the conditions that make a previous conclusion valid.
Preserve validity boundaries
Important conclusions should carry the conditions under which they remain supported: version, date, scope, assumptions, source authority and unresolved disagreement.
Context engineering is therefore connected to the Answer Validity Boundary. The context assembler should not strip away the metadata that determines whether evidence still applies.
Context engineering is also a security boundary
Data that reaches the model has crossed an important system boundary. Context assembly must therefore respect authorization, tenant isolation, confidentiality and data-minimization rules.
A retriever may technically find a passage the current user cannot access. The correct design is to prevent that passage from entering model context rather than rely on the model to ignore it.
Tool outputs can also contain untrusted instructions or adversarial content. Context engineering should preserve the distinction between application instructions and external data so retrieved text cannot silently acquire instruction authority.
A practical context-engineering architecture
| Layer | Responsibility |
|---|---|
| Authoritative systems | Own current business/system state and official records. |
| Knowledge sources | Own documents, policies, specifications, research or external evidence. |
| Memory store | Preserves selected information across turns or sessions. |
| Retrieval layer | Locates task-relevant candidates from external sources. |
| Tool/runtime layer | Reads state, performs actions and returns observations. |
| Context assembler | Selects, filters, deduplicates, orders and formats model-visible information. |
| Model | Reasons and generates over the assembled context. |
| Validation/evaluation | Checks whether selected context and resulting output satisfy task-specific requirements. |
The context assembler is conceptually important even when no module has that exact name. In a small application it may be ordinary application code. In a large agent platform it may combine session management, retrieval, memory, tool middleware, compaction and policy enforcement.
A practical context construction policy
| Rule | Why it matters |
|---|---|
| Start from the current task | Do not carry information merely because it existed earlier. |
| Re-read volatile state | Memory and old context can be stale. |
| Retrieve just enough evidence | Large candidate sets can dilute decisive information. |
| Preserve source metadata | Version, date and authority determine whether evidence still applies. |
| Remove duplicate content | Redundancy consumes tokens without adding information. |
| Prefer structured summaries for large tool output | Expose decisive fields instead of raw noise where fidelity permits. |
| Keep rules with exceptions | Separating a rule from its exception creates false certainty. |
| Make precedence explicit | Do not ask the model to infer which conflicting source wins. |
| Keep durable state outside context | Context is temporary working memory, not the database. |
| Compact with retention tests | Verify that identifiers, constraints, provenance and unresolved state survive. |
| Measure order sensitivity | Correctness should not depend accidentally on arbitrary document ordering. |
| Evaluate context separately from model quality | A stronger model cannot compensate reliably for missing or unauthorized evidence. |
How to evaluate context engineering
| Property | Question | Example test |
|---|---|---|
| Sufficiency | Does the context contain everything required to solve the task? | Remove one evidence item and observe whether the answer becomes unsupported. |
| Relevance | How much context is unnecessary for the task? | Measure quality as irrelevant passages are added or removed. |
| Authority | Are decisive claims grounded in the correct source class? | Inject a more fluent but non-authoritative conflicting source. |
| Freshness | Does current state override stale copies? | Change authoritative state after a previous turn and rerun. |
| Position robustness | Does answer quality depend strongly on evidence position? | Randomize candidate ordering across repeated trials. |
| Conflict handling | Does the model follow explicit precedence rules? | Present old and new state together. |
| Compaction retention | Does summarization preserve constraints and validity boundaries? | Compare pre/post-compaction task performance. |
| Token efficiency | Does extra context improve quality enough to justify latency/cost? | Run controlled context-size ablations. |
| Security | Can unauthorized or adversarial content enter model context? | Test tenant, permission and prompt-injection boundaries. |
Context assembly is a distinct RAG failure layer
A RAG pipeline can succeed at retrieval and still fail downstream. The relevant source may appear at rank 2, yet the context assembler can drop it, truncate it, combine it with stale contradictory material or exceed the token budget.
This is why retrieval traces should be compared with the actual context sent to the model. Without that comparison, context failures are easily misdiagnosed as embedding or model failures.
Original implementation evidence
Source of Truth Research Engine: bounded research instead of unlimited context
The Source of Truth Research Engine separates discovery, acquisition, extraction, verification, contradiction analysis and synthesis into bounded research stages instead of sending one huge research task and all accumulated material into a single model call.
Its evidence model stores Sources, Artifacts, Claims, Relations, Contradictions and provenance outside the model context. The model can receive the subset needed for the current research step while durable evidence remains in the external store.
That is a concrete context-engineering pattern: durable research state lives outside the model window; the active model context is reconstructed for the current stage.
Aaasaasa AI Client: runtime, permissions and context are separate concerns
Aaasaasa AI Client separates provider/model selection, runtime location, workspace permissions, local resources and tool access. This prevents the model context from becoming the owner of authorization or application state.
Direct Chat and agentic runtimes can have different tool capabilities. Workspace permission profiles are enforced by the runtime rather than merely described in natural-language context. This distinction is important: context can tell a model what it should do, while the runtime must still enforce what it is actually allowed to do.
The implementation evidence here is architectural separation, not a claim that every advanced context-management technique described in this article is already implemented.
| Implementation pattern | Context-engineering lesson |
|---|---|
| External evidence store | Durable knowledge does not need to remain in the model window. |
| Bounded research stages | Different steps can receive different context instead of accumulating one giant history. |
| Claims + provenance outside context | Evidence identity survives beyond temporary inference state. |
| Runtime-enforced permissions | Security authority does not depend on the model remembering an instruction. |
| Separate local/provider/model/runtime concepts | Context is only one layer of the wider AI application architecture. |
Common context-engineering failure modes
| Failure mode | What goes wrong |
|---|---|
| Replay the entire conversation forever | Old assumptions, repetition and token growth overwhelm current intent. |
| Put every retrieved result into the prompt | Noise, duplication and conflicting versions dilute decisive evidence. |
| Use memory as current state | Stale information silently replaces authoritative live state. |
| Return raw tool output | Large logs or responses consume attention without adding decision value. |
| Hide tool descriptions behind vague names | The model cannot reliably decide which capability to use. |
| Compact without retention tests | Critical constraints, identifiers or exceptions disappear. |
| Mix instructions and untrusted data | External content can be interpreted as higher-authority instruction. |
| Use one static context template for every task | Different tasks receive irrelevant information and miss task-specific evidence. |
| Ignore source version/date | Stale but relevant evidence can dominate current authoritative state. |
| Treat a larger context window as a quality guarantee | Capacity increases while attention and conflict problems remain. |
Common misconceptions
| Misconception | Correction |
|---|---|
| “Context engineering is just prompt engineering with a new name.” | Prompts are one component; context engineering also covers retrieval, memory, state, tool results, history and compaction. |
| “Context means chat history.” | History is only one possible context source. |
| “More context is always better.” | Additional information can reduce signal, introduce conflicts and increase cost. |
| “If retrieval found it, the model saw it.” | Retrieved candidates can be filtered, truncated or omitted before inference. |
| “Long context removes the need for RAG.” | Large windows increase capacity but do not solve freshness, authority, permissions or dynamic retrieval. |
| “Memory should always be loaded.” | Memory should be selected according to the current task. |
| “A summary preserves everything important.” | Compaction is lossy unless explicitly evaluated for retention. |
| “Instructions can enforce permissions.” | Authorization must be enforced by runtime/application controls, not only by context. |
| “One context recipe works for every model.” | Context sensitivity varies by model, task, corpus and runtime. |
| “Context engineering is only for agents.” | Agents amplify the need, but ordinary RAG and conversational applications also require context construction. |
A practical context-engineering sequence
Construct context from the current decision backward
Context-engineering checklist
| Question | Expected answer |
|---|---|
| What exact decision will the model make next? | A bounded task, not a vague long-term objective. |
| Which information can materially change that decision? | Explicit minimum evidence/state set. |
| Which data is authoritative now? | Current source/version and freshness rule. |
| Which data is optional background? | Separated from decisive evidence. |
| What must not enter context? | Unauthorized, unnecessary or overly sensitive data. |
| Which memory items are relevant? | Selected by task, not replayed automatically. |
| Which tool outputs should be reduced? | Large responses are transformed into decision-relevant form. |
| Which constraints must survive compaction? | Identifiers, exceptions, obligations, unresolved state and provenance. |
| How is precedence represented? | Current/authoritative information can reliably override stale or weaker sources. |
| How will you know context failed? | Context-specific evals and traces exist. |
| Can the answer be reproduced? | Model input or reconstructable context trace is available where appropriate. |
| Can a stronger or larger model change the strategy? | Context policy is version-aware and reevaluated empirically. |
Edge cases and limitations
Some tasks are simple enough that context engineering reduces to a short system prompt and one user message. Adding retrieval, memory and compaction would only introduce unnecessary architecture.
Some tasks require high recall and may intentionally include more context before later synthesis. Research, discovery and legal review can prefer omission avoidance over minimal token count.
Some information should never be summarized before use. Exact contracts, code, cryptographic material, numerical records and regulatory text may require verbatim or structured retrieval where compression could alter meaning.
Long-context behavior varies substantially between models. A strategy validated on one model, context length or tool harness should not automatically be transferred to another.
The model can still ignore or misinterpret excellent context. Context engineering improves the information environment; it does not guarantee reasoning correctness.
What would change this answer?
Future models may become more robust to long context, positional effects and conflicting information. That could reduce the amount of manual curation required.
The architectural distinction would still remain useful because permissions, freshness, memory persistence, source authority and external application state exist outside the model regardless of context-window size.
The recommended balance between preloaded and just-in-time context also changes with latency requirements, tool reliability, corpus size, model cost and how dynamic the underlying information is.
Related canonical knowledge
Context engineering sits between retrieval and generation. RAG explains how external knowledge is retrieved; R01 separates embeddings, vector search and reranking; context engineering explains what eventually reaches the model.
Source-of-Truth architecture answers a different question: not which information is present in context, but which source is authorized to establish a claim.
The existing article Why More Context Can Make AI Answers Worse is the diagnostic companion to this canonical definition. It focuses on context pollution, position effects, top-k growth, compaction loss and answer degradation rather than redefining context engineering itself.
Frequently asked questions
Context engineering FAQ
What is context engineering?
How is context engineering different from prompt engineering?
Is RAG the same as context engineering?
Is memory the same as context?
Why can more context make an answer worse?
What is context compaction?
Should current application state be stored in context?
Is context engineering only needed for AI agents?
Glossary
Key context-engineering terms
- Context engineering
- The design and runtime management of the information supplied to a language model for a particular inference step.
- Context window
- The model's finite token capacity for the input and, depending on the model interface, associated generated tokens or active sequence.
- Prompt engineering
- The design of instructions, examples and prompt structure intended to elicit useful model behavior.
- Context assembly
- The process of selecting, filtering, ordering and formatting model-visible information before inference.
- Just-in-time retrieval
- Loading information dynamically when the current task requires it instead of preloading all potentially relevant data.
- Compaction
- Reducing accumulated context into a smaller representation while attempting to preserve information needed for future steps.
- Context pollution
- Degradation caused by irrelevant, stale, contradictory or redundant information occupying the model's working context.
- Application state
- The current authoritative condition of the external system, workflow or domain that exists independently of the model context.
- Memory
- Information stored outside the immediate model invocation for possible use in later turns or sessions.
- Retrieved context
- External information selected by a retrieval system and made available, wholly or partly, to the model.
- Position robustness
- The degree to which model correctness remains stable when the location or order of relevant context changes.
- Validity boundary
- The scope, time, assumptions, versions and evidence conditions within which a conclusion remains supported.
Conclusion
Context engineering is the layer that decides what the model gets to see before it answers. That makes it broader than prompting and downstream of retrieval, while remaining distinct from durable memory and authoritative application state.
A strong context architecture does not treat the context window as a database. It keeps durable state and knowledge outside the model, loads what is required for the current decision, preserves authority and provenance, removes unnecessary noise and refreshes volatile information when needed.
The practical objective is therefore not maximum context. It is minimum sufficient, high-signal, correctly authorized and validity-preserving context for the next model decision.
Primary sources and current guidance
The sources below support the current context-engineering terminology, long-context behavior and operational context-management patterns. Project sections are explicitly implementation evidence rather than universal claims.
Anthropic — Effective context engineering for AI agentsOfficial engineering guidance defining context engineering, just-in-time retrieval, compaction, structured memory and context curation for agents.
OpenAI — Context Engineering: Short-Term Memory Management with SessionsOfficial cookbook guidance on context management, trimming and compression for long-running agent sessions.
OpenAI — Agents guideCurrent OpenAI developer guidance on agent runtimes, context across steps and orchestration ownership.
Lost in the Middle: How Language Models Use Long ContextsResearch showing that long-context model performance can depend strongly on the position of relevant information in the input.
Related Articles
Why More Context Can Make AI Answers Worse
A larger context window does not guarantee a better answer. This article explains how signal dilution, conflicting evidence, stale state, position sensitivity, and lossy compression can reduce AI reliability—and introduces a practical Context Pressure Test.
Front- and Backend Development
Front-end and back-end development is an essential part of web development and involves the creation of web applications and websites. Front-end development focuses on the user interface, while back-end development is responsible for programming and managing the server side.
AI Agent Memory Is Not RAG: How to Separate Memory, Retrieval, State and Context
Agent memory, RAG, state, and context are often used as if they were interchangeable. They are not. This practical architecture model separates the four layers, shows where each belongs, and explains what breaks when systems collapse them into one.
The GPU Is Not the Product: Future-Proof Private AI Architecture
Private AI infrastructure should not be designed around one GPU or one model. A more resilient approach combines fast inference GPUs, memory-rich AI systems, physical-AI nodes and optional frontier cloud models behind a capability-aware routing layer.
git-with-automatic-upload-and-synchronization-to-a-production-server
Mastering the SEO Workflow: Essential Optimization Strategies for Organic Growth
A structured SEO workflow is crucial for sustainable organic growth. Learn the ten foundational strategies, from keyword research and technical optimization to content quality and performance analysis.
Qwen 3.6 in Production: Release Runbook, AI Rollback, and LLMOps Versioning
Qwen 3.6 is not just another model upgrade. It is a release event, a rollback scenario, and a versioning problem at the same time. This article explains how Qwen 3.6 should be handled in production through LLMOps discipline, prompt and model traceability, controlled rollout, and evidence-based rollback readiness.
Vector Databases, Embeddings and Reranking: Three Different Parts of Retrieval
Embeddings represent meaning, vector databases retrieve candidates, and rerankers refine results. Learn how these three retrieval layers differ and work together in RAG.
The Answer Validity Boundary: The Missing Layer Between Relevance and Reliable AI Answers
A source can be relevant, authoritative and still be wrong for the question being asked. The missing layer is applicability: the conditions under which an answer holds, and the changes that force it to be reconsidered. This article introduces the Answer Validity Boundary as a source-design pattern for humans, AI search and RAG systems.
MLOps vs LLMOps: What Changes When the Model Is an LLM
MLOps operates machine-learning systems; LLMOps extends those practices to prompts, context, retrieval, providers, tools, evaluations and runtime behavior around large language models.
New Qwen 3.5-Plus: Open-source AI is getting serious now
Discover the groundbreaking features and benefits of Alibaba's Qwen 3.5-Plus, a revolutionary open-source AI for developers.
RBAC vs Tenant Isolation: Two Different Security Boundaries
RBAC controls what a user may do; tenant isolation controls which tenant’s resources that action may reach. Learn why multi-tenant SaaS security requires both boundaries.