Context engineering: what to put in an AI agent's context and why

Context engineering is the practice of deciding which information an AI model sees at each step, so it has what it needs for the task and nothing that distracts it. This guide explains why it matters more than prompt wording, and shows concrete techniques: memory files, scoped rules, just-in-time retrieval, compaction, notes and subagents.

What is context engineering?

Context engineering is the work of choosing what goes into a language model's context window at each step of a task. Anthropic defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference" (Anthropic Engineering).

The context window is everything the model sees when it produces its next output:

• the system prompt and your instructions,
• tool definitions,
• files, documents or search results you loaded,
• the conversation so far, including every tool call and result,
• any memory or notes carried over from earlier sessions.

For a single chat question, that is mostly your prompt. For a coding agent that runs fifty steps, your prompt is a small slice; the rest accumulates as the agent reads files and runs commands. Context engineering is how you keep that growing pile useful. The guiding rule from Anthropic's article: find "the smallest set of high-signal tokens that maximize the likelihood of your desired outcome."

Why does context matter more than clever prompts?

Context matters more because the model can only act on what it sees, and what it sees degrades as the window fills. A perfectly worded prompt fails if the agent never loaded the file that holds the answer, or if that file is buried under 200,000 tokens of old logs.

Two effects drive this:

• Context rot. As the number of tokens grows, the model's ability to recall information from that context decreases. Anthropic notes this affects all models, to varying degrees.
• A finite attention budget. Every token competes for the model's attention. Irrelevant tokens dilute the relevant ones.

Large windows help but don't remove the problem. At the time of writing, current Claude models (Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1) all have a 1M token context window (models overview). That is room for a large codebase, not a reason to load one wholesale.

There is also a cost angle. Every token in context is paid for on every turn it stays there. A stable, well-ordered prefix can be cached cheaply (cache reads cost 5% of the base input price on Opus 5.5 and Sonnet 5.5), while a bloated, constantly changing context is paid at full price again and again. Our guide to reducing LLM API costs covers caching in detail.

What goes into an AI agent's context?

An agent's context has a few layers, and each one needs a different strategy. The table shows what each layer is for and the common mistake.

The rest of this guide takes these layers one at a time.

How do you write the always-loaded instructions?

Keep always-loaded instructions short, specific and current, and move anything that only matters sometimes into files that load on demand. These tokens are paid for in every session and compete with every task.

Claude Code's documentation gives concrete numbers (memory docs):

• Target under 200 lines per CLAUDE.md file. Longer files consume more context and reduce adherence.
• Imports (@path/to/file) organize a long file but don't reduce its cost, because imported files also load at launch.
• If two instructions contradict each other, Claude may pick one arbitrarily.
• CLAUDE.md files in subdirectories load only when Claude works with files there.

Anthropic describes the target as the "right altitude": between brittle, hard-coded logic and vague, high-level guidance. Compare:

Write clean, well-tested code and follow best practices.

Run `npm test -- --run` before finishing. Use Zod for request validation insrc/api. Never edit files in src/generated; run `npm run codegen` instead.

For rules that apply to one part of the codebase, use path-scoped rules. In Claude Code, a markdown file in .claude/rules/ with a paths field loads only when Claude reads or edits a matching file:

---paths:  - "src/api/**/*.ts"---## API rules- Validate every request body with the schemas in src/api/schemas.- Return errors in the { error: { code, message } } format.- Add an integration test in tests/api for every new route.

Our guide to CLAUDE.md best practices goes deeper on structure and examples.

How do you load context just in time?

Load data when the agent needs it, through tools, instead of pasting everything in at the start. The agent keeps lightweight references (file paths, URLs, query names) and fetches the content at the moment it becomes relevant.

This is how good coding agents already work. Claude Code doesn't load your repository into context; it searches with grep and glob, reads the files that match and ignores the rest. You can help it:

• Point, don't paste. "The bug is in the checkout flow; start from src/checkout/submit.ts" beats pasting five files.
• Name the source of truth. "The API contract is openapi.yaml" tells the agent where to look before guessing.
• Give logs as files. Save a 3,000-line log to logs/failure.txt and ask the agent to search it, instead of pasting it into the chat.
• Prefer narrow tools. A tool that returns the one Sentry issue you need beats one that dumps the last 500 errors.

For knowledge outside the codebase (product docs, support tickets, policies), retrieval augmented generation does the same job at larger scale: search an index, load the top matches. See what RAG is for a working example.

The trade-off is speed. Just-in-time loading takes extra tool calls. For small, always-relevant facts, putting them in the instructions file is faster. For anything large or rarely needed, load on demand.

How do you manage context in long agent sessions?

Long sessions need active cleanup: summarize old history, persist important state outside the window, and isolate side quests. Anthropic's article names three techniques for this, and Claude Code has commands for each.

Compaction

Compaction summarizes the conversation and continues from the summary. Claude Code compacts automatically as the window fills, and you can trigger it yourself with focus instructions (commands reference):

/compact Keep the failing test names, the root cause we found in the sessionmiddleware, and the remaining TODOs. Drop the dependency investigation.

Related commands:

A good habit: /clear between unrelated tasks. Leftover context from task A is noise for task B.

Structured notes

The agent writes notes to a file outside the context window and reads them back later. A PROGRESS.md with the plan, decisions, completed steps and open questions lets an agent pick up after compaction, a crash or a new session without re-deriving everything. Claude Code's auto memory does a version of this for preferences and corrections, loading the first 200 lines or 25KB each session.

Before you start, create PROGRESS.md with the plan as a checklist. After eachstep, tick it off and add one line about any decision you made and why.If you lose context, read PROGRESS.md first.

Notes matter most when nobody is watching. If agents run unattended overnight, on your own server or a hosted box such as Orkio, a progress file is how you see in the morning what happened and why.

Subagents

A subagent works in its own clean context window and returns a short summary. Anthropic notes that a subagent may explore extensively but returns a condensed summary, often 1,000 to 2,000 tokens. Use subagents for research, broad code searches or reviewing many files, so the main session receives the conclusion instead of every file it took to get there. Our guide to Claude Code subagents shows how to define them.

How do tools and MCP servers affect context?

Every tool adds its name, description and schema to the context, and every tool result lands in the history. Too many tools, or tools that return huge outputs, crowd out the actual task.

Practical rules:

• Fewer, clearer tools. Anthropic's guidance is that tools should be self-contained and extremely clear about their intended use. If a human can't say which of two tools applies, the model can't either.
• Load tools on demand. Claude Code's tool search is on by default, so MCP tool definitions load when needed rather than all at once (MCP docs).
• Cap output. Claude Code warns when an MCP tool result passes 10,000 tokens and caps MCP output at 25,000 tokens by default (MAX_MCP_OUTPUT_TOKENS changes it).
• Return summaries, not dumps. If you write tools, return the 20 relevant rows with a count of the rest, not 5,000 rows.

Choosing which servers deserve a slot is covered in our list of the best MCP servers.

What are the most common context engineering mistakes?

Most context problems come from adding too much rather than too little. Here are the patterns that cause the most trouble and their fixes.

For the instruction-writing side of the same problem, see our prompt engineering guide.

How to start engineering context today

You can improve an agent's context in an afternoon. Work through these steps on one real project.

1. Run /context in your next Claude Code session and note what takes the most space.
2. Trim your CLAUDE.md to under 200 lines: build, test and lint commands, conventions, layout, and the few rules people keep breaking.
3. Move area-specific rules into .claude/rules/ files with paths.
4. Stop pasting large files and logs; reference paths and let the agent search.
5. Start long tasks by asking for a PROGRESS.md checklist, and compact with explicit focus instructions.
6. Remove MCP servers you didn't use this week.
7. After a week, compare: fewer corrections, fewer repeated explanations and lower cost per task mean the context is doing its job.

Frequently asked questions