For two years everyone obsessed over the perfect prompt. Then teams building real agents noticed the prompt was rarely the problem. The problem was what surrounded it. The model needed a fact it never received, or it drowned in ten documents when it needed one paragraph. The skill that fixes that has a name now: context engineering. Andrej Karpathy and others have argued it is the more accurate description of what building with LLMs actually involves.
What Is Context Engineering?
Short answer. Context engineering is the practice of assembling the right information, tools, and instructions in a model’s context window so it can complete a task reliably. It designs the whole input (what to retrieve, what tools to expose, what history to keep, what to leave out) rather than just the wording of one instruction.
The mental model: a context window is scarce, expensive real estate, and your job is to fill it with exactly what the model needs for the next step and nothing else. Too little and it guesses; too much and it loses the thread and your costs balloon. Context engineering is the curation discipline that sits between your data and the model.
Context Engineering vs Prompt Engineering
They are not rivals; prompting is a subset. But the emphasis has shifted, and the distinction is worth being precise about:
| Context engineering | Prompt engineering | |
|---|---|---|
| Focus | The whole input | One instruction |
| Covers | Knowledge, tools, memory, format | Wording and examples |
| Main risk | Wrong or noisy context | Ambiguous phrasing |
| Scales to | Multi-step agents | Single responses |
A flawless prompt over missing context still fails. That is why, as systems moved from single replies to agents that loop, context engineering became the dominant concern.
Why Agents Need It Most
A chatbot answers once. An agent runs a loop, and every pass adds tool outputs, observations, and conversation history to the window. Left unmanaged, that pile grows until the signal is buried, the model starts contradicting itself, and each step costs more than the last.
Where RAG Fits
Short answer. Retrieval-augmented generation (RAG) is one technique inside context engineering: fetching relevant documents at query time and placing them in the window. It answers “what knowledge do I pull in.” Context engineering is the broader practice that also handles tools, memory, formatting, and what to exclude.
RAG gets the headlines, but it is a tool, not the whole job. You still have to decide what is worth retrieving, how to chunk it, how much to include, and how to format it next to the instruction and the tool definitions. Treat RAG as the retrieval step within a wider design, not as a synonym for the discipline.
Why Structure Upstream Makes This Easier
Here is the part most context-engineering advice skips: retrieval is only as good as what you retrieve from. If your knowledge is a pile of unstructured docs, even great retrieval drags in noise. If it is structured into discrete, linked concepts, an agent can pull the exact piece it needs. That is the bet behind Google’s Open Knowledge Format: knowledge as navigable concept files instead of one big blob. Good structure upstream is the cheapest context-engineering win there is.
The fastest way to improve an agent’s context is to give it a clean source of truth. TinyTables keeps your data structured and current as work happens, so what your agent retrieves is signal, not sludge. Free to start, no code.
Give your agents clean contextFrequently Asked Questions
What is context engineering?
Context engineering is the practice of assembling the right information, tools, and instructions in a model's context window so it can do a task reliably. Where prompt engineering tunes the wording of a single instruction, context engineering designs the whole input: what knowledge to retrieve, what tools to expose, what history to keep, and what to leave out. It is now the core skill for building agents.
What is the difference between context engineering and prompt engineering?
Prompt engineering is about phrasing one instruction well. Context engineering is about everything around it: which documents to pull in, which tools to offer, how much conversation history to retain, and how to format it all. A perfect prompt over missing or messy context still fails. For agents that run many steps, context engineering is the bigger lever by far.
Why does context engineering matter for AI agents?
Agents run in loops and accumulate context fast. Each step adds tool results, observations, and history. Without deliberate management, the window fills with noise, costs rise, and quality drops. Good context engineering keeps the right facts in view at each step and the rest out, which is the difference between an agent that stays coherent over a long task and one that drifts.
How does RAG fit into context engineering?
Retrieval-augmented generation (RAG) is one technique within context engineering: fetching relevant documents at query time and placing them in context. It answers the 'what knowledge do I pull in' question. Context engineering is the broader discipline that also covers tools, memory, formatting, and what to exclude. RAG is a tool in the kit, not the whole kit.
How do formats like OKF help with context engineering?
The hardest part of context engineering is having clean, structured knowledge to retrieve from in the first place. Formats like the Open Knowledge Format store knowledge as linked concept files an agent can navigate precisely, pulling the one concept it needs instead of dumping a whole document into the window. Good structure upstream makes good context assembly downstream much easier.