Context Engineering: When AI Can't Fit Everything in Its 'View'
> Source: commit 9621a05 | easy-learn-ai project daily update
1. A Story About a Tray
Imagine walking into a busy restaurant where a waiter carries a small tray that holds only four or five plates. The kitchen is firing ten dishes at once, but the tray can only carry part of them. The waiter must decide: which dishes go first? Which can wait? Which can be stacked to save space?
When an AI works, it faces the same tray.
Each time you talk to an AI, it 'sees' more than just your latest message. It also sees your conversation history, the files you uploaded, and hidden system instructions and background knowledge — this whole package is the information 'in front of it' for that task.
The problem is that this 'view' has limited capacity. Like the tray that can't hold ten dishes, AI's view can't hold everything. Too little and it has no material, so it makes things up; too much and it loses focus, drifts, or squeezes out important information.
The craft of carefully arranging this 'view' is called context engineering.
2. Why the 'View' Can't Hold Everything
Think of an AI as a student taking an exam. On the desk in front of them they can spread only a limited set of reference materials. The size of this desk is, in technical terms, the context window — the maximum amount of text an AI can process at once.
Early AI models had tiny desks — a few pages. Today's models can hold an entire book, but the desk is still finite. And the bigger the desk, the harder it is for the AI to find things — like searching a huge library: the larger it is, the longer it takes.
What's on the desk isn't just what you provide. It includes:
- Your question — the protagonist
- Relevant materials — uploaded documents, retrieved web pages
- Chat history — everything discussed so far
- System prompts — hidden instructions on how to answer
- Recency — recent matters more than ancient. Place the latest information at the end of the context (AI is more sensitive to endings); compress or remove old content.
- Relevance — directly relevant material goes in; marginal material stays on standby outside; irrelevant material is removed outright.
- Density — express the same meaning more concisely. Cut filler, repetition, and decoration; keep core facts.
- Structure — clear formats (lists, tables, hierarchies) are easier for AI to understand and use than walls of text.
Together these easily overflow. Once they do, the AI either refuses to answer or starts 'forgetting' — pushing out the earliest content, like plates on a tray getting buried by later ones.
3. Too Little vs. Too Much: Two Predicaments
Context engineering is fundamentally about balancing two extremes.
Too little: the AI starts hallucinating
With too little context, the AI enters a state of 'making things up without material,' known professionally as hallucination — confidently talking nonsense.
Imagine asking someone who has never been to Beijing, 'What's the best hotpot restaurant in Sanlitun?' Without material, and reluctant to admit ignorance, they might invent a plausible-sounding answer — a restaurant that may not exist.
AI does the same. When its view lacks necessary background, it fills gaps with the 'general knowledge' from training — which is often imprecise or outdated.
Too much: the AI loses focus
The other extreme is overload. Throw in an entire manual, dozens of chat pages, and ten reference documents, and the AI doesn't know where to look.
It's like asking someone to read ten documents simultaneously and then immediately answer a specific question — attention gets scattered, key details get missed, information from different sources gets confused.
Research shows that when context is too long, AI's information usage efficiency drops significantly — it tends to attend to the beginning and end of the context, while the middle gets ignored. This is the 'lost in the middle' phenomenon.
4. Four Basic Moves: Write, Select, Compress, Isolate
Industry practice has distilled four basic strategies — like the waiter's four techniques:
1. Write — store things externally first
Some items aren't needed now but might be later. Rather than occupying precious tray space, jot them on a note and retrieve them when needed.
AI can do the same: write intermediate conclusions or temporary search results to external storage (a file or database), then read them back later. Its view keeps only currently relevant information.
2. Select — put in only what's needed
Don't stuff in all materials; pick only the most relevant parts for the current question.
It's like a doctor treating a patient: the complete medical record may span hundreds of pages, but during consultation the doctor pulls only the test reports and medication history relevant to current symptoms — not decades of records.
In practice, this is usually done via retrieval-augmented generation (RAG): search for the most relevant document fragments based on the user's question, and place only those fragments into the context rather than the entire document library.
3. Compress — make long things short
As chat history accumulates and nears the window limit, summarize old conversations into a few sentences to free space.
Picture a long meeting: every hour, a secretary distills prior discussion into a summary of 'what we've agreed on so far.' Later attendees read only the summary instead of dozens of pages of verbatim transcripts.
AI can do the same: periodically compress accumulated context into summaries that preserve core information in shorter text.
4. Isolate — separate concerns, avoid interference
If one tray can't hold all the dishes, hire more waiters — each carrying their own tray with a few dishes.
In AI, this is the multi-agent architecture: split a complex task into subtasks handled by different AIs, each with its own context window, with results merged at the end.
5. The 'Sweet Spot'
Good context engineering is neither minimal nor maximal. It pursues a sweet spot — just enough information for accurate judgment without overwhelming the model.
Finding it involves several considerations:
6. Why Context Engineering Suddenly Got Hot
The concept has been frequently discussed only in the last year or two — not because it wasn't needed before, but due to two shifts:
First, AI got much stronger. Early models stumbled at basic conversation; no one expected them to handle complex context. Today's models can do far more, and people realized the bottleneck is often not the model's 'brain' but giving it the wrong information 'in front of it.'
Second, applications got more complex. From simple Q&A chat to workflows requiring retrieval, tool use, multi-step reasoning, and long-term memory — all of which demand much more from context management.
As the saying goes: 'Prompt engineering is how you talk to AI; context engineering is what you show it.' The former is rhetoric; the latter is curation. Combining both is what truly unlocks AI's potential.
7. Summary
The essence of context engineering is the craft of information curation. Its core question: within the AI's limited view, what should go in, what should stay out, and how should it be arranged?
Like that restaurant waiter's tray — a good waiter never leaves it empty (guests go hungry) nor piles it until it tips over. He carefully arranges what goes out each trip and what follows, ensuring every dish arrives at the right moment.
In the AI era, each of us can become such an information curator.
---
*This article is based on the 'Context Engineering' module added in easy-learn-ai commit 9621a05.*