"The agentic future will be built one context at a time. Engineer them well."
Imagine standing before a grand yet invisible castle — built not of stone, but woven from countless words, instructions, memories, and decisions. That castle is the future AI agent. And the most critical tool for building it is not expensive training data or massive compute, but the seemingly humble yet endlessly powerful "Context." Through hands-on experience, the Manus team shows us: when trying to evolve large language models from chatbots into true agents that independently complete complex tasks, the traditional fine-tuning path is rugged, while Context Engineering is the highway to production-grade systems.
🌱 Why Context Engineering Instead of Fine-tuning?
At the crossroads of building general-purpose agents, developers face two paths. One is classic Fine-tuning: training an end-to-end model from scratch — tempting in theory, but harsh in reality. Feedback loops take weeks, and any upgrade to a new base model can wipe out all previous effort. The other is In-Context Learning: leveraging frontier LLM capabilities directly by carefully crafting the input context to steer behavior.
The Manus team firmly chose the latter, coining the term Context Engineering — how to construct, manage, and optimize the context fed into the model. The process is full of trial and error, prompt tweaking, and architectural iteration, which the author jokingly calls "Stochastic Graduate Descent" — a nod to gradient descent and to the countless late nights grad students spend rewriting prompts.
> "Stochastic" refers to the randomness — context engineering relies heavily on manual trial and intuition; "Graduate" hints that this is a craft requiring long-term accumulation, not a one-shot algorithm.
This choice is deliberate: agent tasks involve long-horizon, multi-step, uncertain real-world work, and success hinges on whether the model can precisely recall, reason, and act within massive context.
🔑 KV-Cache: The Invisible Lifeline of Agent Systems
Agent tasks have a distinctive signature: extremely long input contexts (often hundreds of thousands of tokens) and very short outputs (a few hundred tokens) — a ratio up to 100:1. Under this extreme asymmetry, inference cost and time-to-first-token (TTFT) are almost entirely determined by the KV-cache hit rate:
Only when cache hit rates approach 100% can costs be driven down. The Manus team therefore shaped their entire architecture around KV-cache friendliness, with three key practices:
1. Prefix Stability: Transformers are autoregressive — once early tokens change, all downstream KV cache is invalidated. The most common anti-pattern is putting a second-precise timestamp at the start of the system prompt — every call differs, and the cache always misses. The fix: move dynamic information (like current time) to the end of the context and keep the head fully static.
2. Append-only Context: Never modify past actions or observations. Even if history contains errors, don't edit or delete. This preserves serialized determinism — even JSON key order must be fixed to prevent tiny differences from invalidating the cache.
3. Explicit cache breakpoints: In inference frameworks lacking automatic incremental caching, manually insert special markers to force cache refreshes and avoid hidden bugs.
These seemingly trivial details determine whether a system costs several dollars per run as a demo, or a few cents per run in production.
🛠 Taming Tool Explosion with Logits Mask
As agent capabilities expand, available tools multiply: browsers, code executors, file I/O, database queries... Stuffing all tool definitions into the context wastes precious space and causes interference. Worse, with RAG-style dynamic tool loading/unloading, historical calls may point at tool definitions that have suddenly "vanished," completely confusing the model.
Manus's solution is elegant: don't add or remove tools in the prompt — mask the logits at decoding time (Logit Masking, aka Constrained Decoding).
A finite state machine tracks the currently allowed tool set; at each generated token, disallowed tools' probabilities are set to −∞:
Implementation details include:
- Response prefilling with special tokens (e.g.,
im_start) to quickly enter tool-call mode. - Tool name normalization (e.g., all prefixed with
browser_orshell_) for easy prefix masking. - Three invocation modes: Auto (model chooses freely), Required (must call a tool), Specified (only a designated subset allowed).
- Keep only URLs or file paths in context, not full webpage text.
- The model learns to call
read_file(path)to load only the needed portion on demand. - Loaded content can be selectively compressed/summarized before being appended to context, maintaining signal-to-noise ratio.
- Slightly randomize serialization templates (different wording, different field order).
- Diversify the phrasing of observation summaries.
- Even fine-tune tool call formats.
This inference-time intervention is more reliable than prompt engineering and saves context — moving complex control logic from expensive context into the nearly free decoding phase.
💾 The File System: Unlimited External "VRAM" for Agents
Even with 128k or million-token windows, real-world observations (full webpages, long PDFs, code repositories) can still overflow. Stuffing them in directly is not only expensive but triggers the classic "Lost-in-the-middle" phenomenon: models attend least to information in the middle of the context.
Manus's approach: treat the file system as unlimited-capacity, persistent external memory.
The author further muses: this combination of "context + external storage + on-demand loading" effectively lets Transformers emulate the classic Neural Turing Machine. Future State Space Models (SSMs) with native file read/write abilities may become a more natural agent architecture.
📜 Recitation Against Attention Decay
In tasks exceeding 50 steps, models most easily forget the original user goal. This isn't stupidity — it's inherent to attention:
Recent key-value pairs naturally receive higher weight (Recency Bias).
Manus's clever countermeasure: have the agent maintain a todo.md file, updating it after each step and reciting the current progress and remaining goals in full at the end of the context. This "Recitation" forcibly pulls the global plan to the position of strongest attention — like giving the model a sticky note it can never forget.
It's like solving a complex math problem by re-copying the final goal after every step — it sounds redundant, but drastically reduces the chance of going off track.
🔄 Keep Wrong Turns: Let Failure Be the Teacher
Most people instinctively hide errors: when the agent goes wrong, wipe the context and restart for a "clean" history. Manus does the opposite: firmly keep erroneous actions and their full error observations.
The reason is profound: error trajectories are precious negative samples. When the model repeatedly sees "Action A → Error X," it naturally lowers the probability of choosing A again. The ability to recover from mistakes is a core manifestation of agent intelligence. Erasing errors erases learning opportunities.
It's like learning to drive: if you pretend every mistake never happened, you'll never learn to avoid that pothole.
🎲 Avoid the Few-Shot Trap: Inject Structured Noise
LLMs are born imitators. If the context is filled with highly similar action-observation pairs (e.g., batch-processing 20 resumes), the model easily falls into pattern repetition: infinite loops, hallucinations, or mechanically copying earlier steps.
The countermeasure is actively injecting structured noise:
🌓 From Demo to Production: The Philosophy of Context Engineering
Looking back at the Manus team's practices, a clear thread emerges: the core challenge of moving agent systems from flashy demos to reliable production is maintaining long-horizon planning capability and system stability under limited context windows and expensive inference.
Their answer condenses into four maxims:
1. KV-cache is the lifeline — every design decision must bow to the caching mechanism. 2. Inference-time intervention (logit masking) is more reliable and token-efficient than prompt engineering. 3. Bigger context isn't better — use external storage and dynamic loading to keep signal-to-noise high. 4. True robustness comes from embracing errors and learning from failure, not from pursuing one perfect plan.
These insights apply not only to building Manus-like general agents but also offer valuable reference for anyone working on RAG, long-context reasoning, or tool-calling optimization. The agentic future won't be won by one supermodel, but built brick by brick by countless engineers, one context optimization at a time.
As the original article's closing maxim goes: "The agentic future will be built one context at a time. Engineer them well."
Every one of us can be that alchemist.
References
1. Manus Team. Context Engineering for AI Agents: Lessons from Building Manus. https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus 2. Liu, J. et al. Lost in the Middle: How Language Models Use Long Contexts. arXiv preprint arXiv:2307.03172. 3. Vaswani, A. et al. Attention Is All You Need. Advances in Neural Information Processing Systems 30 (2017). 4. Gu, A. et al. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv preprint arXiv:2312.00752. 5. OpenAI. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774.