Prompt Cache Moves House: When an Engineering Optimization Finds Its True Place
*A reflection on a category migration in the easy-learn-ai project — from commit c4d4957*
1. A Tiny Change in a Table
In the easy-learn-ai project's README, there is a table classifying modules, each with a name, a category, and a status. On June 10, one module's category changed: "Understanding Prompt Cache" moved from Compression & Deployment to Prompts.
It is a tiny change — one row shifted. But it touches an ongoing cognitive migration: what exactly *is* Prompt Cache?
2. Two Ways of Understanding Prompt Cache
Prompt Cache is an engineering concept. The principle is simple: when a large language model processes input, it tokenizes the prompt and computes attention layer by layer. If two requests share the same prompt prefix (system prompt, context, conversation history), the computation for that prefix can be cached and reused instead of recomputed.
From this angle, Prompt Cache clearly belongs to deployment optimization: it reuses compute, cuts inference cost, and improves latency.
But look from another angle: what *triggers* Prompt Cache? Prompt repetition. It caches prompt prefixes — system prompts, role definitions, few-shot examples, conversation history — all of which are part of prompt design. Your choice of system prompt, your context structure, how much history you retain: these design decisions directly determine the cache hit rate.
In other words, Prompt Cache efficiency is not a pure infrastructure problem but a prompt design problem. If your prompt structure is unstable and the token sequence shifts every request, the cache never hits. If your context management is chaotic — history arbitrarily truncated or reordered — the cache is useless.
So the reclassification is a shift in cognitive frame: from "how to make it faster" to "how to design it so it caches better."
3. Classification Is Never Neutral
In knowledge organization, classification is never neutral. It shapes the path learners take to a concept and the priority practitioners give a problem.
- Under Compression & Deployment, a learner's first reaction: "This is an engineer's job; nothing to do with my prompts."
- Under Prompts, the reaction becomes: "Should I think about cache efficiency when writing prompts?"
- easy-learn-ai project, commit c4d4957: feat: update README, optimize prompt module classification and add content
- easy-learn-ai article: "How Important Is Prompt Caching for Agents?" https://mmh1.top/#/ai-article/prompt-cache
- Anthropic, Prompt Caching, https://www.anthropic.com/news/prompt-caching
The change alters not the knowledge itself but how knowledge connects — pulling a concept once isolated at the engineering layer into the application layer's field of view.
4. Lessons from Claude Code in Practice
The easy-learn-ai article library includes a piece titled "How Important Is Prompt Caching for Agents?" Its lead: "Cache hit rate is an Agent's SLO — the Claude Code team's counterintuitive lesson."
An SLO (Service Level Objective) is a commitment about how your system should perform. For Claude Code, cache hit rate is not a nice-to-have metric but a core one affecting user experience. Claude Code is a coding agent engaged in long interactions: each turn, it re-reads project files, understands code context, and loads conversation history. Recomputing all of that every time would make response times unacceptable. Prompt Cache lets it reuse previously computed prefixes.
But caching is not free. It demands a stable prompt structure — system prompts must not change arbitrarily, context must not be randomly reorganized. It requires deliberately placing frequently reused parts at the front of the prompt and per-request content at the back. That is a prompt design strategy. When Anthropic's engineers discuss Prompt Cache, they discuss how to structure prompts to maximize hit rate — prompt engineering, not pure infrastructure engineering.
5. The Counterintuitive Cost
Where is the counterintuition? The more carefully you design prompts to be structured and reusable, the higher the cache efficiency. Prompt engineering thus has a hidden payoff: it improves not only output quality but system efficiency.
Conversely, treating prompts as disposable — rewritten and restructured per request — wastes both accumulated design work and compute: every request recomputes what could have been cached.
We assumed prompt design is a "soft skill" and cache optimization a "hard skill." In fact, they sit on the same causal chain: good prompt design is the precondition for efficient caching.
6. The Trend Behind the Reclassification
This move is not isolated. It reflects a broader trend: prompt engineering is evolving from "a set of tricks" into "systems thinking."
Early prompt engineering operated at the single-prompt level: writing good prompts, designing few-shot examples, using chain of thought. Today it extends to the system level: prompt stability (cache impact), versioning (consistency), modularity (reuse), and context management (long-conversation quality). These are prompt *system* design problems. The reclassification is a microcosm: from "writing good prompts" to "designing good prompt systems."
7. Takeaways
For learners: prompt engineering is not an isolated art. It connects model behavior, system efficiency, and user experience. How you design prompts affects output quality *and* operating cost.
For engineers: don't treat Prompt Cache as a "decide at deployment" optimization. Consider cache efficiency at the design stage: Which parts are stable and reusable? Which change per request? How should they be ordered? Answer these while writing the prompt — not after discovering a 30% hit rate in production.
8. Closing Thought
A one-row change in a README table seems trivial, but small changes sometimes mirror deep cognitive shifts. Moving Prompt Cache from "Compression & Deployment" to "Prompts" is not just relocation — it is a redrawing of boundaries.
Technology does not accumulate linearly. The same concept gets placed in different frames at different stages. Prompt Cache was first understood as a deployment optimization; it is now understood as a prompt design strategy. Classification is not an endpoint — it is a temporary tool for understanding. But when classification becomes more accurate, the path to understanding becomes clearer. That is the meaning of Prompt Cache moving house.