Prompt Cache: Why Claude Code Is ~10x Faster Than Most AI Assistants
> Source commit: 515b759 (easy-learn-ai)
The Problem: Redoing the Same Work Every Turn
Most LLM interactions resend the full conversation history with every request. The model re-reads, re-processes, and re-encodes every earlier token. After twenty rounds of dialogue, each turn recomputes all nineteen previous turns. Prompt Cache exists to eliminate this waste.
The Core Idea
If the beginning of the current request is identical to a previous one, the stored intermediate computation results for that prefix are read from cache instead of being recomputed. The mechanism is strict prefix matching—simple, but extremely effective.
Key Numbers
- 90% cost savings on cache hits—you pay only 10% of the normal price for cached tokens.
- 25% premium the first time a prefix is written to cache. One hit immediately outweighs the write cost.
- 5 minutes default cache TTL. There is also a 1-hour tier, which costs 2x on writes but pays off after two hits.
- 1024 / 4096 tokens minimum prefix length required for caching. Short prompts don't qualify.
Prefix Ordering: Pack the Suitcase Right
Because caching is prefix matching, content ordering is critical: the least-changeable content goes first. Claude Code's request structure, from front to back:
1. System instructions + tool definitions — identical across all users and sessions; highest hit rate, potentially shared cross-user. 2. Project docs (CLAUDE.md) — stable within a project. 3. Session context — unchanged while the session lasts. 4. Chat messages — the only truly dynamic part, billed at full price.
Three Pitfalls That Break the Cache Chain
Any change in the prefix invalidates everything after it. Real failures Anthropic encountered:
1. A live timestamp in the system prompt — it changes every second, invalidating the entire prefix. 2. Storing tool definitions in unordered containers (dict/set) — iteration order varies between runs, breaking the match. 3. Updating a tool's parameters — even a small field change invalidates the whole prefix from the tool definitions onward.
Seven Best Practices
1. Don't modify the prompt—use messages. Inject new info (time, file changes, mode switches) as a <system-reminder> in the next user/tool message rather than editing the system prompt.
2. Don't switch models mid-session. Different models process prompts differently; switching invalidates all cached prefixes and incurs new write premiums.
3. Independent caches per subtask. Don't let independent subtasks share one giant prefix.
4. Never touch tool definitions unless the payoff justifies invalidating everything downstream.
5. Express state transitions via tools. Claude Code's Plan Mode uses a switch_mode tool instead of rewriting the system prompt.
6. Lazy-load tools. Activate rarely used tools on demand so they don't sit in the prefix wasting tokens or risking invalidation.
7. Cache-safe forking for compaction. When compressing context, the new summarization request should reuse the exact same system prompt, tools, and history, appending the compaction instruction as a new message at the end—so the entire cache prefix is reused and only the instruction is billed at full price. Reserve a "compaction buffer" in the context window for this.
Why It Matters
Prompt Cache reveals a design philosophy: in the AI era, cost optimization is not a patch—it's the origin of architecture. Claude Code was built around caching from day one: prompt partitioning, tool structure, CLAUDE.md injection, Plan Mode, and compaction forking all serve one constraint—keep the prefix stable, maximize cache reuse.
Caching also changes the economics of long conversations: instead of costs growing linearly per turn, there is a fixed one-time write premium followed by incremental pricing per turn. The final takeaway: the elegant solution isn't a more complex algorithm—it's refusing to compute the same thing twice.