Prompt Cache: Why Claude Code Is Up to 10x Faster Than Most AI Assistants

> Source commit: 515b759 (easy-learn-ai)

1. The Overlooked Problem: Redundant Work on Every Turn

Every time you message an AI assistant, the request includes the entire conversation history. The model re-reads, re-understands, and re-encodes everything from your first "hello" onward. After twenty rounds of conversation, the model recomputes all nineteen previous exchanges every single turn—like re-reading every preceding page each time you turn one.

Prompt Cache solves this.

2. Core Concept

If the beginning of this request is identical to the last one, the precomputed results for that portion are reused. The API server saves the prefix of your prompt and, on the next request, compares token-by-token from the start. As long as the prefix matches exactly, the intermediate computation is read from cache. It is strict prefix matching—simple, but remarkably effective.

3. Key Numbers

• 90% savings on cache hit — you pay 10% of the normal price for cached tokens.
• 25% surcharge on first write — writing a prefix into cache costs 25% more than a normal request; a single subsequent hit more than covers it.
• 5 minutes default TTL — caches expire after 5 minutes of inactivity. A 1-hour tier exists at double the write price (+100%), worth it after two hits.
• 1024 / 4096 minimum — a cacheable segment must be at least 1024 tokens (4096 in some cases). Shorter prompts are never cached.

4. Cache as Infrastructure

At Anthropic, cache hit rate is monitored like uptime: a few percentage points of drop triggers an on-call alert, investigated like a production incident. For Claude Code—long, multi-turn coding sessions with tens of thousands of tokens of context per turn—without caching there is no product. In Anthropic's own words: "Prompt caching is everything."

5. Prefix Matching: Order Everything Like Packing a Suitcase

Since matching is by prefix, the most stable content goes first. Claude Code's request structure:

1. System prompt + tool definitions — identical across all users and sessions; highest hit rate, shareable across users. 2. Project docs (CLAUDE.md) — stable within a project. 3. Session context — unchanged for the life of the session. 4. Chat messages — the only truly dynamic part, billed at full price.

6. Three Pitfalls That Break the Whole Chain

Any change anywhere in the prefix invalidates everything after it:

1. A live timestamp in the system prompt — it changes every second, invalidating the cache from the first token. 2. Unordered containers (dict/set) for tool definitions — Python dict/set iteration order can vary between runs, reordering tools and breaking the match. 3. Editing a tool's parameters — adding one subagent or renaming a field invalidates tens of thousands of cached tokens.

7. Seven Best Practices from the Claude Code Team

1. Don't touch the prompt—use messages. Inject updates (time, file changes, mode switches) as user messages or tool results. Claude Code uses a <system-reminder> tag. 2. Don't switch models mid-session. Different models handle prompts and tools differently; switching invalidates all cached prefixes and forces a fresh write. 3. Independent caches per subtask. Don't share one giant prefix across independent subtasks. 4. Never modify tool definitions casually. They sit early in the prefix; changes invalidate everything downstream. 5. Express Plan Mode via tools. Claude Code uses a dedicated switch_mode tool instead of rewriting the system prompt, keeping the prefix stable. 6. Lazy-load tools. Activate rarely-used tools on demand so they don't pollute the prefix or waste tokens. 7. Cache-safe forking for compaction. When compressing a full context window into a summary, create the new request with an identical system prompt, tools, and session context, then append the compaction instruction as a final new message. The API sees the same prefix, so the entire cache is reused; reserve a "compaction buffer" for the instruction and summary.

8. Why It Matters

Prompt Cache reflects a deeper design philosophy: in the AI era, cost optimization is not an afterthought but the starting point of architecture. Claude Code was built around caching from day one—its prompt partitioning, tool structure, CLAUDE.md injection, Plan Mode, and compaction logic all serve one constraint: keep the prefix stable, maximize cache reuse.

Economically, caching transforms long conversations from linearly growing costs into a fixed one-time write premium plus incremental per-turn charges, fundamentally changing the viability of long-session AI products. The closing insight: many AI optimizations are complex (quantization, distillation, MoE, speculative decoding), but Prompt Cache's elegance is its simplicity. The real question is never "how to compute faster" but "why compute the same thing twice."

---

*Source: easy-learn-ai, commit 515b759.*