Prompt Cache's technical principle is not complicated. During Transformer autoregressive generation, every generated token requires attention computation over all preceding tokens. Without caching, the K and V tensors of the entire history must be recomputed at each step—O(n²) complexity. With KV Cache enabled, the K/V of preceding tokens are read directly from cache, and a new token only computes its own share, reducing complexity to O(n).
This technique has existed since the GPT-2 era, but between 2024 and 2026 it suddenly became an industry focal point because it upgraded from an *inference optimization* into a commercial bottleneck.
Core insight of Prompt Cache: LLM input prompts contain a large amount of repeated text segments—system messages, prompt templates, context documents, few-shot examples. These fragments are repeatedly fed into the model across many requests. Prompt Cache pre-computes and stores the attention states (KV Cache) of these fragments; when the same fragment appears again, it is reused directly and only the newly added part is computed.
From KV Cache to Prompt Cache
Traditional KV Cache only works within a single request—the K/V of preceding tokens are cached so the next token can use them directly. Prompt Cache extends this to the cross-request, cross-session level: when user A uses a system prompt and user B arrives later, that prompt's attention states are already computed and can be loaded directly.
Yale's paper "Prompt Cache: Modular Attention Reuse for Low-Latency Inference" provided a prototype implementation: latency reduced by 8x on CPU and 60x on GPU, without modifying model parameters.
Key challenge: Transformer positional encoding embeds each token's position into its attention states. If the same text appears at a different position, its attention state changes and cannot be reused directly. The paper's solution is Prompt Markup Language (PML)—schemas define reusable "prompt modules," each assigned a unique position ID that does not depend on global position. Experiments found that LLMs can handle attention states with non-contiguous position IDs: as long as a token's relative positions remain unchanged, output quality is unaffected.
2026 Vendor Prompt Cache Strategies Compared
DeepSeek V4: Compression Is Justice
- 90% KV cache size reduction: via the CSA (Cache-Strided Attention) / HCA (Hybrid Cache Attention) hybrid attention architecture
- 1M context as standard across the lineup
- Cache-hit price at 10%: cached tokens are billed at only 10% of standard input
- cache_control: explicit cache control at the API level; developers can mark which parts are cacheable
- 4 cache breakpoints: split the prompt into multiple cacheable segments, not just one whole prefix
- Cross-session global sharing, 1-hour TTL: the same cache can be reused across different sessions
- Cache-hit price = 1/10 of standard input
- Automatic caching: hidden behind the API, developers need no extra work
- Cache-hit price = 50% of standard input
- No explicit cache-control markers; hit rate is determined automatically by the system
- CachedContent API: an object-oriented abstraction for enterprise-grade long-material reuse
- 2M context window: currently the largest advertised context
- Explicit cache management: developers must explicitly create, update, and delete cached content
DeepSeek's strategy is architectural compression + pricing leverage. It shrinks KV cache size to 1/10 of the original, turning long context from a "VRAM killer" into an affordable option.
Anthropic Claude: A Predictable Enterprise Contract
Anthropic's strategy is predictability. Caching is not a black box but a promise written into the API contract. Boris Cherny, creator of Claude Code, himself admitted: "With the 1M context window, cache misses are very costly. If you step away from your computer for over an hour and return to an old session, you usually get no cache hits at all."
OpenAI GPT-5: Automatic Caching, Default 50% Discount
OpenAI's strategy is ease of use first. Developers don't need to learn a new API; the system decides which parts can be cached. The cost is lower transparency and control.
Google Gemini: Object-Oriented Abstraction
Comparison Table
| Vendor | Caching Strategy | Cache Breakpoints | Hit Discount | TTL | Developer Control | |--------|------------------|-------------------|--------------|-----|-------------------| | DeepSeek | Architecture compression + automatic caching | Undisclosed | 10% | Undisclosed | Low | | Anthropic | Explicit cache_control | 4 | 10% | 1 hour | High | | OpenAI | Automatic caching | None | 50% | Undisclosed | Low | | Google | Object-oriented API | Undisclosed | Undisclosed | Developer-controlled | Very high |
---
> Key finding: Four vendors, four strategies, no unified standard. DeepSeek does architectural compression, Anthropic does contract predictability, OpenAI does ease of use, Google does enterprise-grade object management. This is itself an arms race of caching strategies.