UnlimitedOCR: How a 3B Model "Reads" 40 Pages in One Pass
> Project: Baidu UnlimitedOCR (open source, MIT license) > Core breakthrough: R-SWA (Reference Sliding Window Attention) + DeepEncoder visual compression > Release date: June 22, 2026
---
A Counter-Intuitive Result
3B parameters. 500M active. One forward pass over a 40-page PDF.
How do the competitors compare?
| Model | Parameters | OmniDocBench v1.5 | |---|---|---| | UnlimitedOCR | 3B (500M active) | 93.23% | | Qwen3-VL | 235B | 89.15% | | Qwen2.5-VL | 72B | 87.02% | | Gemini-2.5 Pro | undisclosed | 88.03% |
This is not "punching above its weight." It is doing the right thing with the right architecture and leaving parameter-scaling efforts behind.
---
The "Amnesia" Problem in Long-Document OCR
Traditional end-to-end OCR models have a fatal weakness: the longer the output, the slower and dumber they get.
The reason is straightforward. Standard multi-head attention (MHA) caches the Key and Value of every previously generated token. The KV cache grows linearly with output length:
$$C_{MHA}(T) = L_m + T$$
- $L_m$ is the length of the input (visual tokens + prompt)
- $T$ is the length of generated output so far
- Standard attention: every new word requires re-reading everything you have written. At word 1000 you are flipping back to word 1 for verification.
- R-SWA: your eyes stay locked on the original text (full visual reference), but you only remember the last line or two you wrote (128-token window). Earlier text? Trust the process, keep writing.
- ASR (automatic speech recognition): long-audio transcription
- Translation: long-form continuous translation
- Any generation task requiring long output and constant memory
- 32K context is still a hard ceiling—"unlimited" only means "no memory loss within the window"
- Long prefill remains: visual tokens are compressed, but the 40-page prefill still has to ingest 10K+ tokens
- Multi-page limits in Base mode: multi-page inference is restricted to Base mode (1024 resolution); Gundam mode (640 + crop) is single-page only
- ASR/translation transfer is future work and not yet validated
- GitHub: https://github.com/baidu/Unlimited-OCR
- HuggingFace: https://huggingface.co/baidu/Unlimited-OCR
- Paper: https://arxiv.org/pdf/2606.23050
- License: MIT
- Core innovation: R-SWA (Reference Sliding Window Attention)
Result: 10 pages is manageable, 20 pages slows down noticeably, and 40 pages either runs out of VRAM or becomes painfully slow.
The engineering workaround has been page-by-page parsing + result stitching, processing each page separately and sewing the Markdown fragments together at the end. But stitching introduces new problems: cross-page tables get split in half, reading order scrambles, and contextual information is lost. An acronym defined on page 1 is forgotten by page 30.
This is the "amnesia" problem in OCR: the model is not really *reading* the document, it is *transcribing page by page and clearing its memory between pages.*
---
R-SWA: Making the KV Cache a Constant
The core innovation is Reference Sliding Window Attention (R-SWA). The design is intuitive, even "naive"—and that naivety is what makes it work.
Mechanism
Each newly generated token attends to only two things:
1. Full reference — all visual tokens (raw document image) plus the prompt. These are preserved in full forever, because they are the source of truth for recognition. 2. Sliding window — the most recent 128 output tokens. Anything older is forgotten.
Mathematically:
$$C_{R-SWA}(T) = L_m + \min(n, T) \leq L_m + n$$
where $n = 128$ is the window size.
No matter how long the output, the KV cache is strictly bounded by a constant. When $T \gg n$, the cache ratio approaches zero. Memory is flat, and per-step latency is flat.
Analogy
Imagine you are transcribing a book.
This is not "forgetting." It is selective working-memory management.
---
DeepEncoder: 16x Visual Compression
R-SWA solves the decoder-side memory problem, but the input side still faces a challenge: 40 pages of PDF images is a lot of data.
UnlimitedOCR's answer is DeepEncoder, a cascaded visual compressor:
1. SAM-ViT (window attention) handles local detail 2. CLIP-ViT (global attention) captures overall semantics 3. A bridge layer performs 16x token compression
Effect: a 1024x1024 PDF page is reduced to 256 visual tokens. 40 pages × 256 = 10,240 visual tokens, plus output tokens, fit comfortably in a 32K context window.
Critically, visual tokens in R-SWA do not participate in state transitions—they are read as "reference" but never written to the KV cache. This means the clarity of the original image information does not degrade as decoding proceeds.
---
Performance: Not Just "Readable" but "Read Well and Read Fast"
Accuracy
| Metric | DeepSeek OCR | UnlimitedOCR | Gain | |---|---|---|---| | Overall (v1.5) | 87.01 | 93.23 | +6.22% | | Text Edit Distance | 0.073 | 0.038 | closer to source | | Formula CDM | 83.37 | 92.61 | +9.24 | | Table TEDS | 84.97 | 90.93 | +5.96 | | Reading Order | 0.086 | 0.045 | more accurate |
On 40-page documents, edit distance stays below 0.11 and Distinct-35 reaches 97%—meaning almost no mechanical repetition.
Speed
| Output Length | DeepSeek OCR | UnlimitedOCR | Gap | |---|---|---|---| | 256 tokens | 7,229 TPS | 7,230 TPS | tie | | 1024 tokens | 7,423 TPS | 7,841 TPS | +5.6% | | 2048 tokens | 7,167 TPS | 7,881 TPS | +10.0% | | 4096 tokens | 6,430 TPS | 7,905 TPS | +22.9% | | 6144 tokens | 5,823 TPS | 7,848 TPS | +34.8% |
The longer the output, the larger the advantage—because standard attention's KV cache balloons with output length while R-SWA stays constant.
---
Why Baidu? Why Now?
Several contextual notes:
1. Technical lineage: UnlimitedOCR is continued from DeepSeek-OCR (4,000 steps, encoder frozen, decoder trained). This demonstrates that a strong base + a precise change = a huge gain. It is a victory of architectural design, not compute.
2. Talent flow: One core contributor is listed as "YY†", strongly suspected by industry observers to be Wei Haoran, former lead of the DeepSeek-OCR team behind GOT-OCR 2.0 and DeepSeek-OCR. If true, this is a significant win for Baidu in the top-tier talent war.
3. Open-source strategy: MIT license, simultaneous GitHub + HuggingFace release, 10K+ stars in 5 days, top of four GitHub/HuggingFace charts. Baidu is demonstrating that Chinese open-source models are closing the gap rapidly.
4. Business logic: Kunlun chip is preparing a Hong Kong IPO (target valuation ~$50B). UnlimitedOCR showcases the synergy potential of in-house silicon plus in-house models.
---
The Bigger Picture: R-SWA Is Not Just for OCR
The paper positions R-SWA as General Parsing Attention—applicable beyond OCR to:
This is an architectural contribution, not an OCR-specific trick. It reveals a deeper design principle: not every task needs full historical context. For "image/audio to structured text" mapping tasks, recent local context plus full original-input reference is often more efficient and more accurate than an ever-growing KV cache.
---
Limitations and Honesty
UnlimitedOCR is not a silver bullet:
---
A Signal
The AI industry is shifting from "the bigger the better" to "the more right the better."
UnlimitedOCR used 3B parameters to beat a 235B general multimodal model—not because 3B > 235B, but because an architecture purpose-built for long-document parsing beats a generalist on document parsing.
This signal does not belong to OCR alone. It belongs to every vertical being squeezed by "general large models." When a specialized architecture finds the right angle, it can leapfrog at a fraction of the cost.
For developers and researchers, the takeaway: stop staring at parameter counts. Stare at the problem.
---