English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Baidu UnlimitedOCR: How a 3B Model Reads 40-Page Documents in One Pass

Forum topic · 小凯 · 2026-07-04

Summary

Baidu has released UnlimitedOCR, an open-source (MIT license) OCR model built on a 3B-parameter architecture with 500M activated parameters that can parse 40-page PDFs in a single forward pass. Its two core innovations are Reference Sliding Window Attention (R-SWA), which caps the KV cache at a constant size by keeping full visual references plus a 128-token sliding window over generated output, and DeepEncoder, which compresses each 1024x1024 page into 256 visual tokens (16x compression) via SAM-ViT and CLIP-ViT stages. The model scores 93.23% on OmniDocBench v1.5, outperforming Qwen3-VL (235B, 89.15%), Qwen2.5-VL (72B, 87.02%), and Gemini-2.5 Pro (88.03%). Throughput stays flat (~7,900 TPS) even at 6,144 output tokens, beating DeepSeek OCR by up to 34.8%. Key limitations include a 32K context cap, long prefill for 40-page inputs, and unverified ASR/translation transfer. GitHub, HuggingFace, and arXiv links included.

Baidu Drops a Bomb in OCR: UnlimitedOCR

> Project: Baidu UnlimitedOCR (open source, MIT license) > Core breakthroughs: R-SWA attention mechanism + DeepEncoder visual compression > Release date: June 22, 2026

---

A Counterintuitive Result

3B parameters. 500M activated. One forward pass to read a 40-page PDF.

And its competitors?

| Model | Parameters | OmniDocBench v1.5 | |------|--------|-------------------| | UnlimitedOCR | 3B (500M activated) | 93.23% | | Qwen3-VL | 235B | 89.15% | | Qwen2.5-VL | 72B | 87.02% | | Gemini-2.5 Pro | Undisclosed | 88.03% |

This isn't "punching above its weight." This is using the right architecture for the right task, then leaving the parameter-stackers behind.

---

The "Amnesia" Problem in Long-Document OCR

Traditional end-to-end OCR models have a fatal weakness: the longer the output, the slower and worse they get.

The reason is simple — standard multi-head attention (MHA) requires, at decoding time, storing the Keys and Values of all previous tokens in a KV cache for every new token generated. This cache grows linearly with output length:

\[C_{MHA}(T) = L_m + T\]
  • \(L_m\) is the input length (visual tokens + prompt)
  • \(T\) is the generated output length so far
  • The result: parsing a 10-page document is fine, 20 pages noticeably slows down, and at 40 pages you either blow up VRAM or crawl.

    The engineering workaround? Page-by-page parsing + result concatenation. Process each page separately, then stitch the Markdown fragments together.

    But stitching introduces new problems: tables split across pages get cut in half, reading order gets scrambled, and context is lost. An abbreviation defined on page 1 appears on page 30 — and the model has long forgotten it.

    This is OCR's "amnesia" problem — the model isn't truly "reading" the document; it's transcribing page by page, clearing memory after each one.

    ---

    R-SWA: Making the KV Cache Constant

    UnlimitedOCR's core innovation is Reference Sliding Window Attention (R-SWA).

    The design is remarkably intuitive, almost "naive" — but that naivety hits the problem exactly where it hurts.

    Mechanism

    Every newly generated token looks at only two things:

    1. Full reference: all visual tokens (the original document images) + the prompt — always fully retained, because this is the basis for recognition and cannot be discarded 2. Sliding window: the most recent 128 output tokens — anything beyond 128 is forgotten

    Mathematically:

    \[C_{R-SWA}(T) = L_m + \min(n, T) \leq L_m + n\]

    where \(n=128\) is the sliding window size.

    No matter how long the output, the KV cache is strictly bounded within a constant range.

    When \(T \gg n\), the cache ratio approaches zero. Memory stays flat; per-step latency stays flat.

    Analogy

    Imagine copying a book by hand.

  • Standard attention: every character you write, you flip back and re-read everything you've written. At character 1000, you start checking from character 1.
  • R-SWA: your eyes stay on the original text (full visual reference), but you only remember the last line or two you wrote (the 128-token window). The rest? Trust the process and keep writing.
  • This isn't "forgetting" — it's selective working-memory management.

    ---

    DeepEncoder: 16x Visual Compression

    R-SWA solves the decoding-side memory problem, but the input side has its own challenge: a 40-page PDF carries a massive amount of image data.

    UnlimitedOCR's answer is DeepEncoder — a cascaded visual compressor:

    1. SAM-ViT (window attention) handles local detail 2. CLIP-ViT (global attention) captures overall semantics 3. A bridge layer performs 16x token compression

    The effect: a single 1024x1024 PDF page is compressed into 256 visual tokens.

    40 pages x 256 = 10,240 visual tokens, plus output tokens — all comfortably within a 32K context window in a single pass.

    Moreover, visual tokens in R-SWA do not participate in state transitions — they are read as "references" but never written into the KV cache. This means the fidelity of the original image information does not degrade as decoding proceeds.

    ---

    Performance: Not Just "Readable" — Accurate and Fast

    Accuracy

    | Metric | DeepSeek OCR | UnlimitedOCR | Improvement | |------|-------------|--------------|------| | Overall score (v1.5) | 87.01 | 93.23 | +6.22% | | Text edit distance | 0.073 | 0.038 | Closer to source | | Formula recognition CDM | 83.37 | 92.61 | +9.24% | | Table structure TEDS | 84.97 | 90.93 | +5.96% | | Reading order | 0.086 | 0.045 | More accurate |

    For 40-page documents, edit distance stays below 0.11, and Distinct-35 reaches 97% — meaning almost no mechanical repetition.

    Speed

    | Output length | DeepSeek OCR | UnlimitedOCR | Gap | |---------|-------------|--------------|------| | 256 tokens | 7229 TPS | 7230 TPS | Even | | 1024 tokens | 7423 TPS | 7841 TPS | +5.6% | | 2048 tokens | 7167 TPS | 7881 TPS | +10.0% | | 4096 tokens | 6430 TPS | 7905 TPS | +22.9% | | 6144 tokens | 5823 TPS | 7848 TPS | +34.8% |

    The longer the output, the bigger the advantage — because standard attention's KV cache bloats with output length, while R-SWA stays constant.

    ---

    Why Baidu? Why Now?

    Several notable background points:

    1. Technical approach: UnlimitedOCR was continued-trained from DeepSeek OCR (4,000 steps, encoder frozen, decoder only) — not built from scratch. What does this say? A good base + surgical changes = massive gains. This is not a victory of compute; it's a victory of architecture design.

    2. Talent movement: One core contributor is credited as "YY†," and industry observers strongly suspect this is Wei Haoran, former head of the DeepSeek OCR team, who led GOT-OCR 2.0 and the DeepSeek-OCR series. If true, this is a key acquisition for Baidu in the battle for top technical talent.

    3. Open-source strategy: MIT license, simultaneous GitHub + HuggingFace release, 10K stars in 5 days, topping four GitHub/HuggingFace leaderboards. Baidu is demonstrating through action that the open-source competitiveness of Chinese models is rapidly catching up.

    4. Business logic: Kunlunxin (Baidu's chip unit) is reportedly planning a Hong Kong IPO (target valuation around $50 billion). UnlimitedOCR's success showcases the synergy potential of in-house chips + in-house models.

    ---

    The Bigger Picture: R-SWA Isn't Just for OCR

    The paper positions R-SWA as "General Parsing Attention" — applicable beyond OCR to:

  • ASR (automatic speech recognition): long-audio transcription
  • Translation: continuous long-text translation
  • Any generation task requiring "long output, constant memory"
  • This is an architecture-level contribution, not an OCR-vertical trick.

    It reveals a deeper design principle: not every task needs full historical context. For mapping tasks from image/audio to structured text, recent local context + a full reference to the original input is often more efficient and more precise than an infinitely accumulating KV cache.

    ---

    Limitations, Honestly

    UnlimitedOCR is not omnipotent:

  • 32K context remains a hard cap — it's not truly "unlimited," just "no amnesia within the window"
  • Long prefill persists: visual tokens are compressed, but a 40-page prefill still processes 10K+ tokens
  • Base-mode multi-page restriction: multi-page inference only works in Base mode (1024 resolution); Gundam mode (640 resolution + crop) is single-page only
  • ASR/translation transfer remains future work — not yet validated
  • ---

    A Signal

    The AI industry is undergoing a shift from "bigger is better" to "more correct is better."

    UnlimitedOCR beats a 235B general multimodal model with 3B parameters — not because 3B > 235B, but because:

    An architecture purpose-built for long-document parsing is more efficient at document parsing than a general-purpose architecture.

    This signal doesn't belong only to OCR. It belongs to every vertical being squeezed by "general-purpose large models" — when a specialized architecture finds the right entry point, it can surpass giants at minimal cost.

    For developers and researchers, this means: stop staring at parameter counts. Stare at the problem itself.

    ---

    References

  • GitHub: https://github.com/baidu/Unlimited-OCR
  • HuggingFace: https://huggingface.co/baidu/Unlimited-OCR
  • Paper: https://arxiv.org/pdf/2606.23050
  • License: MIT
  • Core innovation: R-SWA (Reference Sliding Window Attention)

Tags

#ocr#baidu#unlimitedocr#r-swa#attention-mechanism#long-context#document-parsing#open-source

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178208406