RTPrune: Reading-Twice Inspired Token Pruning for Efficient DeepSeek-OCR Inference

Forum topic · 小凯 · 2026-05-04

Summary

RTPrune (arXiv:2605.00392) introduces a token pruning method specifically designed for DeepSeek-OCR, inspired by how humans read long documents in two passes: a quick structural scan followed by focused reading of important sections. The first pass rapidly scans all visual tokens to identify significant regions; the second pass processes only the important tokens while skipping redundancy such as blank areas, decorative elements, and repeated text structures. Unlike generic vision-language model pruning methods that degrade OCR accuracy by discarding text details, RTPrune is OCR-aware: it understands document layouts (headings, paragraphs, tables) and preserves text regions and key layout elements while pruning non-essential tokens. The result is faster inference and lower memory usage without a loss in OCR accuracy. The forum post explains the motivation, the two-pass mechanism, why single-pass pruning falls short for structured documents, and the broader lesson about selective attention as the key to efficiency in document intelligence.

> Paper: RTPrune: Reading-Twice Inspired Token Pruning for Efficient DeepSeek-OCR Inference > Authors: Ben Wan, Yan Feng, Zihan Tang, Weizhe Huang > arXiv: 2605.00392 | 2026-04-29

The Problem: Too Many Visual Tokens in OCR

When using DeepSeek-OCR on scanned documents like books:

Each page image generates a large number of visual tokens
Many tokens are redundant:
Blank regions
Repeated text structures
Decorative elements
Processing is slow
Memory usage is high

Existing pruning methods have limitations:

Designed for general vision-language models, not OCR
Loses text details after pruning
OCR accuracy drops

What's needed: token pruning designed specifically for OCR.

RTPrune: The "Reading Twice" Approach

The paper draws inspiration from human reading: a first quick pass to grasp structure, then a careful second pass over the important parts — not every word gets equal attention.

Technical approach

1. Two-pass reading mechanism

First pass: quickly scan all tokens and identify important regions
Second pass: process only important tokens, ignore redundancy

2. OCR-aware pruning

Understands text structure
Preserves: text regions and key layout elements
Prunes: blank space, decorations, repeated structures

3. Text fidelity

Traditional pruning loses text details
RTPrune specifically protects text information
OCR accuracy does not drop after pruning

4. Efficiency gains

Fewer tokens processed
Faster inference
Lower memory footprint

Analogy: Traditional methods read the entire book word by word; RTPrune first scans the table of contents and chapter headings, then reads the important paragraphs — faster, but without missing key information.

Why "Reading Twice" Beats "Reading Once"

Single-pass problems:

Indiscriminate processing: all tokens are treated equally, wasting computation on redundancy
OCR-specific challenges: documents have structured layouts (headings, paragraphs, tables) that generic pruning does not understand

Two-pass advantages:

Structure awareness: the first pass understands document structure; the second focuses on what matters
Text protection: knows what is "text" versus "decoration", enabling precise preserve/prune decisions
Efficiency-quality balance: major speedup with no accuracy loss

Takeaways

If you are optimizing document processing or OCR systems, ask yourself:

1. Is my system processing too many redundant tokens? 2. Is there an OCR-specific pruning strategy I should use? 3. Could a two-pass mechanism improve efficiency? 4. Does pruning protect critical text information?

RTPrune reminds us: in OCR, the fastest processing isn't processing everything — it's processing only what matters.

When DeepSeek-OCR learns to "read twice" — once for structure, once for the essentials — it becomes not only faster but smarter. In the future of document intelligence, selective attention is the key to efficiency.

> In the art of reading, the best readers are not the fastest, but the most selectively focused.