Docling: The Universal Document Parser That Turns PDF/DOCX/EPUB/Video into LLM Breakfast
> You want to feed a PDF to an LLM. Sounds simple—just extract text with PyPDF2. Then you discover: tables get shattered into scrambled text fragments, two-column papers are read in the wrong order, formulas turn into gibberish, and scanned PDFs are pure images. You switch to pdfplumber—tables improve but formulas still fail. You switch to unstructured—feature-rich but painfully slow. Eventually you realize you're writing a document-parser comparison report instead of doing your actual work.
This was the daily reality of anyone building RAG before 2024. The IBM Research team clearly suffered from it too—they built Docling, an MIT-licensed document processing toolkit dedicated to converting messy document formats into structured data LLMs can digest. It gains ~94 stars/day, has 30k+ GitHub stars, is an LF AI & Data Foundation project, and has an arXiv paper (2408.09869).
The Core Abstraction: DocTags as a Universal Intermediate Representation
Docling's core design is DocTags—a unified document intermediate representation. Whether the input is PDF, DOCX, PPTX, HTML, EPUB, or Apple Pages, it's first parsed into DocTags, then exported to Markdown, JSON, or other formats.
The key insight: format diversity is an input-side problem, not an output-side problem. LLMs don't care whether you give them PDF or DOCX—they only care about text structure and quality. DocTags decouples "parsing various formats" from "generating LLM input"—parsers only need to convert each format into DocTags, and downstream applications only need to handle DocTags.
This is the same idea as a compiler IR: LLVM IR decouples frontends (C/C++/Rust) from backends (x86/ARM/RISC-V); DocTags decouples frontends (PDF/DOCX/HTML) from backends (Markdown/JSON/vector databases). This "narrow waist" design recurs throughout systems because its compositional advantages are enormous—adding a new format means writing one new parser; adding a new output means writing one new exporter.
PDF Understanding: More Than Text Extraction
Docling's PDF processing isn't simple text extraction. It performs structured understanding:
- Page layout analysis: identifying headings, body text, headers/footers, page numbers
- Reading order detection: correct reading order for two-column papers, column detection in multi-column pages
- Table structure recognition: converting tables to HTML or Markdown while preserving row/column relationships
- Formula recognition: converting image formulas to LaTeX
- Image classification: distinguishing charts, diagrams, photos
- Metadata extraction: title, authors, abstract, references
- OCR quality on scanned PDFs depends on underlying Tesseract/EasyOCR
- Complex formulas (multi-line, matrices) recognize worse than specialized tools
- Video processing is frame-level sampling, not true video understanding
- High memory usage on large files
Before 2024, these required combining 5–6 tools: LayoutParser for layout, TableTransformer for tables, Pix2Struct for formulas, Tesseract for OCR. Docling packages them into one, using IBM's own GraniteDocling 258M model for visual understanding.
258M parameters is an interesting number. Mainstream VLMs are 7B+ (Qwen-VL, LLaVA), yet Docling uses a 258M model for document understanding—another case of "task-specialized small model > general large model." Document understanding doesn't need general world knowledge; it needs layout awareness and structure recognition. Such narrow tasks are fine with small models, at an order of magnitude lower inference cost.
Format Coverage: From PDF to Video
Docling's supported formats read like a buffet menu:
Documents: PDF, DOCX, PPTX, XLSX, HTML, Markdown, AsciiDoc, RTF, ODT/ODS/ODP (OpenDocument)
Special formats: EPUB (e-books), XBRL (financial reports), Apple Pages (.pages, both container generations), Email (.eml/.msg)
Multimedia: images (OCR + VLM), audio (ASR transcription), video (MP4/AVI/MOV/MKV/WebM—keyframe extraction + ASR transcription)
Plain text: .txt, .qmd, .Rmd
This coverage far exceeds the original paper (which only covered PDF). The video and audio support is notable—Docling can convert an MP4 into structured output of "keyframe images + speech transcription," directly usable for video RAG.
XBRL support deserves special mention. XBRL is the standard format for financial reports—SEC-listed companies file annual reports in it. Traditionally handled by Arelle or other specialized tools, Docling brings XBRL into its unified pipeline, meaning financial RAG applications can process annual reports, press releases, and analyst reports through one pipeline—regardless of source format.
MCP Server: Documents as an Agent Tool
Docling ships an MCP (Model Context Protocol) server. LLM agents can call Docling as a tool—when an agent encounters a PDF, instead of parsing it itself, it calls Docling's API via MCP and receives structured text.
This matters more than it seems. Current agent frameworks (LangChain, LlamaIndex) handle documents by parsing them ahead of time into a vector database. But if documents are dynamic—an agent discovers mid-task it needs to read a certain PDF—runtime parsing is required. The MCP server makes Docling the agent's "document-sensing organ," letting agents parse arbitrary documents on demand during task execution.
This aligns with Anthropic's MCP ecosystem direction—exposing specialized tools to agents rather than making agents implement everything themselves.
Integration Ecosystem: Four Mainstream Frameworks
Docling integrates with LangChain, LlamaIndex, Crew AI, and Haystack—covering the mainstream of agent/RAG frameworks. The strategy: "don't be a framework, be a plugin for frameworks." Docling doesn't want to be "yet another RAG framework"—it wants to be the document-processing layer for all RAG frameworks. Users don't switch frameworks; they just add a Docling loader.
Compared to unstructured.io: similar technical approach (layout analysis + OCR + structuring), but unstructured leans toward "data ETL" while Docling leans toward "LLM input preprocessing."
IBM's Open Source Strategy
Docling is an IBM Research project, donated to the LF AI & Data Foundation:
1. Trademark belongs to the foundation: IBM doesn't monopolize the brand 2. Contributor agreement: third parties can contribute without signing IBM's CLA 3. Transparent governance: project direction isn't decided by a single company
IBM's recent open-source AI strategy is consistent—Granite models, Docling, Merlin (RAG evaluation) are all MIT/Apache licensed with foundation governance. This contrasts with Meta's Llama strategy (custom license with commercial restrictions). IBM's approach is friendlier to enterprises—no license risk for commercial products.
The arXiv paper (2408.09869) was published in August 2024; by now the project has expanded from pure PDF parsing to video, audio, financial reports, and e-books—a "single-point breakthrough to platform" evolution similar to PyTorch's path.
Unsolved Problems
Docling's docs candidly list "Coming soon" and "Limitations":
Coming soon: enhanced metadata extraction, more layout models, improved table understanding
Limitations (inferred from GitHub issues):
Conclusion: Document Processing's "Narrow Waist" Moment
Docling's core contribution isn't any single parsing algorithm—layout analysis, table recognition, and OCR are mature technologies. Its contribution is turning document processing into a layer with a standard interface: input any format, output DocTags, downstream apps process uniformly.
This is the same design philosophy as TCP/IP for networking, LLVM IR for compilers, SQL for databases—find the system's "narrow waist" and decouple upstream from downstream. Document processing previously had no narrow waist; every RAG framework handled PDF/DOCX/HTML itself, reinventing the wheel. Docling standardized that wheel.
When document processing becomes a composable layer (via MCP server, via LangChain/LlamaIndex loaders), agents can focus on task logic instead of document formats. It looks like a small engineering detail, but the impact on the agent ecosystem is profound—an agent's capability boundary is often determined not by how strong the model is, but by how complete the toolchain is.
94 stars/day shows developers have been waiting a long time for this narrow waist.
---
Project: https://github.com/docling-project/docling Docs: https://docling-project.github.io/docling/ Paper: https://arxiv.org/abs/2408.09869 Model: https://huggingface.co/ibm-granite/granite-docling-258M