Docling: The Universal Document Parser That Turns PDF, DOCX, EPUB and Video into LLM Breakfast
> Feeding a PDF to an LLM looks simple—just extract text with PyPDF2. Then you discover: tables get shredded into out-of-order text fragments, two-column papers are read in the wrong order, formulas turn into garbage, and scanned PDFs are pure images. You switch to pdfplumber—tables improve, formulas still fail. You switch to unstructured—feature-rich but slow as a snail. Eventually you're writing a research report on "which document parser to choose" instead of doing your actual work.
This was the daily reality of every RAG builder before 2024. The IBM Research team clearly suffered through this too—they built Docling, an MIT-licensed document processing toolkit dedicated to turning messy document formats into structured data that LLMs can digest. 94 stars/day, 30k+ total stars on GitHub, an LF AI & Data Foundation project, arXiv paper 2408.09869.
Core Abstraction: DocTags as a Universal Intermediate Representation
Docling's core design is DocTags—a unified intermediate representation for documents. Whether the input is PDF, DOCX, PPTX, HTML, EPUB, or Apple Pages, everything is first parsed into DocTags, then exported to Markdown, JSON, or other formats.
The key insight: format diversity is an input-side problem, not an output-side problem. LLMs don't care whether you hand them a PDF or a DOCX—they only care about the structure and quality of the text. DocTags decouples "parsing various formats" from "generating LLM input": parsers only need to convert each format into DocTags, and downstream applications only need to handle DocTags.
This is the same idea as a compiler's IR: LLVM IR decouples frontends (C/C++/Rust) from backends (x86/ARM/RISC-V), and DocTags decouples frontends (PDF/DOCX/HTML) from backends (Markdown/JSON/vector databases). This "narrow waist" design recurs throughout systems engineering because its compositional advantages are enormous—adding a new format requires only a new parser; adding a new output requires only a new exporter.
PDF Understanding: More Than Text Extraction
Docling's PDF processing isn't simple text extraction—it's structured understanding:
- Page layout analysis: identifying headings, body text, headers/footers, page numbers
- Reading-order detection: correct ordering for two-column papers, column recognition in multi-column pages
- Table structure recognition: converting tables to HTML or Markdown while preserving row/column relationships
- Formula recognition: converting image formulas to LaTeX
- Image classification: distinguishing charts, diagrams, and photos
- Metadata extraction: title, authors, abstract, references
- Scanned-PDF OCR quality depends on the underlying Tesseract/EasyOCR
- Complex formulas (multi-line, matrices) recognition rates fall short of specialized tools
- Video processing is frame-level sampling, not true video understanding
- High memory usage on large files
Before 2024, achieving this required combining 5–6 tools: LayoutParser for layout, TableTransformer for tables, Pix2Struct for formulas, Tesseract for OCR. Docling packages them into one, using IBM's own GraniteDocling 258M model for visual understanding.
258M parameters is an interesting number. Mainstream VLMs today are 7B+ parameters (Qwen-VL, LLaVA), yet Docling uses a 258M small model for document understanding—another proof that "task-specialized small models > general-purpose large models." Document understanding doesn't need general world knowledge; it needs layout awareness and structure recognition. Such narrow tasks are fine with small models, at an order of magnitude lower inference cost.
Format Coverage: From PDF to Video
Docling's format support list reads like a buffet menu:
Documents: PDF, DOCX, PPTX, XLSX, HTML, Markdown, AsciiDoc, RTF, ODT/ODS/ODP (OpenDocument)
Special formats: EPUB (e-books), XBRL (financial reports), Apple Pages (.pages, both container generations), Email (.eml/.msg)
Multimedia: images (OCR + VLM), audio (ASR transcription), video (MP4/AVI/MOV/MKV/WebM—keyframe extraction + ASR transcription)
Plain text: .txt, .qmd, .Rmd
This coverage goes far beyond the original paper (which covered only PDF). Video and audio support especially—Docling can now convert an MP4 into a structured output of "keyframe images + speech transcript," directly usable for video RAG.
XBRL support deserves a special mention. XBRL is the standard format for financial reporting—SEC-listed companies' annual reports all use it. The traditional approach requires Arelle or other XBRL-specific tools; by folding it into a unified document pipeline, Docling means financial RAG applications can process annual reports, press releases, and analyst reports through a single pipeline—regardless of source format.
MCP Server: Documents as an Agent Tool
Docling provides an MCP (Model Context Protocol) server. LLM agents can invoke Docling as a tool—when an agent encounters a PDF, instead of parsing it itself, it calls Docling's API via MCP and receives structured text.
The impact is bigger than it looks. Current agent frameworks (LangChain, LlamaIndex) handle documents via "parse upfront, store in a vector database." But if documents are dynamic—an agent only discovers mid-task that it needs to read some PDF—runtime parsing is required. The MCP server turns Docling into the agent's "document-perception organ," letting agents parse arbitrary documents on demand during task execution.
This aligns with Anthropic's MCP ecosystem direction—exposing specialized tools to agents rather than making agents implement everything. Docling's MCP server makes "document parsing" a composable agent capability.
Integration Ecosystem: Four Mainstream Frameworks
Docling's integration list: LangChain, LlamaIndex, CrewAI, Haystack. These four cover the mainstream of agent/RAG frameworks—LangChain as the most popular general framework, LlamaIndex focused on RAG, CrewAI for multi-agent orchestration, Haystack for enterprise search.
This "don't build a framework, be a plugin for every framework" strategy is smart. Docling doesn't want to be "yet another RAG framework"—it wants to be the document-processing layer for all RAG frameworks. Users don't need to switch frameworks to use Docling; they just add a Docling loader to their existing stack.
Compare with unstructured.io: it also does document parsing but positions itself more as "data ETL," while Docling leans toward "LLM input preprocessing." Their technical approaches are similar (layout analysis + OCR + structuring), but their ecosystem positioning differs.
IBM's Open Source Strategy
Docling is an IBM Research project, but it has been donated to the LF AI & Data Foundation. This means:
1. Trademark belongs to the foundation: IBM doesn't exclusively own the brand 2. Contributor agreements: third parties can contribute without signing IBM's CLA 3. Transparent governance: project direction isn't decided by a single company
IBM's recent open source AI strategy is consistent—the Granite model family, Docling, Merlin (RAG evaluation) are all MIT/Apache licensed with foundation governance. This contrasts with Meta's Llama strategy (custom license with commercial-use restrictions). IBM's approach is friendlier to enterprises—no license risk, safe for commercial products.
Docling's arXiv paper (2408.09869) was published in August 2024—over two years ago now. The project has expanded from pure PDF parsing to video, audio, financial reports, and e-books—an evolution path "from single-point breakthrough to platform," similar to PyTorch's journey from deep learning framework to general AI compiler.
Unsolved Problems
Docling's documentation is candid, listing "Coming soon" and "Limitations":
Coming soon: enhanced metadata extraction, more layout models, improved table understanding
Limitations (inferred from GitHub issues):
Conclusion: Document Processing's "Narrow Waist" Moment
Docling's core contribution isn't any single parsing algorithm—layout analysis, table recognition, and OCR are all mature technologies. Its contribution is turning document processing into a layer with a standard interface: input any format, output DocTags, and downstream applications handle everything uniformly.
This is the same design philosophy as TCP/IP for networking, LLVM IR for compilers, and SQL for databases—find the system's narrow waist and decouple upstream from downstream. Document processing never had this narrow waist before; every RAG framework had to handle PDF/DOCX/HTML itself, reinventing the wheel. Docling standardized that wheel.
When document processing becomes a composable layer (via the MCP server, via LangChain/LlamaIndex loaders), agents can focus on task logic instead of document formats. It looks like a minor engineering detail, but its impact on the agent ecosystem is profound—an agent's capability boundary is often determined not by how strong the model is, but by how complete the toolchain is.
The 94 stars/day growth shows developers have been waiting a long time for this "narrow waist."
---
Project: https://github.com/docling-project/docling Docs: https://docling-project.github.io/docling/ Paper: https://arxiv.org/abs/2408.09869 Model: https://huggingface.co/ibm-granite/granite-docling-258M