IBM AI Releases Granite-Docling-258M: An Open-Source, Enterprise-Ready Document AI Model

IBM has introduced Granite-Docling-258M, a cutting-edge open-source vision-language model (licensed under Apache-2.0) tailored for comprehensive document conversion tasks. This model excels in preserving the original layout-accurately extracting tables, code snippets, mathematical equations, lists, captions, and reading sequences-by generating a structured, machine-readable output rather than simplified Markdown formats. It is accessible via Hugging Face, featuring an interactive demo and a specialized MLX build optimized for Apple Silicon devices.

Advancements Over SmolDocling

Serving as the production-grade successor to SmolDocling-256M, Granite-Docling incorporates a more powerful Granite 165M language model backbone and upgrades its visual processing with the SigLIP2 (base, patch16-512) encoder. The model maintains the Idefics3-style pixel-shuffle connector, culminating in a 258 million parameter architecture. This upgrade delivers consistent improvements in layout analysis, full-page OCR, and recognition of code, equations, and tables. Additionally, IBM has resolved previous issues such as repetitive token loops, enhancing model stability for real-world applications.

Model Architecture and Training Details

• Core Structure: Built on an Idefics3-inspired framework combining the SigLIP2 vision encoder, pixel-shuffle connector, and Granite 165M large language model.
• Training Environment: Utilizes nanoVLM, a streamlined PyTorch-based toolkit designed for efficient vision-language model training.
• Output Format: Produces DocTags, a proprietary IBM markup language that encodes document elements, their spatial coordinates, and interrelations, enabling precise downstream conversion to Markdown, HTML, or JSON.
• Computational Resources: Trained on IBM’s high-performance Blue Vela cluster equipped with NVIDIA H100 GPUs.

Performance Enhancements: Granite-Docling vs. SmolDocling

Benchmarking with docling-eval, LMMS-Eval, and specialized datasets reveals significant gains:

• Layout Detection: Mean Average Precision (MAP) improved from 0.23 to 0.27; F1 score increased from 0.85 to 0.86.
• Full-Page OCR: F1 score rose from 0.80 to 0.84, accompanied by a reduction in edit distance errors.
• Code Recognition: Achieved an F1 score of 0.988 compared to 0.915, with edit distance dropping from 0.114 to 0.013.
• Equation Parsing: F1 score enhanced from 0.947 to 0.968.
• Table Extraction (FinTabNet at 150 dpi): Structural TEDS improved from 0.82 to 0.97; content-inclusive TEDS rose from 0.76 to 0.96.
• Additional Benchmarks: MMStar score increased from 0.17 to 0.30; OCRBench score jumped from 338 to 500.
• Reliability: Enhanced mechanisms prevent infinite token loops, ensuring smoother production deployment.

Expanding Language Capabilities

Granite-Docling introduces preliminary support for non-Latin scripts including Japanese, Arabic, and Chinese. While English remains the primary focus, this multilingual extension lays groundwork for broader global applicability.

Revolutionizing Document AI with DocTags

Traditional OCR pipelines that convert documents directly to Markdown often lose critical structural nuances, complicating retrieval-augmented generation (RAG) and downstream analytics. Granite-Docling’s output, DocTags, is a compact, LLM-compatible structural grammar that preserves document topology-such as table layouts, inline and floating mathematical expressions, code blocks, captions, and reading order-complete with explicit spatial coordinates. This rich representation significantly enhances indexing accuracy and contextual grounding for RAG systems and analytical tools.

Deployment and Integration Options

• Docling Ecosystem: The docling CLI and SDK seamlessly integrate Granite-Docling, enabling conversion of PDFs, office documents, and images into various structured formats. IBM envisions this model as a modular component within Docling pipelines rather than a standalone vision-language model.
• Supported Runtimes: Compatible with Transformers, vLLM, ONNX, and MLX. The dedicated MLX build is fine-tuned for Apple Silicon hardware. An interactive Hugging Face Space demo is also available, requiring no GPU resources.
• Licensing: Distributed under the permissive Apache-2.0 license.

Why Choose Granite-Docling?

In enterprise document AI workflows, compact vision-language models that maintain document structure are invaluable for reducing inference costs and simplifying processing pipelines. Granite-Docling consolidates multiple specialized models-covering layout detection, OCR, table parsing, code recognition, and equation understanding-into a single, efficient system. By generating a richer intermediate representation, it enhances the fidelity of downstream retrieval and document conversion tasks. The substantial improvements in table extraction metrics, code and equation recognition accuracy, and operational stability make Granite-Docling a compelling upgrade for production environments.

Live Demonstration

Conclusion

Granite-Docling-258M represents a major leap forward in lightweight, structure-aware document AI technology. By integrating IBM’s Granite backbone, the SigLIP2 vision encoder, and the nanoVLM training framework, it delivers enterprise-grade performance across diverse document elements-including tables, equations, code, and multilingual text-while remaining open-source and resource-efficient. Its measurable superiority over the SmolDocling predecessor and smooth compatibility with Docling pipelines position Granite-Docling as a foundational tool for precise, reliable document conversion and retrieval-augmented generation workflows.