Grammar-Constrained Decoding for Structured DocTags
Implement a grammar-constrained decoding layer using an FSM or CFG parser during inference to force language models to select tokens maintaining valid DocTags syntax.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
As described in Section 5.3 (Qualitative Results), the model relies on standard autoregressive generation which frequently results in malformed structures, such as missing closing tags or missing location identifiers.
Limitation
The model occasionally produces syntactically invalid markup and 'endless repetition' loops, which prevents downstream parsers from correctly rendering the converted document.
Proposed Solution
Implement a grammar-constrained decoding layer (e.g., using a FSM or CFG parser) during inference to force the LLM to only select tokens that maintain valid DocTags syntax, specifically enforcing balanced XML tags and coordinate-range validity for <loc_x> tags.
Why This Matters
Ensuring 100% syntactical validity is critical for production-grade document conversion where automated downstream processing is required.
How to Get Started
- Define the formal grammar for DocTags, focusing on balanced XML tags and coordinate-range validity.
- Integrate a finite state machine or context-free grammar parser into the generation loop of a target model.
- Validate the constrained decoder on a test set of malformed document generation tasks.
- Measure the reduction in syntax errors and structural failures compared to standard autoregressive generation.
Code linked to this paper on Hugging Face: github.com/docling-project/docling
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Not checked against the literature yet.
Source Paper
SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversionAhmed Nassar, Andres Marafioti, Matteo Omenetti +10 more · 2025
SmolDocling is a 256M parameter vision-language model designed for efficient, end-to-end document conversion, mapping page images directly to a structured 'DocTags' markup format. It significantly reduces computational overhead compared to larger models while maintaining robust performance across diverse document elements like tables, charts, and equations.
Related Ideas
- Grammar-Constrained Command GenerationAutoDev: Automated AI-Driven Development🟡Fairly easy
- Robust Control via Constrained DecodingRecursive Language Models🟡Fairly easy
- Instruction-Controllable VLMs for Strict Formatting via Preference OptimizationStepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports🟢Easy
- Query-Driven Dynamic Layout Parsingdots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model🔴Hard
- Optimizing Parameter Efficiency via Language-Specific TokenizationFlavors of Moonshine: Tiny Specialized ASR Models for Edge Devices🟡Fairly easy