Grammar-Constrained Decoding for Structured DocTags
Implement a grammar-constrained decoding layer using an FSM or CFG parser during inference to force language models to select tokens maintaining valid DocTags syntax.
Source paper
Ahmed Nassar, Andres Marafioti, Matteo Omenetti, Maksym Lysak, Nikolaos Livathinos, Christoph Auer, Lucas Morin, Rafael Teixeira de Lima, Yusik Kim, A. Said Gurbuz, Michele Dolfi, Miquel Farré, Peter W. J. Staar
Generated by AI. The paper’s authors did not write these.
Implement a grammar-constrained decoding layer using an FSM or CFG parser during inference to force language models to select tokens maintaining valid DocTags syntax.
Incorporate a dynamic cropping and tiling strategy to re-encode high-resolution patches of detected table or formula regions and cross-attend them with global page embeddings.
Augment model vocabulary with a dedicated SMILES tokenizer and fine-tune on specialized chemical-instruction sets to improve molecule recognition and structure reconstruction.
SmolDocling is a 256M parameter vision-language model designed for efficient, end-to-end document conversion, mapping page images directly to a structured 'DocTags' markup format. It significantly reduces computational overhead compared to larger models while maintaining robust performance across diverse document elements like tables, charts, and equations.
The model utilizes a SigLIP vision encoder with pixel-shuffle compression and a 135M parameter SmolLM-2 language backbone to autoregressively generate a unified XML-like representation that captures content, spatial location, and hierarchical document structure in a single pass.