Specialized SMILES-Tokenization for Chemical Structure Recognition
Augment model vocabulary with a dedicated SMILES tokenizer and fine-tune on specialized chemical-instruction sets to improve molecule recognition and structure reconstruction.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
Appendix C.1 highlights that while the model attempts molecule recognition, it consistently fails to reconstruct detailed chemical structures, as standard tokenizers are not optimized for SMILES strings.
Limitation
The current model exhibits low fidelity in specialized domains like chemistry, where the visual structure of a molecule must be mapped to a precise, non-natural language string (SMILES).
Proposed Solution
Augment the DocTags vocabulary with a dedicated SMILES tokenizer and fine-tune the model on the PatCID dataset using a specialized 'chemical-instruction' set that encourages the model to treat atom-bond relationships as distinct structural tokens.
Why This Matters
Expanding into scientific domains like chemistry would make SmolDocling a viable tool for pharmaceutical and materials science research automation.
How to Get Started
- Extract SMILES string data from the PatCID dataset to analyze current tokenizer limitations.
- Integrate a specialized SMILES tokenizer into the model vocabulary.
- Construct a chemical-instruction tuning dataset focused on atom-bond relationships.
- Fine-tune the model and evaluate chemical structure reconstruction performance against baselines.
Code linked to this paper on Hugging Face: github.com/docling-project/docling
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Works like 'AtomDisc: An Atom-level Tokenizer that Boosts Molecular LLMs' and 'Improving the quality of chemical language model outcomes with atom-in-SMILES tokenization' explore specialized atom-level or improved SMILES tokenization for molecular tasks. However, none of the listed papers combine a DocTags vocabulary with a chemical instruction set specifically fine-tuned on the PatCID dataset for document-level chemical structure recognition.
- AtomDisc: An Atom-level Tokenizer that Boosts Molecular LLMs and Reveals Structure--Property Associations · 2025 · 7 citations
- Improving the quality of chemical language model outcomes with atom-in-SMILES tokenization · 2023 · 75 citations
We searched OpenAlex and arXiv for “Augment model vocabulary with a dedicated SMILES tokenizer and fine-tune on specialized chemical-instruction sets to improve molecule recognition and structure reconstruction.” and “Specialized SMILES-Tokenization for Chemical Structure Recognition” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversionAhmed Nassar, Andres Marafioti, Matteo Omenetti +10 more · 2025
SmolDocling is a 256M parameter vision-language model designed for efficient, end-to-end document conversion, mapping page images directly to a structured 'DocTags' markup format. It significantly reduces computational overhead compared to larger models while maintaining robust performance across diverse document elements like tables, charts, and equations.
Related Ideas
- End-to-End Benchmarking of Native Scientific Perception Within Interactive Agentic WorkflowsSage S2.1 Flash Technical Report🟠Medium
- Generative Vision-Language Model for Table RecognitionMinerU: An Open-Source Solution for Precise Document Content Extraction🔴Hard
- CTC-Informed Semantic Alignment for ASRMOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models🟡Fairly easy
- Incorporating Handwriting Recognition CapabilitiesDolphin: Document Image Parsing via Heterogeneous Anchor Prompting🟢Easy
- Instruction-Controllable VLMs for Strict Formatting via Preference OptimizationStepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports🟢Easy