Vertical and non-Manhattan Layout Parsing
Augment synthetic document rendering pipelines with vertical layouts so vision models learn to parse ancient and non-Manhattan scripts.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
The model's Stage 1 layout analysis and Stage 2 text recognition are trained on standard horizontal text. The paper admits 'Dolphin primarily supports documents with standard horizontal text layout, showing limited capability in parsing vertical text like ancient manuscripts.'
Limitation
The current reading-order prediction and cropping mechanisms assume a left-to-right, top-to-bottom flow, making the model fail on vertical text (Asian scripts) or complex non-Manhattan layouts (artistic magazines).
Proposed Solution
Augment the synthetic data generation pipeline to render documents with vertical text flow and irregular layouts. Retrain the Stage 1 position embeddings and Stage 2 recognition module to handle vertical aspect ratios and top-to-bottom reading directions.
Why This Matters
Crucial for digitizing historical archives (e.g., ancient Chinese/Japanese manuscripts) and processing diverse commercial designs, expanding the model's utility beyond standard academic/business documents.
How to Get Started
- Build a synthetic document rendering pipeline capable of generating top-to-bottom vertical text flows.
- Collect and annotate a small benchmark dataset of ancient manuscripts containing irregular layouts.
- Retrain the Stage 1 position embeddings to handle non-standard vertical aspect ratios and reading directions.
- Fine-tune the Stage 2 recognition module on the augmented vertical text dataset.
Code linked to this paper on Hugging Face: github.com/bytedance/dolphin
Suggested by AI. Adapt it to your setup.
Has This Been Done?
The listed works all focus on 3D room layout reconstruction and Manhattan world inference from images or point clouds. None of them discuss document layout parsing, synthetic data generation for text, or vertical text flow and reading directions.
We searched OpenAlex for “Augment synthetic document rendering pipelines with vertical layouts so vision models learn to parse ancient and non-Manhattan scripts.” and “Vertical and non-Manhattan Layout Parsing” and an AI compared the idea with the top 6 results. A small search can miss relevant work.
Source Paper
Dolphin: Document Image Parsing via Heterogeneous Anchor PromptingHao Feng, Shu Wei, Xiang Fei +10 more · 2025
This paper introduces Dolphin, a multimodal document image parsing model that employs an "analyze-then-parse" paradigm to extract structured content from documents. By first identifying layout elements (e.g., paragraphs, tables) and their reading order, and then using these elements as anchors for parallel content parsing with specialized prompts, Dolphin achieves state-of-the-art performance and high efficiency compared to existing end-to-end VLMs.
Related Ideas
- Dynamic Multi-Scale Layout AnalysisMinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing🟡Fairly easy
- Multilingual Layout and OCR GeneralizationMinerU: An Open-Source Solution for Precise Document Content Extraction🟡Fairly easy
- Iterative Semantic Correction for Layout AnalysisPaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model🟠Medium
- Query-Driven Dynamic Layout Parsingdots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model🔴Hard
- Multi-Granularity Visual Encoding for Small-Scale Document ElementsSmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion🟠Medium