Skip to content

Source paper

Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting

Hao Feng, Shu Wei, Xiang Fei and 10 more

Hao Feng, Shu Wei, Xiang Fei, Wei Shi, Yingdong Han, Lei Liao, Jinghui Lu, Binghong Wu, Qi Liu, Chunhui Lin, Jingqun Tang, Hao Liu, Can Huang

arXiv 2505.14059 · 20253 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces Dolphin, a multimodal document image parsing model that employs an "analyze-then-parse" paradigm to extract structured content from documents. By first identifying layout elements (e.g., paragraphs, tables) and their reading order, and then using these elements as anchors for parallel content parsing with specialized prompts, Dolphin achieves state-of-the-art performance and high efficiency compared to existing end-to-end VLMs.

How it works (AI summary)

Dolphin uses a two-stage architecture sharing a unified Vision-Language Model (Swin Transformer encoder + mBART decoder). Stage 1 (Layout Analysis) detects element bounding boxes and types in reading order using a global image prompt. Stage 2 (Content Parsing) crops these elements and processes them in parallel using element-specific prompts (e.g., specific prompts for HTML tables vs. LaTeX formulas) to generate the final structured text.