Multilingual Layout and OCR Generalization
Extend document processing pipelines to support multiple languages by swapping OCR weights and fine-tuning layout backbones on a diverse multilingual corpus.
Source paper
Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, Conghui He
Generated by AI. The paper’s authors did not write these.
Extend document processing pipelines to support multiple languages by swapping OCR weights and fine-tuning layout backbones on a diverse multilingual corpus.
Replace heuristic reading order algorithms with a Graph Neural Network that predicts directed edge weights between detected text blocks to reconstruct document reading sequences.
Replace specialized table recognition modules with a fine-tuned Vision-Language Model that directly translates table image crops into clean HTML or LaTeX code.
This paper introduces MinerU, an open-source document extraction tool designed to convert diverse and complex PDF documents—such as textbooks, academic papers, and exams—into high-quality machine-readable formats like Markdown and JSON. The authors address the limitations of existing tools by creating a pipeline that integrates specialized deep learning models for layout analysis, formula detection, and content recognition, validated against a newly constructed diverse evaluation dataset.
MinerU employs a multi-stage pipeline called 'PDF-Extract-Kit'. First, it uses PyMuPDF for preprocessing. Second, it utilizes specific models for content parsing: a fine-tuned LayoutLMv3 for layout analysis, YOLO for formula detection, UniMERNet for formula recognition, TableMaster for tables, and PaddleOCR for text. Finally, it applies rule-based post-processing to manage bounding box overlaps and reconstruct the reading order.