Skip to content
AI-generated research idea🟠Medium🔬Significant ResearchNo close match found

Graph-Neural-Network Based Reading Order Regression

Replace heuristic reading order algorithms with a Graph Neural Network that predicts directed edge weights between detected text blocks to reconstruct document reading sequences.

Time

a couple of months

Compute level

Single GPU or less

Resources

Annotated layout dataset with reading order (can be derived from source LaTeX or Word docs), Standard GPU.

Prerequisites

Graph Neural Networks, Transformer architectures.

Scope

New model architecture (small scale), modify post-processing pipeline.

The Gap

Section 2.3 describes the reading order algorithm as a heuristic approach: 'MinerU developed a segmentation algorithm based on the human reading order, top to bottom, left to right' after dividing the page into columns.

Limitation

Heuristic sorting fails on complex layouts common in magazines, newsletters, or textbooks with marginalia, where the reading path does not follow a strict geometric grid. This leads to fragmented sentences and incoherent context for downstream LLM training.

Proposed Solution

Replace the rule-based segmentation with a Graph Neural Network (GNN). Construct a fully connected graph where nodes are detected text blocks (from the Layout Analysis step). Train the GNN to predict edge weights representing the probability that node B follows node A, minimizing the distance to the ground truth reading sequence.

Why This Matters

Correct reading order is the single most important factor for semantic coherence when converting PDFs to Markdown for RAG (Retrieval-Augmented Generation) applications. Solving this enables the processing of highly complex non-academic documents.

How to Get Started

  1. Extract text block coordinates and ground-truth reading sequences from source LaTeX and Word documents.
  2. Construct a fully connected graph where nodes represent text bounding boxes and edges represent potential reading flows.
  3. Train a Graph Neural Network to predict binary edge weights minimizing sequence distance to the ground truth.
  4. Evaluate the GNN-based ordering against rule-based baselines using Kendall's tau distance on test documents.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

None of the listed papers address graph neural network-based reading order regression for document layouts. The search results only provide general surveys or applications of graph neural networks in other domains.

We searched OpenAlex for “Replace heuristic reading order algorithms with a Graph Neural Network that predicts directed edge weights between detected text blocks to reconstruct document reading sequences.” and “Graph-Neural-Network Based Reading Order Regression” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

MinerU: An Open-Source Solution for Precise Document Content Extraction

Bin Wang, Chao Xu, Xiaomeng Zhao +15 more · 2024

This paper introduces MinerU, an open-source document extraction tool designed to convert diverse and complex PDF documents—such as textbooks, academic papers, and exams—into high-quality machine-readable formats like Markdown and JSON. The authors address the limitations of existing tools by creating a pipeline that integrates specialized deep learning models for layout analysis, formula detection, and content recognition, validated against a newly constructed diverse evaluation dataset.

Related Ideas

Is this a good idea?

Share onXLinkedIn