Skip to content
AI-generated research idea🟠Medium💡Moderate InnovationNo close match found

Self-Supervised Consistency Learning for Real-World Robustness

Improve real-world document understanding robustness by penalizing representation divergence between clean and augmented unlabelled document images.

Time

1-2 weeks

Compute level

Large cluster

Resources

8x A100 GPUs (80GB) for self-supervised training and data augmentation pipelines on large image corpora.

What the paper used

Synthetic training data generated via a 72B teacher model and 7B student model, pre-trained on a massive multi-million document corpus, fine-tuned on 300k diverse samples.

Prerequisites

basic ML knowledge

Scope

modify existing code

The Gap

Section 3.3 ('Data Engine') relies heavily on 'Teacher-Led Structured Generation' and 'Synthetic Data', where a teacher model creates synthetic training samples.

“Training a unified model for multilingual document parsing imposes extreme demands on the training data. It requires a corpus that is not only massive in scale but also exceptionally diverse”
From the paper, Section 3.3, page 4. Matched against the PDF text.

Limitation

The model's performance is upper-bounded by the teacher model's biases and the 'synthetic gap.' The synthetic data may lack the specific noise distributions (stains, folds, sensor noise) of real-world low-resource documents, causing the model to struggle with realistic degradations not present in the clean, rendered synthetic corpus.

Proposed Solution

Integrate a self-supervised consistency loss using real, unlabeled document images. During training, feed both clean and augmented (noisy, rotated, masked) versions of real documents into the model and penalize divergence in their latent representations or structural outputs, reducing reliance on the teacher's synthetic distribution.

Why This Matters

improves robustness on noisy, real-world scans where synthetic data often fails to capture the true distribution of artifacts.

How to Get Started

How to test it: We hypothesize that self-supervised consistency loss on noisy real documents will improve real-world robustness. We will compare against our base model using OCR Edit distance on real-world noisy document test sets, stopping if degradation occurs on clean benchmarks.

  1. Gather a collection of unlabeled real-world document images and define a suite of realistic augmentations including noise, rotation, and masking.
  2. Implement a self-supervised consistency loss function to match latent representations or structural outputs between clean and augmented inputs.
  3. Train the document parsing model with the consistency objective and evaluate its robustness on noisy real-world benchmarks.

Code linked to this paper on Hugging Face: github.com/rednote-hilab/dots.ocr

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

None of the listed papers address self-supervised consistency learning specifically using real unlabeled document images to align clean and augmented representations. The surveys and domain adaptation papers are generally unrelated to this document-centric approach.

We searched OpenAlex for “Improve real-world document understanding robustness by penalizing representation divergence between clean and augmented unlabelled document images.” and “Self-Supervised Consistency Learning for Real-World Robustness” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model

Yumeng Li, Guang Yang, Hao Liu +2 more · 2025

This paper introduces dots.ocr, a unified Vision-Language Model designed for end-to-end document layout parsing, text recognition, and relational understanding across 126 languages. The authors achieve this by training a 1.2B parameter vision encoder from scratch on a massive, synthetically generated multilingual dataset produced by a novel three-stage data engine, demonstrating state-of-the-art performance on OmniDocBench and the newly introduced XDocParse benchmark.

Related Ideas

Is this a good idea?

Share onXLinkedIn