Self-Supervised Consistency Learning for Real-World Robustness
Improve real-world document understanding robustness by penalizing representation divergence between clean and augmented unlabelled document images.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
Section 3.3 ('Data Engine') relies heavily on 'Teacher-Led Structured Generation' and 'Synthetic Data', where a teacher model creates synthetic training samples.
“Training a unified model for multilingual document parsing imposes extreme demands on the training data. It requires a corpus that is not only massive in scale but also exceptionally diverse”
Limitation
The model's performance is upper-bounded by the teacher model's biases and the 'synthetic gap.' The synthetic data may lack the specific noise distributions (stains, folds, sensor noise) of real-world low-resource documents, causing the model to struggle with realistic degradations not present in the clean, rendered synthetic corpus.
Proposed Solution
Integrate a self-supervised consistency loss using real, unlabeled document images. During training, feed both clean and augmented (noisy, rotated, masked) versions of real documents into the model and penalize divergence in their latent representations or structural outputs, reducing reliance on the teacher's synthetic distribution.
Why This Matters
improves robustness on noisy, real-world scans where synthetic data often fails to capture the true distribution of artifacts.
How to Get Started
How to test it: We hypothesize that self-supervised consistency loss on noisy real documents will improve real-world robustness. We will compare against our base model using OCR Edit distance on real-world noisy document test sets, stopping if degradation occurs on clean benchmarks.
- Gather a collection of unlabeled real-world document images and define a suite of realistic augmentations including noise, rotation, and masking.
- Implement a self-supervised consistency loss function to match latent representations or structural outputs between clean and augmented inputs.
- Train the document parsing model with the consistency objective and evaluate its robustness on noisy real-world benchmarks.
Code linked to this paper on Hugging Face: github.com/rednote-hilab/dots.ocr
Suggested by AI. Adapt it to your setup.
Has This Been Done?
None of the listed papers address self-supervised consistency learning specifically using real unlabeled document images to align clean and augmented representations. The surveys and domain adaptation papers are generally unrelated to this document-centric approach.
We searched OpenAlex for “Improve real-world document understanding robustness by penalizing representation divergence between clean and augmented unlabelled document images.” and “Self-Supervised Consistency Learning for Real-World Robustness” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language ModelYumeng Li, Guang Yang, Hao Liu +2 more · 2025
This paper introduces dots.ocr, a unified Vision-Language Model designed for end-to-end document layout parsing, text recognition, and relational understanding across 126 languages. The authors achieve this by training a 1.2B parameter vision encoder from scratch on a massive, synthetically generated multilingual dataset produced by a novel three-stage data engine, demonstrating state-of-the-art performance on OmniDocBench and the newly introduced XDocParse benchmark.
Related Ideas
- Self-Supervised RL for Real-World AdaptationRobust-R1: Degradation-Aware Reasoning for Robust Visual Understanding🟠Medium
- Generative Synthesis of Hard Case Training DataMinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing🔴Hard
- Invariance Training via Adversarial RobustificationTowards Robust Mathematical Reasoning🟠Medium
- Self-Supervised Geometric Distillation for Long-Horizon Data WallsLoGeR: Long-Context Geometric Reconstruction with Hybrid Memory🔴Hard
- Uncertainty-Aware Robust DistillationLiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation🟡Fairly easy