Coordinate-Independent Relative Pose and Scale Representation
A method that replaces absolute coordinate regression with relative graph-based embeddings to eliminate distance-from-origin accumulation errors in long-sequence reconstructions.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
The model uses direct coordinate regression and absolute camera poses relative to a reference frame, resulting in larger pose errors for cameras far from the reconstruction origin.
“We observe larger camera-position errors for cameras farther from the reconstruction origin than for those near it. This suggests that pose accuracy may be sensitive to the coordinate representation and the numerical scale of absolute translations.”
Limitation
Pose accuracy degrades as cameras get farther from the reconstruction origin due to sensitivity to the coordinate representation and the numerical scale of absolute translations.
Proposed Solution
Reformulate the geometry decoding head to operate strictly within local relative coordinate frames linked by continuous relative pose graphs, eliminating global origin dependence during autoregressive sequence expansion.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
This is expected to enhance long-sequence stability and reduce drift over hundreds of frames without requiring computationally expensive optimization.
How to Get Started
How to test it: Hypothesis: relative coordinate representation reduces trajectory drift on long video sequences. Baseline: pi4 baseline evaluated on Oxford Spires. Metric: Absolute Trajectory Error (ATE). Stopping criteria: achieving lower ATE than the baseline on sequences exceeding 500 frames.
- Implement relative frame transformation layers in the geometry branch output heads.
- Modify loss functions to compute relative pose errors entirely within local neighborhood graphs.
- Train the modified network on ScanNet++ and evaluate on the Oxford Spires benchmark.
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Works like LONG3R and VGGT-Align handle streaming and long-sequence 3D reconstruction with attention to global consistency and local frames, while RePAST explores relative pose attention scene representation transformers. However, none specifically reformulate the geometry decoding head to operate strictly within continuous relative pose graphs to eliminate global origin dependence during autoregressive expansion as proposed.
- LONG3R: Long Sequence Streaming 3D Reconstruction · 2025 · 49 citations
- VGGT-Align: Bridging Local Reconstruction and Global Consistency for Long-Sequence 3D Reconstruction · 2026 · 3 citations
- RePAST: Relative Pose Attention Scene Representation Transformer · 2023 · 7 citations
We searched OpenAlex and arXiv for “relative pose representation long sequence 3D reconstruction drift reduction” and “Coordinate-Independent Relative Pose and Scale Representation” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
π^4: A Foundation Model for 4D Scene Geometry and MotionYifan Wang, Weiguang Zhao, Ni Yang +4 more · 2026
The paper introduces pi4, a unified foundation model for 4D scene perception that reconstructs dense geometry, camera poses, and tracks from uncalibrated RGB observations. By employing a shared DINOv3 encoder, looped Transformer decoders, and a geometry-conditioned motion branch, the model jointly achieves state-of-the-art multi-view scene reconstruction and 3D point tracking.
Related Ideas
- Non-Linear Latent Tracking for Robust Depth Estimation in Unconstrained Long VideosDVD: Deterministic Video Depth Estimation with Generative Priors🔴Hard
- Overcoming Perspective Ambiguity in 2D-to-3D VLM ReasoningStepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports🟠Medium
- Global Geometric Memory for Loop ClosureWorldWarp: Propagating 3D Geometry with Asynchronous Video Diffusion🟠Medium
- Multi-Agent World Modeling: Extending Camera-Conditioned DiTs to Dynamic EntitiesWorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation⛔Very hard
- 4D Spatiotemporal Pretraining for Dynamic ScenesUtonia: Toward One Encoder for All Point Clouds🔴Hard