Skip to content

Source paper

InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams

Shuai Yuan, Yantai Yang, Xiaotian Yang and 4 more

Shuai Yuan, Yantai Yang, Xiaotian Yang, Xupeng Zhang, Zhonghao Zhao, Lingming Zhang, Zhipeng Zhang

arXiv 2601.02281 · 202622 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces InfiniteVGGT, a transformer-based model for continuous 3D geometry reconstruction from infinite video streams without memory overflow. The authors propose a rolling memory mechanism that dynamically prunes the KV cache based on key-diversity (cosine similarity) rather than attention weights, enabling compatibility with FlashAttention and long-term stability.

How it works (AI summary)

The model utilizes a causal video transformer with a 'rolling memory' KV cache. To prevent memory growth, it employs a training-free pruning strategy that retains a set of 'immutable anchor tokens' (from the first frame) and selects the most diverse recent tokens based on a layer-wise adaptive budget determined by the cosine similarity of keys.