Hybrid Sliding-Window and Fixed-State Depth Memory for Deeply Looped Transformers
Compress older layer updates into a fixed-size recurrent state while keeping recent updates explicit to cut memory traffic and latency in parameter-shared transformers.