Skip to content
AI-generated research idea🟠Medium💡Moderate InnovationRelated work found

Temporal-Consistency Enhanced Super-Resolution

Replace simple channel concatenation in video super-resolution networks with 3D temporal cross-attention and a warping-error loss to eliminate shimmering artifacts between frames.

Time

a couple of months

Compute level

Large cluster

Resources

Large compute cluster

Prerequisites

transformer architectures

Scope

new model architecture

The Gap

Section 3.2 describes the Video Super-Resolution (VSR) network, which takes 'low-resolution latents and noise' as input via channel concatenation. While it aligns spatially, the paper does not mention explicit mechanisms to handle temporal consistency during upscaling. Upscalers often introduce 'shimmering' artifacts where high-frequency textures (like grass or hair) change randomly between frames.

Limitation

The current VSR approach treats frames semi-independently in the latent space, potentially introducing high-frequency temporal flickering in fine details.

Proposed Solution

Replace the simple channel concatenation in the VSR network with a 3D Temporal Cross-Attention mechanism. Inject the Low-Resolution (LR) latent features as keys/values into the VSR transformer blocks. Furthermore, introduce a temporal consistency loss (e.g., warping error based on optical flow) specifically during the training of the VSR module to penalize pixel-shimmering.

Why This Matters

Eliminating flickering artifacts is essential for video generation to be perceived as 'high definition' rather than just 'high resolution.'

How to Get Started

  1. Set up the baseline video super-resolution codebase and verify inference pipelines using synthetic datasets.
  2. Implement the 3D Temporal Cross-Attention module by modifying the transformer block architecture.
  3. Integrate an optical flow estimation model to compute warping error for the temporal consistency loss.
  4. Train the modified model on a large-scale video dataset and evaluate against baseline metrics using PSNR, SSIM, and temporal consistency scores.

Code linked to this paper on Hugging Face: github.com/Tencent-Hunyuan/HunyuanVideo-1.5

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Papers like Enhancing temporal consistency in video super-resolution using optical flow-guided deep networks and Temporal Consistency Learning of Inter-Frames for Video Super-Resolution explore optical flow-guided losses and temporal consistency in video super-resolution. However, they do not specifically evaluate the combination of 3D temporal cross-attention for LR feature injection with warping error penalties.

We searched OpenAlex for “Replace simple channel concatenation in video super-resolution networks with 3D temporal cross-attention and a warping-error loss to eliminate shimmering artifacts between frames.” and “Temporal-Consistency Enhanced Super-Resolution” and an AI compared the idea with the top 6 results. A small search can miss relevant work.

Source Paper

HunyuanVideo 1.5 Technical Report

Bing Wu, Chang Zou, Changlin Li +78 more · 2025

This paper introduces HunyuanVideo 1.5, an open-source 8.3B parameter video generation model that achieves state-of-the-art performance comparable to proprietary models like Sora and Kling. The authors propose a unified Diffusion Transformer (DiT) architecture optimized with a novel Selective and Sliding Tile Attention (SSTA) mechanism and a high-compression 3D VAE, trained via a progressive multi-stage pipeline involving text-to-image, text-to-video, and image-to-video tasks.

Related Ideas

Is this a good idea?

Share onXLinkedIn