Skip to content

Source paper

DreamID-V:Bridging the Image-to-Video Gap for High-Fidelity Face Swapping via Diffusion Transformer

Xu Guo, Fulong Ye, Xinghui Li and 6 more

Xu Guo, Fulong Ye, Xinghui Li, Pengqi Tu, Pengze Zhang, Qichao Sun, Songtao Zhao, Xiangwang Hou, Qian He

arXiv 2601.01425 · 202636 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

The paper introduces DreamID-V, a video face-swapping framework that addresses the lack of temporal consistency and paired training data in existing methods. The authors bridge the gap between image and video domains by creating a synthetic data pipeline (SyncID-Pipe) that generates high-fidelity video-face pairs and employing a Diffusion Transformer (DiT) aimed at preserving identity and motion continuity.

How it works (AI summary)

The core methodology utilizes 'SyncID-Pipe,' a pipeline that pre-trains an Identity-Anchored Video Synthesizer (IVS) to generate video clips from start/end frames and pose sequences, creating 'Bidirectional ID Quadruplets' for explicit supervision. The model architecture is a Diffusion Transformer (DiT) with Modality-Aware Conditioning to separate identity, pose, and context, trained using a Synthetic-to-Real curriculum and fine-tuned with Identity-Coherence Reinforcement Learning (IRL) to minimize flickering.