Reinforcement Learning with Renderability Rewards (RLVR)
Guide generative models to produce visually valid code by integrating a headless renderer into a reinforcement learning loop that rewards correct graphical outputs.
Source paper
Yiying Yang, Wei Cheng, Sijin Chen, Honghao Fu, Xianfang Zeng, Yujun Cai, Gang Yu, Xingjun Ma
Generated by AI. The paper’s authors did not write these.
Guide generative models to produce visually valid code by integrating a headless renderer into a reinforcement learning loop that rewards correct graphical outputs.
Enable iterative vector graphic editing by fine-tuning a model on a synthetic dataset of original animations, edit instructions, and modified outputs.
Integrate explicit 2D point trajectories from CoTracker into the Lottie Tokenizer via a cross-attention adapter to eliminate temporal motion errors in video-to-vector graphics generation.
This paper presents OmniLottie, an end-to-end framework leveraging a multimodal Large Language Model (Qwen2.5-VL) to generate Lottie vector animations from text, images, or video instructions. The authors introduce a specialized "Lottie Tokenizer" to compress complex JSON structures into efficient command sequences and release MMLottie-2M, a large-scale dataset of vector animations, to train the model for high visual fidelity and semantic alignment.
The core methodology involves a custom Lottie Tokenizer that converts hierarchical JSON files into linear, quantized command sequences to remove redundant metadata. These token sequences are then used to fine-tune Qwen2.5-VL in an autoregressive manner, enabling the model to predict vector animation parameters directly from interleaved multimodal inputs.