Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationRelated work found

Instruction-Guided Iterative Editing Module

Enable iterative vector graphic editing by fine-tuning a model on a synthetic dataset of original animations, edit instructions, and modified outputs.

Time

2-3 days

Compute level

Multiple GPUs

Resources

A multi-GPU node (e.g., 4-8x A100 GPUs) for generating synthetic edit pairs from MMLottie-2M and fine-tuning the VLM backbone.

Prerequisites

basic ML knowledge, data synthesis scripting

Scope

modify existing code

The Gap

While the introduction claims Lottie is 'editing-friendly,' Section 4.2 and the experiments focus entirely on generating animations from scratch. There is no mechanism to refine existing outputs (e.g., 'Make the red circle blue') without regenerating the entire sequence, which discards the user's preferred structural elements.

“Future work could explore constrained decoding, reinforcement learning with renderability rewards, or agentic integration with professional tools like After Effects to improve reliability and practical applicability.”
From the paper, Section 6, page 11. Matched against the PDF text.

Limitation

The current autoregressive architecture is one-shot; it lacks the capability to accept an existing Lottie file and an edit instruction to perform localized updates.

Proposed Solution

Develop a synthetic 'edit instruction' dataset by programmatically perturbing the MMLottie-2M data (e.g., shifting hue, scaling duration). Fine-tune the model to accept a concatenated input of [Original Lottie Tokens, Edit Prompt] and output [Edited Lottie Tokens], enabling iterative refinement similar to InstructPix2Pix but for vector data.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

This transforms the tool from a novelty generator into a practical workflow asset, allowing designers to tweak results rather than relying on random regeneration.

How to Get Started

How to test it: Hypothesis: Fine-tuning on synthetic edit instruction pairs enables robust localized Lottie updates. Baseline: One-shot generation from scratch given the edit prompt. Metric: CLIP similarity and structural preservation score on edited regions. Stopping condition: Superior edit preservation compared to full regeneration.

  1. Programmatically generate a synthetic edit dataset from the MMLottie-2M corpus by applying parameter perturbations and corresponding textual instructions.
  2. Format the training data to accept concatenated input sequences containing both the original Lottie tokens and the edit prompt.
  3. Fine-tune an open-source sequence generation model on the synthetic edit dataset using a standard GPU setup.
  4. Evaluate the editing model on qualitative user instructions to verify structural preservation and target modifications.

Code linked to this paper on Hugging Face: github.com/OpenVGLab/OmniLottie

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Works such as SVGenius explore LLMs in SVG editing and generation, while others like RefVideo-6M and SEED-Data-Edit focus on instructional editing datasets for images or video. However, none of the listed papers specifically implement programmatic perturbation of vector tokens on datasets like MMLottie-2M for instruction-guided Lottie animation editing.

We searched OpenAlex for “Enable iterative vector graphic editing by fine-tuning a model on a synthetic dataset of original animations, edit instructions, and modified outputs.” and “Instruction-Guided Iterative Editing Module” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

Yiying Yang, Wei Cheng, Sijin Chen +5 more · 2026

This paper presents OmniLottie, an end-to-end framework leveraging a multimodal Large Language Model (Qwen2.5-VL) to generate Lottie vector animations from text, images, or video instructions. The authors introduce a specialized "Lottie Tokenizer" to compress complex JSON structures into efficient command sequences and release MMLottie-2M, a large-scale dataset of vector animations, to train the model for high visual fidelity and semantic alignment.

Related Ideas

Is this a good idea?

Share onXLinkedIn