Skip to content

Source paper

Agent Lightning: Train ANY AI Agents with Reinforcement Learning

Xufang Luo, Yuge Zhang, Zhiyuan He and 5 more

Xufang Luo, Yuge Zhang, Zhiyuan He, Zilong Wang, Siyun Zhao, Dongsheng Li, Luna K. Qiu, Yuqing Yang

arXiv 2508.03680 · 2025134 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

The paper introduces "Agent Lightning," a framework designed to decouple Reinforcement Learning (RL) training from agent execution, enabling the optimization of LLMs within diverse agent frameworks (e.g., LangChain, AutoGen) with minimal code changes. By formulating agent execution as a Markov Decision Process (MDP) and utilizing a hierarchical RL algorithm called "LightningRL," the system decomposes complex multi-turn trajectories into individual transitions, allowing for efficient, scalable policy updates using standard single-turn RL methods.

How it works (AI summary)

The framework utilizes a "Training-Agent Disaggregation" architecture where a centralized Server manages model updates and a distributed Client executes agent logic to collect data. The core algorithm, LightningRL, treats agent execution as a set of isolated transitions (input, output, reward); it employs a credit assignment module that broadcasts episode-level rewards to all steps in a trajectory, enabling the use of value-free RL methods like GRPO on complex, multi-step agent workflows.