Skip to content

Source paper

dLLM: Simple Diffusion Language Modeling

Zhanhui Zhou, Lingjie Chen, Hanghang Tong, Dawn Song
arXiv 2602.22661 · 2026110 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces dLLM, a unified, open-source framework designed to standardize the training, inference, and evaluation of Diffusion Language Models (DLMs). It provides modular components for reproducing existing models (like LLaDA and Dream), recipes for converting autoregressive models and BERT encoders into DLMs with minimal compute, and a comprehensive evaluation pipeline to benchmark performance and speed.

How it works (AI summary)

The framework is built around three core abstractions: a modular Trainer supporting Masked and Block Diffusion objectives via HuggingFace adapters; a Sampler interface that decouples inference logic (including Fast-dLLM acceleration) from model architecture; and an Evaluation harness aligned with official benchmarks. The authors demonstrate the framework by releasing checkpoints of BERT-based chat models and converted Qwen models trained via Supervised Fine-Tuning (SFT).