Training Diffusion Language Models at Scale

By 2024 the mathematics of masked diffusion was settled and the results were respectable on academic benchmarks with models in the hundreds of millions of parameters. That is exactly the regime where promising alternatives to autoregression have gone to die before.

The open question was not whether discrete diffusion worked, but whether it scaled. Autoregressive language modelling has an unusually clean scaling story: pour in parameters and tokens, and loss follows a power law with no surprises across four orders of magnitude. Any challenger has to demonstrate the same, because a method that plateaus at one billion parameters is a curiosity regardless of its elegance.

There were specific reasons to worry. Bidirectional attention costs more per pass. The training signal is sparser, for reasons the middle of this lesson makes precise. And no infrastructure existed for it, so every efficiency trick accumulated over years of autoregressive engineering had to be rebuilt or abandoned.

Two distinct answers arrived, and the difference between them is the practical core of this lesson.

The first answer came from Shen Nie and colleagues with LLaDA (Large Language Diffusion with mAsking, 2025), an 8-billion-parameter masked diffusion model trained from scratch under the standard pre-training then supervised fine-tuning pipeline.

The striking thing about the recipe is how little is exotic. The mask predictor is a vanilla Transformer with the causal mask removed, so attention is bidirectional. There is no timestep embedding of the kind image diffusion models use, because the corruption level is already implicit in how many positions are masked.

Pre-training follows the objective from the previous lesson exactly. For each sequence, sample a masking ratio uniformly at random, mask that fraction of tokens independently, predict the originals at masked positions, and weight the cross-entropy by the inverse of the ratio.

Supervised fine-tuning needs one adjustment. Prompt and response are concatenated, but only response tokens are masked; the prompt stays clean throughout. That makes the model a conditional generator without changing the objective, which is the same trick autoregressive fine-tuning uses when it masks the loss on prompt tokens.

LLaDA-8B was evaluated against autoregressive models of comparable size across roughly fifteen standard benchmarks, and the headline is that it is competitive rather than merely promising: broadly matching or exceeding equivalently-sized autoregressive baselines such as LLaMA2-7B, and holding up against LLaMA3-8B on a range of tasks.

Two areas of relative strength are worth noting. It performs well on mathematical reasoning, including GSM8K, which is notable because chain-of-thought style reasoning is often assumed to require sequential generation. And it is strong on Chinese-language tasks, reflecting its training data rather than anything intrinsic to diffusion.

The scientific value of the result is narrower and more important than any individual benchmark number. It establishes that masked diffusion follows scaling behaviour through the 8B range, in-context learning emerges as it does in autoregressive models, and instruction-following survives fine-tuning.

The paradigm cleared the bar that kills most alternatives. That does not make it better than autoregression, and the honest reading is that it is a viable second option rather than a replacement.

The most interesting LLaDA result is not a benchmark score. It is a capability that comes from the training objective itself.

The reversal curse is a documented failure of autoregressive models: a model trained on "A is B" often cannot answer "what is B?" with A. It is not a memory failure, it is a factorization artefact. An autoregressive model is trained to estimate $p(\text{next} \mid \text{prefix})$ and only ever sees the forward direction, so nothing in training ever asks it to retrieve a subject from its object.

Masked diffusion has no preferred direction. Because masking is applied at random positions, the model is trained to infer any subset of tokens from any other subset. Predicting the subject given the object is an ordinary training example rather than an out-of-distribution request.

The empirical demonstration used a reversed-order poem completion task, where LLaDA outperformed strong autoregressive systems including GPT-4o by a wide margin, with consistent performance in both the forward and reversed directions.

This is the clearest case of a structural property producing a capability difference rather than a speed difference.

Training 8 billion parameters from scratch costs what it costs, which puts the from-scratch route out of reach for most groups. The second answer avoids it entirely.

Dream 7B, from Jiacheng Ye, Zhihui Xie and colleagues, initialises from an existing autoregressive checkpoint, Qwen2.5 7B, and converts it into a diffusion model through continued training rather than starting from random weights.

The insight is that most of what a language model knows is not about generation order. Syntax, world knowledge, and representation quality are direction-agnostic and transfer. What has to change is narrower: the attention mask goes from causal to bidirectional, and the objective goes from next-token prediction to masked denoising. Dream also uses adaptive noise scheduling, adjusting the corruption level per token rather than applying one global ratio.

The reported outcome is a model performing in the range of larger autoregressive models such as LLaMA3-8B, at a fraction of the from-scratch cost.

This matters strategically. If diffusion models can be built by converting existing checkpoints, the paradigm inherits the entire investment already made in autoregressive pre-training.

The from-scratch and adaptation routes converge on the same artefact through very different cost structures. Adaptation reuses the knowledge in an autoregressive checkpoint and pays only to change the attention pattern and objective; from-scratch pays full price but is not constrained by inherited left-to-right structure.

There is a real efficiency asymmetry in training, and it is worth stating precisely because it is often either ignored or exaggerated.

In an autoregressive forward pass, the causal mask means every position is simultaneously a prediction target. A sequence of length $L$ yields $L$ supervised predictions from one pass, which is the quiet reason next-token prediction is such an efficient objective.

In masked diffusion, only masked positions contribute to the loss. With the masking ratio drawn uniformly from $(0, 1]$, the expected fraction of supervised positions is one half, so a sequence of length $L$ yields about $L/2$ supervised predictions per pass. Unmasked positions still cost full attention compute while contributing no gradient signal.

So a naive comparison suggests roughly half the supervision per unit of compute. Two things soften it. Each masked prediction conditions on bidirectional context and is therefore a harder, more informative training signal than a left-only prediction. And the objective is genuinely different: the model learns to fill any subset from any other, which is a broader task than next-token prediction.

The fair conclusion is that diffusion training is somewhat less compute-efficient per token, not catastrophically so.

Alignment methods were all designed for autoregressive models, and porting them exposes a mismatch specific to diffusion.

The problem is a training and inference divide. Training masks positions at random and predicts them in one shot. Inference unmasks progressively over many rounds, so the model sees partially-decoded states whose statistics differ from anything in training. Reinforcement learning makes this acute, because computing the log-probability of a generated sequence requires a likelihood the model only bounds, and credit assignment must span a multi-round decoding trajectory rather than a token sequence.

Several lines of work address it directly. MDPO targets the training-inference divide for preference optimisation on masked diffusion models. Other approaches apply reinforcement learning along the decoding trajectory itself, treating the sequence of unmasking decisions as the thing to optimise, which also lets them reward using fewer decoding steps.

Sparse mixture-of-experts has arrived too, with LLaDA-MoE demonstrating that conditional computation composes with diffusion pre-training in the way it does for autoregressive models.

The pattern is consistent: nothing about alignment is impossible here, but very little transfers unchanged.

A practical comparison of the two routes, plus the gotchas each carries.

Three gotchas are worth carrying forward. First, benchmark parity is not parity: the models compared here differ in data, tokenizer, and training budget, so single-number comparisons across paradigms carry large error bars. Second, an adapted model may retain left-to-right habits from its initialisation, and whether it fully inherits the reversal-curse fix is a question to check rather than assume. Third, and most consequential, none of these training results say anything about serving cost.

A model that trains well and serves slowly is not deployable, and the naive diffusion serving story is genuinely bad. That is the subject of the next lesson.