← Back to feed

Every major improvement to image diffusion transformers over the last two years targeted the same set of components: attention mechanisms, tokenizers, conditioning methods, autoencoders, loss functions. None of them touched the residual stream — the running sum that layers write to and read from as information flows through the network.

A new paper on arxiv (2605.20708), “Rethinking Cross-Layer Information Routing in Diffusion Transformers,” argues that the residual stream was the bottleneck the whole time, and demonstrates a fix that reaches equivalent image quality in 8.75 times fewer training iterations.

What Was Wrong

Standard transformer residual streams accumulate layer outputs additively. Each layer adds its contribution to a running total. This works adequately for language models, where the task is roughly uniform across sequence positions. For diffusion models, it creates three compounding problems the authors quantify:

1. Forward magnitude explosion. Layer outputs accumulate without bound, driving the stream to large magnitudes that destabilize later layers.
2. Backward gradient fade. The same exponential growth in forward magnitude produces vanishing gradients during backpropagation.
3. Feature collapse. Adjacent layers in a static residual stack tend to compute nearly identical representations — they converge to similar outputs because they’re reading and writing to the same running stream without differentiation.

The third problem is particularly damaging for diffusion: denoising is not one task. A model denoising an image at high noise levels is doing something structurally different from the same model denoising at near-zero noise. A static residual stream treats both identically.

The Fix: Per-Timestep Layer Selection

Diffusion-Adaptive Routing (DAR) replaces the fixed residual connection with a learned routing function. Instead of every layer adding to a shared stream, each layer selects which earlier layer outputs to incorporate — and that selection changes with the denoising timestep.

The routing function is lightweight. It does not add a new loss, a new tokenizer, a new dataset, or a new attention mechanism. The only change is in how layer outputs propagate forward through the network. The model learns, during training, that high-noise timesteps benefit from information flows that differ from low-noise timesteps, and routes accordingly.

The result: the same final image quality that standard DiT architectures reach after a full training run arrives after 1 in 8.75 of the original iterations.

Why This Matters Beyond Training Cost

Training cost reduction is the obvious headline. But the implications extend to existing deployed models. The architectural insight — that adjacent DiT layers produce near-identical features because of static routing — suggests that post-training techniques targeting redundancy (layer pruning, speculative decoding analogues for diffusion) have more headroom than previously assumed.

It also raises a broader methodological point. The diffusion community has spent significant effort on attention efficiency (Flash-style optimizations), multimodal conditioning, and output representation. The residual stream — the mechanism by which all of those components share information — was treated as infrastructure, not as a design choice. This paper is an argument that treating it as infrastructure left significant performance on the table.

The code and checkpoints are not yet released as of the paper posting date.