T3D: Few-Step Diffusion Language Models via Trajectory Self-Distillation with Direct Discriminative Optimization
T3D (Trajectory Self-Distillation via DDO) is a self-distillation framework for diffusion large language models (DLLMs) that trains a few-step student by matching the teacher’s generation trajectories. We show theoretically that trajectory-level distillation enables few-step decoding by reducing factorization error, and empirically that T3D consistently outperforms existing few-step decoding baselines.
Paper (Preprint) Code Slides DIFFUSION LANGUAGE MODEL FEW-STEP GENERATION