Neural denoising diffusion models of language
2025-03-12 — 2025-03-12
Wherein a class of neural diffusion techniques is adapted to discrete text tokens, and a non‑Gaussian, discrete diffusion process is described as an alternative to autoregressive generation in low‑data regimes
approximation
Bayes
generative
Monte Carlo
neural nets
optimization
probabilistic algorithms
probability
score function
statistics
Neural diffusion models, but for generating words instead of pictures. A special kind of discrete diffusion.
1 Incoming
2 References
Ghazvininejad, Levy, Liu, et al. 2019. “Mask-Predict: Parallel Decoding of Conditional Masked Language Models.”
Li, Thickstun, Gulrajani, et al. 2022. “Diffusion-LM Improves Controllable Text Generation.” In.
Prabhudesai, Wu, Zadeh, et al. 2025. “Diffusion Beats Autoregressive in Data-Constrained Settings.”
Rütte, Fluri, Ding, et al. 2025. “Generalized Interpolating Discrete Diffusion.”
Savinov, Chung, Binkowski, et al. 2022. “Step-Unrolled Denoising Autoencoders for Text Generation.”
Strudel, Tallec, Altché, et al. 2022. “Self-Conditioned Embedding Diffusion for Text Generation.”
Ye, Gong, Chen, et al. 2024. “Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models.” In.
Zheng, Yuan, Yu, et al. 2024. “A Reparameterized Discrete Diffusion Model for Text Generation.” In.
Zou, Kim, and Kang. 2023. “A Survey of Diffusion Models in Natural Language Processing.”