Neural denoising diffusion models of language

March 12, 2025 — March 12, 2025

approximation
Bayes
generative
Monte Carlo
neural nets
optimization
probabilistic algorithms
probability
score function
statistics
Figure 1

Neural diffusion models, but for generating words instead of pictures. A special kind of discrete diffusion.

1 Incoming

2 References

Ghazvininejad, Levy, Liu, et al. 2019. Mask-Predict: Parallel Decoding of Conditional Masked Language Models.”
Li, Thickstun, Gulrajani, et al. 2022. Diffusion-LM Improves Controllable Text Generation.” In.
Rütte, Fluri, Ding, et al. 2025. Generalized Interpolating Discrete Diffusion.”
Savinov, Chung, Binkowski, et al. 2022. Step-Unrolled Denoising Autoencoders for Text Generation.”
Strudel, Tallec, Altché, et al. 2022. Self-Conditioned Embedding Diffusion for Text Generation.”
Ye, Gong, Chen, et al. 2024. Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models.” In.
Zheng, Yuan, Yu, et al. 2024. A Reparameterized Discrete Diffusion Model for Text Generation.” In.
Zou, Kim, and Kang. 2023. A Survey of Diffusion Models in Natural Language Processing.”