Neural denoising diffusion models on discrete state spaces

2024-04-16 — 2025-05-16

Wherein diffusion is transferred to discrete state spaces, and token‑level corruption schedules are employed to train neural denoising models for categorisation

approximation
Bayes
classification
generative
Monte Carlo
neural nets
optimization
probabilistic algorithms
probability
score function
statistics
Figure 1

Placeholder. We discuss Diffusion models that don’t assume continuous output spaces. They’re particularly useful for language diffusions.

1 References

Austin, Johnson, Ho, et al. 2021. “Structured Denoising Diffusion Models in Discrete State-Spaces.” In Proceedings of the 35th International Conference on Neural Information Processing Systems. NIPS ’21.
Dieleman, Sartran, Roshannai, et al. 2022. “Continuous Diffusion for Categorical Data.”
Gruver, Stanton, Frey, et al. n.d. “Protein Design with Guided Discrete Diffusion.”
Jo, and Hwang. 2025. “Continuous Diffusion Model for Language Modeling.”
Jo, Kim, and Hwang. 2024. “Graph Generation with Diffusion Mixture.”
Prabhudesai, Wu, Zadeh, et al. 2025. “Diffusion Beats Autoregressive in Data-Constrained Settings.”
Rütte, Fluri, Ding, et al. 2025. “Generalized Interpolating Discrete Diffusion.”
Savinov, Chung, Binkowski, et al. 2022. “Step-Unrolled Denoising Autoencoders for Text Generation.”
Zheng, Yuan, Yu, et al. 2024. “A Reparameterized Discrete Diffusion Model for Text Generation.” In.
Zou, Kim, and Kang. 2023. “A Survey of Diffusion Models in Natural Language Processing.”