Neural denoising diffusion models of language
March 12, 2025 — March 12, 2025
approximation
Bayes
generative
Monte Carlo
neural nets
optimization
probabilistic algorithms
probability
score function
statistics
Suspiciously similar content
Neural diffusion models, but for generating words instead of pictures. A special kind of discrete diffusion.
1 Incoming
- Inception Labs’s Mercury coder is very hyped right now
- GitHub - dvruette/gidd: Code accompanying the paper “Generalized Interpolating Discrete Diffusion”
2 References
Ghazvininejad, Levy, Liu, et al. 2019. “Mask-Predict: Parallel Decoding of Conditional Masked Language Models.”
Li, Thickstun, Gulrajani, et al. 2022. “Diffusion-LM Improves Controllable Text Generation.” In.
Rütte, Fluri, Ding, et al. 2025. “Generalized Interpolating Discrete Diffusion.”
Savinov, Chung, Binkowski, et al. 2022. “Step-Unrolled Denoising Autoencoders for Text Generation.”
Strudel, Tallec, Altché, et al. 2022. “Self-Conditioned Embedding Diffusion for Text Generation.”
Ye, Gong, Chen, et al. 2024. “Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models.” In.
Zheng, Yuan, Yu, et al. 2024. “A Reparameterized Discrete Diffusion Model for Text Generation.” In.
Zou, Kim, and Kang. 2023. “A Survey of Diffusion Models in Natural Language Processing.”