# Generative adversarial networks

October 7, 2016 — December 14, 2020

Game theory meets learning. Hip, especially in combination with deep learning, because it provides an elegant means of likelihood free inference.

I don’t know anything about it. Something about training two systems together to both generate and classify examples of a phenomenon of interest.

Sanjeev Arora gives a cogent intro. He also suggests a link with learning theory. See also Delving deep into Generative Adversarial Networks, a “curated, quasi-exhaustive list of state-of-the-art publications and resources about Generative Adversarial Networks (GANs) and their applications.”

GANs are famous for generating images, but I am interested in their use in simulating from difficult distributions in general.

Try a spreadsheet interface for exploring GAN latent spaces. See also The GAN Zoo, “A list of all named GANs!”

To discover: precise relationship of deep GANs with, e.g. adversarial training in games and bandit problems. Also, why not, let us consider Augustus Odena’s Open Questions about GANs.

## 1 Wasserstein GAN

A tasty hack. The Wasserstein GAN paper (Arjovsky, Chintala, and Bottou 2017) made a splash. The argument is that, kinda-sorta if we squint at it, we can understand the GAN as solving an inference problem with respect to Wasserstein loss. The argument has since been made more precise and extended, but for all its flaws the original article has IMO a good insight and a clear explanation of it.

I will not summarize WGANs better than the following handy sources so let us read these.

- Sanjeev Arora, one of the WGAN authors, critiques the work they did there in Generalization and Equilibrium in Generative Adversarial Networks
- Alexi Pan reads the WGAN paper.
- Mindcodec discusses Wasserstein-type metrics, i.e. optimal transport ones, with an eye to WGAN.
- a deep learning course that culminates in WGAN with some involvement by the authors of the WGAN paper.

Vincent Hermann presents the Kontorovich-Rubinstein duality trick intuitively.

Connection to other types of regularisation? (Gulrajani et al. 2017; Miyato et al. 2018)

## 2 Conditional

How does this work? There are many papers exploring that. How about these two? Mirza and Osindero (2014);(**Isola2017ImagetoImage?**)

## 3 Invertible

I think this requires *cycle consistent loss*, whatever that is? (J.-Y. Zhu et al. 2017) How is it different to autoencoders? I suppose because it maps *between two domains* not *between a latent and a domain*.

## 4 Spectral normalization

Miyato and Koyama (2018);Miyato et al. (2018)

pfnet-research/sngan_projection: GANs with spectral normalization and projection discriminator

## 5 GANs as SDEs

Should look into this (L. Yang, Zhang, and Karniadakis 2020; Kidger et al. 2021).

## 6 GANs as VAEs

See deep generative models for a unifying framing.

## 7 GANs as energy-based models

Che et al. (2020)

## 8 Incoming

## 9 References

*International Conference on Machine Learning*.

*arXiv:1703.00573 [Cs]*.

*arXiv:1702.02604 [Cs, Stat]*.

*Inverse Problems*.

*arXiv:1704.03809 [Cs]*.

*International Conference on Machine Learning*.

*arXiv:1511.06349 [Cs]*.

*Advances in Neural Information Processing Systems 29*.

*arXiv:2003.06060 [Cs, Stat]*.

*ACM Transactions on Graphics*.

*arXiv:1506.05751 [Cs]*.

*ICLR 2019*.

*arXiv:1411.5928 [Cs]*.

*Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence*. UAI’15.

*PMLR*.

*Advances in Neural Information Processing Systems 29*.

*arXiv:1904.12795 [Cs]*.

*Advances in Approximate Bayesian Inference Workshop, NIPS*.

*4th International Conference on Learning Representations (ICLR) Workshop Track*.

*Advances in Neural Information Processing Systems 27*. NIPS’14.

*arXiv:1412.6572 [Cs, Stat]*.

*arXiv:1502.04623 [Cs]*.

*arXiv:1704.00028 [Cs, Stat]*.

*Advances in Neural Information Processing Systems*.

*Trends in Cognitive Sciences*.

*arXiv:2006.04349 [Cs, Stat]*.

*Advances in Neural Information Processing Systems*.

*2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*.

*Advances in Neural Information Processing Systems 29*.

*Proceedings of the 38th International Conference on Machine Learning*.

*arXiv:1705.07215 [Cs]*.

*Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence*.

*arXiv:1503.03167 [Cs]*.

*Proceedings of the 26th Annual International Conference on Machine Learning*. ICML ’09.

*Advances in Neural Information Processing Systems 30*.

*arXiv Preprint arXiv:1603.04733*.

*arXiv:1411.1784 [Cs, Stat]*.

*ICLR 2018*.

*Proceedings of The 31st International Conference on Machine Learning*. ICML’14.

*IEEE Transactions on Audio, Speech, and Language Processing*.

*Annual Review of Statistics and Its Application*.

*arXiv:1904.03418 [Cs, Eess]*.

*arXiv:1610.01945 [Cs, Stat]*.

*Advances in Neural Information Processing Systems 29*.

*Advances in Neural Information Processing Systems*.

*arXiv:1511.06434 [Cs]*.

*Proceedings of ICML*.

*Annual Review of Statistics and Its Application*.

*Proceedings of the 30th International Conference on Neural Information Processing Systems*. NIPS’16.

*arXiv:1611.05416 [Cs]*.

*Proceedings of ICLR*.

*arXiv:1506.03478 [Cs, Stat]*.

*ICLR*.

*arXiv:1601.06759 [Cs]*.

*arXiv:1805.08349 [Cond-Mat, Stat]*.

*International Conference on Machine Learning*.

*Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR’2017), Suzhou, China*.

*SIAM Journal on Scientific Computing*.

*Journal of Computational Physics*.

*IEEE Transactions on Information Theory*.

*Proceedings of European Conference on Computer Vision*.