# Random binary vectors

## The class of distributions that cause you to reinvent Shannon information if you stare at them long enough

Robert Fludd’s piano rolls

Distributions over random boolean vectors. Useful in computer science and piano rolls. Not quite the same as categorical distributions, although those can be written as distributions over boolean vectors, but in a multi-class classification case each realisation has only one class; in an $$n$$-class rv, there are $$n$$ possible outcomes. In a multivariate Bernoulli distribution there are $$2^n$$ possible outcomes.

## Continuous relaxations

Multivariate Gumbel-softmax tricks.

## Paintbox models

Not sure how these work but maybe related. See .

## Matrix models

TBC.

See, e.g. Lumbreras, Filstroff, and Févotte (2020)

## References

Broderick, Tamara, Jim Pitman, and Michael I. Jordan. 2013. Bayesian Analysis 8 (4): 801–36.
Dai, Bin, Shilin Ding, and Grace Wahba. 2013. Bernoulli 19 (4).
Lumbreras, Alberto, Louis Filstroff, and Cédric Févotte. 2020. Data Mining and Knowledge Discovery 34 (6): 1898–1935.
Miettinen, Pauli, and Stefan Neumann. 2020. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, 4922–28. Yokohama, Japan: International Joint Conferences on Artificial Intelligence Organization.
Reuter, Stephan, Ba-Tuong Vo, Ba-Ngu Vo, and Klaus Dietmayer. 2014. IEEE Transactions on Signal Processing 62 (12): 15.
Rukat, Tammo, Chris C. Holmes, Michalis K. Titsias, and Christopher Yau. 2017. In Proceedings of the 34th International Conference on Machine Learning, 2969–78. PMLR.
Teugels, Jozef L. 1990. Journal of Multivariate Analysis 32 (2): 256–68.
Vo, Ba Ngu, Ba Tuong Vo, and Hung Gia Hoang. 2017. arXiv:1606.08350 [Stat], February.
Vo, Ba-Ngu, Ba-Tuong Vo, and Dinh Phung. 2014. IEEE Transactions on Signal Processing 62 (24): 6554–67.
Vo, Ba-Tuong, and Ba-Ngu Vo. 2013. IEEE Transactions on Signal Processing 61 (13): 3460–75.
Vo, Ba-Tuong, Ba-Ngu Vo, and Antonio Cantoni. 2009. IEEE Transactions on Signal Processing 57 (2): 409–23.
Wang, Xi, and Junming Yin. 2020. In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), 500–509. PMLR.
Zhang, Aonan, and John Paisley. 2019. In International Conference on Machine Learning, 7424–33.
Zhou, Mingyuan, Lauren Hannah, David Dunson, and Lawrence Carin. 2012. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, 1462–71. PMLR.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.