Random binary vectors

The class of distributions that cause you to reinvent Shannon information if you stare at them long enough



Robert Fludd’s piano rolls

Distributions over random boolean vectors. Useful in computer science and piano rolls. Not quite the same as categorical distributions, although those can be written as distributions over boolean vectors, but in a multi-class classification case each realisation has only one class; in an \(n\)-class rv, there are \(n\) possible outcomes. In a multivariate Bernoulli distribution there are \(2^n\) possible outcomes.

Continuous relaxations

Multivariate Gumbel-softmax tricks.

Paintbox models

Not sure how these work but maybe related. See (Broderick, Pitman, and Jordan 2013; Zhang and Paisley 2019).

Matrix models

TBC.

See, e.g. Lumbreras, Filstroff, and FΓ©votte (2020)

References

Broderick, Tamara, Jim Pitman, and Michael I. Jordan. 2013. β€œFeature Allocations, Probability Functions, and Paintboxes.” Bayesian Analysis 8 (4): 801–36.
Dai, Bin, Shilin Ding, and Grace Wahba. 2013. β€œMultivariate Bernoulli Distribution.” Bernoulli 19 (4).
Lumbreras, Alberto, Louis Filstroff, and CΓ©dric FΓ©votte. 2020. β€œBayesian Mean-Parameterized Nonnegative Binary Matrix Factorization.” Data Mining and Knowledge Discovery 34 (6): 1898–1935.
Miettinen, Pauli, and Stefan Neumann. 2020. β€œRecent Developments in Boolean Matrix Factorization.” In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, 4922–28. Yokohama, Japan: International Joint Conferences on Artificial Intelligence Organization.
Reuter, Stephan, Ba-Tuong Vo, Ba-Ngu Vo, and Klaus Dietmayer. 2014. β€œThe Labeled Multi-Bernoulli Filter.” IEEE Transactions on Signal Processing 62 (12): 15.
Rukat, Tammo, Chris C. Holmes, Michalis K. Titsias, and Christopher Yau. 2017. β€œBayesian Boolean Matrix Factorisation.” In Proceedings of the 34th International Conference on Machine Learning, 2969–78. PMLR.
Teugels, Jozef L. 1990. β€œSome Representations of the Multivariate Bernoulli and Binomial Distributions.” Journal of Multivariate Analysis 32 (2): 256–68.
Vo, Ba Ngu, Ba Tuong Vo, and Hung Gia Hoang. 2017. β€œAn Efficient Implementation of the Generalized Labeled Multi-Bernoulli Filter.” arXiv:1606.08350 [Stat], February.
Vo, Ba-Ngu, Ba-Tuong Vo, and Dinh Phung. 2014. β€œLabeled Random Finite Sets and the Bayes Multi-Target Tracking Filter.” IEEE Transactions on Signal Processing 62 (24): 6554–67.
Vo, Ba-Tuong, and Ba-Ngu Vo. 2013. β€œLabeled Random Finite Sets and Multi-Object Conjugate Priors.” IEEE Transactions on Signal Processing 61 (13): 3460–75.
Vo, Ba-Tuong, Ba-Ngu Vo, and Antonio Cantoni. 2009. β€œThe Cardinality Balanced Multi-Target Multi-Bernoulli Filter and Its Implementations.” IEEE Transactions on Signal Processing 57 (2): 409–23.
Wang, Xi, and Junming Yin. 2020. β€œRelaxed Multivariate Bernoulli Distribution and Its Applications to Deep Generative Models.” In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), 500–509. PMLR.
Zhang, Aonan, and John Paisley. 2019. β€œRandom Function Priors for Correlation Modeling.” In International Conference on Machine Learning, 7424–33.
Zhou, Mingyuan, Lauren Hannah, David Dunson, and Lawrence Carin. 2012. β€œBeta-Negative Binomial Process and Poisson Factor Analysis.” In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, 1462–71. PMLR.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.