Pólya-Gamma augmentation trick

An infinite weird RV useful in Bayesian Binomial regression (and maybe other things?) (Polson, Scott, and Windle 2013). C&C the optimization-driven approach to a similar problem in Gumbel-max tricks.

See also my former colleague, Louis Tiao, A Primer on Pólya-gamma Random Variables - Part II: Bayesian Logistic Regression.

Gregory Gunderson, in Pólya-Gamma Augmentation, explains the problem we are trying to solve.

…in logistic regression, the dependent variables are assumed to be i.i.d. from a Bernoulli distribution with parameter $p$, and therefore the likelihood function is $$\mathcal{L}(p)\propto \prod _{n=1}^N{p}^{y_n}(1-p{)}^{1-y_n}=p^{\sum y_n}(1-p{)}^{N-\sum y_n}$$ The observations interact with the response through a linear relationship with the log-odds, $$\log \left(\frac{p}{1-p}\right)={\beta }_0+x_1{\beta }_1+x_2{\beta }_2+\cdots +x_D{\beta }_D={\beta }^{\mathrm{\top }}\mathbf{x}$$ If we solve for $p$ in (2), we get $$p=\frac{\exp \left({\mathit{\beta }}^{\mathrm{\top }}{\mathbf{x}}_n\right)}{1+\exp \left({\mathit{\beta }}^{\mathrm{\top }}{\mathbf{x}}_n\right)}$$ and a likelihood of $$\mathcal{L}(\mathit{\beta })\propto \frac{{[\exp \left({\mathit{\beta }}^{\mathrm{\top }}\mathbf{x}\right)]}^{\sum y_n}}{{[1+\exp \left({\mathit{\beta }}^{\mathrm{\top }}\mathbf{x}\right)]}^N}$$ Due to this functional form, Bayesian inference for logistic regression is intractable.

Using Pólya-Gamma RVs we devise an auxiliary variable sample.