Differentiating through the Gamma


Suppose I want to find a distributional gradient for a gamma process. Generalically I woudl find this via monte carlo gradient estimation.

Here is a problem-specific method:

I allow the latent random state to have more dimensions than a univariate. Let’s get specific. An example arises if we raid the random-variate-generation literature for transform methods to generate RNGs and differentiate A Gamma variate can be generated by a transformed normal and a uniform random variable,or two uniforms, depending on the parameter range. Beta variates can come from two gamma variates.

Here is Devroye’s summary for beta generators for \(0 < a, b\leq 1\) and Gamma generators for \(a<1\):

Johnk’s beta generator

REPEAT Generate iid uniform [0,1] random variates \(U, V\)

\[ \begin{array}{c} X \leftarrow U^{\frac{1}{a}}, Y-V^{\frac{1}{b}} \\ \text { UNTL } X+Y \leq 1 \end{array} \]

RETURN \(\frac{X}{X+Y}\)

\((X \text { is beta }(a, b) \text { distributed })\)

Berman’s beta generator

REPEAT Generate iid uniform [0,1] random variates \(U, V\)

\[ \begin{array}{c} X \leftarrow U^{\frac{1}{4}}, Y \leftarrow V^{\frac{1}{b}} \\ \text { UNTIL } X+Y \leq \mathbf{1} \end{array} \]

RETURN \(X\)

\((X \text { is beta }(a, b+1) \text { distributed })\)

Johnk’s gamma generator

REPEAT Generate iid uniform [0,1] random variates \(U, V\)

\[ \begin{array}{r} X+U^{\frac{1}{a}}, Y \leftarrow V^{\frac{1}{1-a}} \\ \text { UNTL } X+Y \leq 1 \end{array} \]

Generate an exponential random variate \(E\) RETURN \(\frac{E X}{X+Y}\)

\((X \text { is gamma ( } a\) ) distributed)

Berman’s gamma generator

REPEAT Generate iid uniform [0,1] random variates \(U, V\)

\[ \begin{array}{r} X \leftarrow U^{\frac{1}{4}}, Y \leftarrow V^{\frac{1}{1-a}} \\ \text { UNTL } X+Y \leq 1 \end{array} \]

Generate a gamma ( 2 ) random variate \(Z\) (either as the sum of two iid exponential random variates or as \(-\log (U * V *)\) where \(U *, V *\) are lid uniform [0,1] random variates

RETURN \(Z X\)

\((X \text { is gamma }(a) \text { distributed })\)

Ahrens, J. H., and U. Dieter. 1974. “Computer Methods for Sampling from Gamma, Beta, Poisson and Bionomial Distributions.” Computing 12 (3): 223–46. https://doi.org/10.1007/BF02293108.

———. 1982. “Generating Gamma Variates by a Modified Rejection Technique.” Communications of the ACM 25 (1): 47–54. https://doi.org/10.1145/358315.358390.

Devroye, Luc. 1986. Non-Uniform Random Variate Generation. New York: Springer.