# Gamma processes


Gamma processes provide the classic subordinator models, i.e. non-decreasing Lévy processes. By “gamma process” in fact I mean specifically a Lévy process with gamma increments.

Gamma processes are a natural model for spiky things

Other processes that happen to have gamma marginals, e.g. are, rather, extensions and are handled separately..

Ground zero for these processes appears to be , and then the weaponisation of these processes to construct the Dirichlet prior in .

Tutorial introductions to gamma processes can be found in . Existence proofs etc are deferred to those sources. You could also see Wikipedia, although that article is not particularly helpful.

The marginal distribution of the an increment of duration $$t$$ is given by the gamma distribution, which we had better cover first.

## Gamma distribution

Let us take a brief divergence into the gamma distribution which is the increment distribution the gamma process (Indeed, all divisible distributions induce Lévy processes.)

The density $$g(x;t,\alpha, \lambda )$$ of the univariate gamma is

$g(x; \alpha, \lambda)= \frac{ \lambda^{\alpha} }{ \Gamma (\alpha) } x^{\alpha\,-\,1}e^{-\lambda x}, x\geq 0.$ This is the shape-rate parameterisation, with rate $$\lambda$$ and shape $$\alpha,$$. We can think of the gamma distribution as the distribution at time 1 of a gamma process.

If $$\rv{g}\sim \operatorname{Gamma}(\alpha, \lambda)$$ then $$\bb E(\rv{g})=\alpha/\lambda$$ and $$\var(\rv{g})=\alpha/\lambda^2.$$

We use various facts about the gamma distribution which quantify its divisibility properties.

1. If $$\rv{g}_1\sim \operatorname{Gamma}(\alpha_1, \lambda),\,\rv{g}_2\sim \operatorname{Gamma}(\alpha_2, \lambda),$$ and $$\rv{g}_1\perp \rv{g}_2,$$ then $$\rv{g}_1+\rv{g}_2\sim \operatorname{Gamma}(\alpha_1+\alpha_2, \lambda)$$ (additive rule)
2. If $$\rv{g}\sim \operatorname{Gamma}(\alpha, \lambda)$$ then $$c \rv{g}\sim \operatorname{Gamma}(\alpha, \lambda/c)$$ (multiplicative rule)
3. If $$\rv{g}_1\sim \operatorname{Gamma}(\alpha_1, \lambda)\perp \rv{g}_1\sim \operatorname{Gamma}(\alpha_2, \lambda)$$ then $$\frac{\rv{g}_1}{\rv{g}_1+\rv{g}_2}\sim \operatorname{Beta}(\alpha_1, \alpha_2)$$ independent of $$\rv{g}_1+\rv{g}_2$$ (stick-breaking rule)

### Moments

Also note that the moment generating function of the gamma distribution is

$\bb{E}[\exp(\rv{g} s)]=\left(1-{\frac {s}{\lambda }}\right)^{-\alpha }{\text{ for }}s<\lambda$ which gives us expressions for all moments fairly easily;

\begin{aligned} \bb{E}[\rv{g}]&=\left.\frac{\dd}{\dd s}\bb{E}[\exp(\rv{g} t)]\right|_{s=0}\\ &=\left.\frac{\dd}{\dd s}\left(1-{\frac {s}{\lambda }}\right)^{-\alpha }\right|_{s=0}\\ &=\left.\frac{\alpha}{\lambda}\left(1-{\frac {s}{\lambda }}\right)^{-\alpha -1}\right|_{s=0}\\ &=\alpha/\lambda\\ \bb{E}[\rv{g}^2] &=\left.\frac{\dd}{\dd s}\frac{\alpha}{\lambda}\left(1-{\frac {s}{\lambda }}\right)^{-\alpha -1}\right|_{s=0}\\ &=\left.\frac{\alpha^2+\alpha}{\lambda^2}\left(1-{\frac {s}{\lambda }}\right)^{-\alpha -2}\right|_{s=0}\\ &=\frac{\alpha(\alpha+1)}{\lambda^2}\\ \bb{E}[\rv{g}^3] &=\left.\frac{\dd}{\dd s}\frac{\alpha^2+\alpha}{\lambda^2}\left(1-{\frac {s}{\lambda }}\right)^{-\alpha -2}\right|_{s=0}\\ &=\frac{\alpha(\alpha+1)(\alpha+2)}{\lambda^3}\left(1-{\frac {s}{\lambda }}\right)^{-\alpha -2}\\ &=\frac{\alpha(\alpha+1)(\alpha+2)}{\lambda^3}\\ \bb{E}[\rv{g}^4] &=\frac{\alpha(\alpha+1)(\alpha+2)(\alpha+3)}{\lambda^4}\\ &\dots\\ \bb{E}[\rv{g}^n] &=\frac{\langle \alpha \rangle_{n}}{\lambda^n}\\ \end{aligned}

Here $$\langle \alpha \rangle_{n}:=\frac{\Gamma(\alpha+n)}{\Gamma(\alpha)}$$ is the rising factorial.

## Multivariate gamma distribution with dependence

I am not sure what general correlations are possible here, but one obvious possibility is to choose a transform matrix $$M$$ with non-negative entries. Then the RV $$\{M\rv{g}(t)\}$$ is still marginally a gamma variate, but the components of the vector are no longer independent. Is this the most general possible gamma distribution? What is the covariance structure of that process? 🏗

## Gamma superpositions

If we wish to know the distribution of the sum of a set of scaled gamma random variables, we can use moment-generating-function approaches . It does not come out so simply as in the Gaussian case, with a recursive coefficient definition.

## The Gamma process

The univariate gamma process $$\{\rv{g}(t;\alpha,\lambda)\}_t$$ is an independent-increment process, with time index $$t$$ and parameters by $$\alpha, \lambda.$$ We assume it is started at $$\rv{g}(0)=0$$.

The marginal density $$g(x;t,\alpha, \lambda )$$ of the process at time $$t$$ is a gamma RV, specifically, $g(x;t, \alpha, \lambda) =\frac{ \lambda^{\alpha t} } { \Gamma (\alpha t) } x^{\alpha t\,-\,1}e^{-\lambda x}, x\geq 0.$ That is, $$\rv{g}(t) \sim \operatorname{Gamma}(\alpha(t_{i+1}-t_{i}), \lambda)$$.

which corresponds to increments per unit time in terms of $$\bb E(\rv{g}(1))=\alpha/\lambda$$ and $$\var(\rv{g}(1))=\alpha/\lambda^2.$$

Note that if $$\alpha t=1,$$ then $$\rv{g}(t;\alpha ,\lambda )\sim \operatorname{Exp}(\lambda).$$

This leads to a method for simulating a path of a gamma process at a sequence of increasing times, $$\{t_1, t_2, t_3, t_L\}.$$ Given $$\rv{g}(t_1;\alpha, \lambda),$$ we know that the increments are distributed as independent variates $$\rv{g}_i:=\rv{g}(t_{i+1})-\rv{g}(t_{i})\sim \operatorname{Gamma}(\alpha(t_{i+1}-t_{i}), \lambda)$$. Presuming we may simulate from the Gamma distribution, it follows that

$\rv{g}(t_i)=\sum_{j < i}\left( \rv{g}(t_{i+1})-\rv{g}(t_{i})\right)=\sum_{j < i} \rv{g}_j.$

A standard $$d$$-dimensional gamma process is the concatenation of $$d$$ independent univariate gamma processes.

## Gamma bridge

Consider a univariate gamma process, $$\rv{g}(t)$$ with $$\rv{g}(0)=0.$$ The gamma bridge, analogous to the Brownian bridge, is the gamma process conditional upon attaining a fixed the value $$S=\rv{g}(1)$$ at terminal time $$1.$$ We write $$\rv{g}_{S}:=\{\rv{g}(t)\mid \rv{g}(1)=S\}_{0< t < 1}$$ for the paths of this process.

We can simulate from the gamma bridge easily. Given the increments of the process are independent, if we have a gamma process $$\rv{g}$$ on the index set $$[0,1]$$ such that $$\rv{g}(1)=S$$, then we can simulate from the bridge paths which connect these points at intermediate time $$t,\, 0<t<1$$ by recalling that we have known distributions for the increments; in particular $$\rv{g}(t)\sim\operatorname{Gamma}(\alpha, \lambda)$$ and $$\rv{g}(1)-\rv{g}(t)\sim\operatorname{Gamma}(\alpha (1-t), \lambda)$$ and these increments, as increments over disjoints sets, are themselves independent. Then, by the stick breaking rule,

$\frac{\rv{g}(t)}{\rv{g}(1)}\sim\operatorname{Beta}(\alpha t, \alpha(1-t))$ independent of $$\rv{g}(1).$$ We can therefore sample from a path of the bridge $$\rv{g}_{S}(t)$$ for some $$t< 1$$ by simulating $$\rv{g}_{S}(t)=B S,$$ where $$B\sim \operatorname{Beta}(\alpha (t),\alpha (1-t)).$$

## Time-warped gamma process

walks us through the mechanics of (deterministically) time-warping Gamma processes, which ends up being not too unpleasant. Predictable stochastic time-warps look like they should be OK. See for an application. Why bother? Linear superpositions of gamma processes can be hard work, and sometime the generalisation from time-warping can come out nicer. 🏗

## Matrix gamma processes

I am tempted to look at matrix-valued extensions, much as the Wishart distribution is a matrix valued extension of the gamma distribution. “Wishart processes” are indeed a thing but the common definition, unlike the common definition of the gamma process, is not necessarily (element-wise, or in any other sense) monotonic. That is, it generalises the square Bessel process, which is indeed marginally gamma distributed (specifically $$\chi^2$$ distributed) but also non-monotonic, so it is not a natural extension of what we have here.

## Centred gamma process

Define $$H_t:=\rv{g}_t-\alpha t /\lambda.$$ Then we have a mean-zero process, in fact, a martingale, since we have subtracted the compensator from it.

We call the marginal distribution at time $$t=1$$ a centred gamma distribution, and will recycle the letter $$H=G-\alpha /\lambda$$. As a linear transform of a random variable, the MGF of the centred gamma distribution is (ignoring questions of region of convergence)…

\begin{aligned} \bb{E}[\exp(H s)]&=\exp (\alpha s/\lambda)\bb{E}[\exp(G s)]\\ &=\exp (\alpha s/\lambda)\left(1-{\frac {s}{\lambda }}\right)^{-\alpha }\\ &=\exp (\alpha s/\lambda - \alpha \log(1-s/\lambda))\end{aligned}

## As a Lévy process

For arguments $$x, t>0$$ and parameters $$\alpha, \lambda>0,$$ we have the increment density as simply a gamma density:

$p_{X}(t, x)=\frac{\lambda^{\alpha t} x^{\alpha t-1} \mathrm{e}^{-x \lambda}}{ \Gamma(\alpha t)},$

This gives us a spectrally positive Lévy measure

$\pi_{X}(x)=\frac{\alpha}{x} \mathrm{e}^{-\lambda x}$

and Laplace exponent

$\Phi_{X}(z)=\alpha \ln (1+ z/\lambda), z \geq 0.$

That is, the Poisson rate, with respect to time $$t$$ of jumps whose size is in the range $$[x, x+dx)$$, is $$\pi(x)dx.$$ We think of this as an infinite superposition of Poisson processes driving different sized jumps, where the jumps are mostly tiny. This is how I think about Lévy process theory, at least.

## Gamma random field

I laboriously reinvented these bemused that no one seemed to use them, before discovering that they are called “completely random measures”.

## References

Ahrens, J. H., and U. Dieter. 1974. “Computer Methods for Sampling from Gamma, Beta, Poisson and Bionomial Distributions.” Computing 12 (3): 223–46. https://doi.org/10.1007/BF02293108.
———. 1982. “Generating Gamma Variates by a Modified Rejection Technique.” Communications of the ACM 25 (1): 47–54. https://doi.org/10.1145/358315.358390.
Applebaum, David. 2004. “Lévy Processes — from Probability to Finance and Quantum Groups.” Notices of the AMS 51 (11): 1336–47. https://core.ac.uk/download/pdf/50531.pdf.
———. 2009. Lévy Processes and Stochastic Calculus. 2nd ed. Cambridge Studies in Advanced Mathematics 116. Cambridge ; New York: Cambridge University Press.
Asmussen, Søren, and Peter W. Glynn. 2007. Stochastic Simulation: Algorithms and Analysis. 2007 edition. New York: Springer.
Avramidis, Athanassios N., Pierre L’Ecuyer, and Pierre-Alexandre Tremblay. 2003. “New Simulation Methodology for Finance: Efficient Simulation of Gamma and Variance-Gamma Processes.” In Proceedings of the 35th Conference on Winter Simulation: Driving Innovation, 319–26. WSC ’03. New Orleans, Louisiana: Winter Simulation Conference. http://www-perso.iro.umontreal.ca/~lecuyer/myftp/papers/wsc03vg.pdf.
Barndorff-Nielsen, Ole E., Makoto Maejima, and Ken-Iti Sato. 2006. “Some classes of multivariate infinitely divisible distributions admitting stochastic integral representations.” Bernoulli 12 (1): 1–33. https://projecteuclid.org/euclid.bj/1141136646.
Barndorff-Nielsen, Ole E., Jan Pedersen, and Ken-Iti Sato. 2001. “Multivariate Subordination, Self-Decomposability and Stability.” Advances in Applied Probability 33 (1): 160–87. https://doi.org/10.1017/S0001867800010685.
Bertoin, Jean. 1996. Lévy Processes. Cambridge Tracts in Mathematics 121. Cambridge ; New York: Cambridge University Press.
———. 1999. “Subordinators: Examples and Applications.” In Lectures on Probability Theory and Statistics: Ecole d’Eté de Probailités de Saint-Flour XXVII - 1997, edited by Jean Bertoin, Fabio Martinelli, Yuval Peres, and Pierre Bernard, 1717:1–91. Lecture Notes in Mathematics. Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-48115-7_1.
———. 2000. Subordinators, Lévy Processes with No Negative Jumps, and Branching Processes. University of Aarhus. Centre for Mathematical Physics and Stochastics …. http://www.maphysto.dk/oldpages/events/LevyBranch2000/notes/bertoin.pdf.
Bhattacharya, Rabi N., and Edward C. Waymire. 2009. Stochastic Processes with Applications. Society for Industrial and Applied Mathematics. http://epubs.siam.org/doi/abs/10.1137/1.9780898718997.fm.
Bondesson, Lennart. 2012. Generalized Gamma Convolutions and Related Classes of Distributions and Densities. Springer Science & Business Media.
Buchmann, Boris, Benjamin Kaehler, Ross Maller, and Alexander Szimayer. 2015. “Multivariate Subordination Using Generalised Gamma Convolutions with Applications to V.G. Processes and Option Pricing.” arXiv:1502.03901 [math, q-Fin], February. http://arxiv.org/abs/1502.03901.
Connor, Robert J., and James E. Mosimann. 1969. “Concepts of Independence for Proportions with a Generalization of the Dirichlet Distribution.” Journal of the American Statistical Association 64 (325): 194–206.
Çinlar, Erhan. 1980. “On a Generalization of Gamma Processes.” Journal of Applied Probability 17 (2): 467–80. https://doi.org/10.2307/3213036.
Émery, Michel, and Marc Yor. 2004. “A Parallel Between Brownian Bridges and Gamma Bridges.” Publications of the Research Institute for Mathematical Sciences 40 (3): 669–88. https://doi.org/10.2977/prims/1145475488.
Ferguson, Thomas S. 1974. “Prior Distributions on Spaces of Probability Measures.” The Annals of Statistics 2 (4): 615–29. https://doi.org/10.1214/aos/1176342752.
Ferguson, Thomas S., and Michael J. Klass. 1972. “A Representation of Independent Increment Processes without Gaussian Components.” The Annals of Mathematical Statistics 43 (5): 1634–43. https://doi.org/10.1214/aoms/1177692395.
Figueroa-López, José E. 2012. “Jump-Diffusion Models Driven by Lévy Processes.” In Handbook of Computational Finance, edited by Jin-Chuan Duan, Wolfgang Karl Härdle, and James E. Gentle, 61–88. Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-17254-0_4.
Foti, Nicholas, Joseph Futoma, Daniel Rockmore, and Sinead Williamson. 2013. “A Unifying Representation for a Class of Dependent Random Measures.” In Artificial Intelligence and Statistics, 20–28. http://proceedings.mlr.press/v31/foti13a.html.
Gusak, Dmytro, Alexander Kukush, Alexey Kulik, Yuliya Mishura, and Andrey Pilipenko. 2010. Theory of Stochastic Processes : With Applications to Financial Mathematics and Risk Theory. Problem Books in Mathematics. New York: Springer New York. http://link.springer.com/book/10.1007%2F978-0-387-87862-1.
Hackmann, Daniel, and Alexey Kuznetsov. 2016. “Approximating Lévy Processes with Completely Monotone Jumps.” The Annals of Applied Probability 26 (1): 328–59. https://doi.org/10.1214/14-AAP1093.
Ishwaran, Hemant, and Mahmoud Zarepour. 2002. “Exact and Approximate Sum Representations for the Dirichlet Process.” Canadian Journal of Statistics 30 (2): 269–83. https://doi.org/10.2307/3315951.
James, Lancelot F., Bernard Roynette, and Marc Yor. 2008. “Generalized Gamma Convolutions, Dirichlet means, Thorin measures, with explicit examples.” Probability Surveys 5: 346–415. https://doi.org/10.1214/07-PS118.
Kyprianou, Andreas E. 2014. Fluctuations of Lévy Processes with Applications: Introductory Lectures. Second edition. Universitext. Heidelberg: Springer.
Lalley, Steven P. 2007. “Lévy Processes, Stable Processes, and Subordinators.”
Lawrence, Neil D., and Raquel Urtasun. 2009. “Non-Linear Matrix Factorization with Gaussian Processes.” In Proceedings of the 26th Annual International Conference on Machine Learning, 601–8. ICML ’09. New York, NY, USA: ACM. https://doi.org/10.1145/1553374.1553452.
Lefebvre, Mario. 2007. Applied Stochastic Processes. Universitext. Springer New York. http://link.springer.com/chapter/10.1007/978-0-387-48976-6_2.
Liou, Jun-Jih, Yuan-Fong Su, Jie-Lun Chiang, and Ke-Sheng Cheng. 2011. “Gamma Random Field Simulation by a Covariance Matrix Transformation Method.” Stochastic Environmental Research and Risk Assessment 25 (2): 235–51. https://doi.org/10.1007/s00477-010-0434-8.
Lo, Albert Y., and Chung-Sing Weng. 1989. “On a Class of Bayesian Nonparametric Estimates: II. Hazard Rate Estimates.” Annals of the Institute of Statistical Mathematics 41 (2): 227–45. https://doi.org/10.1007/BF00049393.
Mathai, A. M. 1982. “Storage Capacity of a Dam with Gamma Type Inputs.” Annals of the Institute of Statistical Mathematics 34 (3): 591–97. https://doi.org/10.1007/BF02481056.
Mathai, A. M., and P. G. Moschopoulos. 1991. “On a Multivariate Gamma.” Journal of Multivariate Analysis 39 (1): 135–53. https://doi.org/10.1016/0047-259X(91)90010-Y.
Mathal, A. M., and P. G. Moschopoulos. 1992. “A Form of Multivariate Gamma Distribution.” Annals of the Institute of Statistical Mathematics 44 (1): 97–106. https://doi.org/10.1007/BF00048672.
Moschopoulos, P. G. 1985. “The Distribution of the Sum of Independent Gamma Random Variables.” Annals of the Institute of Statistical Mathematics 37 (3): 541–44. https://doi.org/10.1007/BF02481123.
Olofsson, Peter. 2005. Probability, Statistics, and Stochastic Processes. Hoboken, N.J: Hoboken, N.J. : Wiley-Interscience. http://dx.doi.org/10.1002%2F9780471743064.
Pérez-Abreu, Victor, and Robert Stelzer. 2014. “Infinitely Divisible Multivariate and Matrix Gamma Distributions.” Journal of Multivariate Analysis 130 (September): 155–75. https://doi.org/10.1016/j.jmva.2014.04.017.
Pfaffel, Oliver. 2012. “Wishart Processes.” arXiv:1201.3256 [math], January. http://arxiv.org/abs/1201.3256.
Rubinstein, Reuven Y., and Dirk P. Kroese. 2016. Simulation and the Monte Carlo Method. 3 edition. Wiley series in probability and statistics. Hoboken, New Jersey: Wiley.
Sato, Ken-iti. 1999. Lévy Processes and Infinitely Divisible Distributions. Cambridge University Press.
Semeraro, Patrizia. 2008. “A Multivariate Variance Gamma Model for Financial Applications.” International Journal of Theoretical and Applied Finance 11 (01): 1–18. https://doi.org/10.1142/S0219024908004701.
Shaked, Moshe, and J. George Shanthikumar. 1988. “On the First-Passage Times of Pure Jump Processes.” Journal of Applied Probability 25 (3): 501–9. https://doi.org/10.2307/3213979.
Singpurwalla, Nozer. 1997. “Gamma Processes and Their Generalizations: An Overview.” In Engineering Probabilistic Design and Maintenance for Flood Protection, edited by Roger Cooke, Max Mendel, and Han Vrijling, 67–75. Boston, MA: Springer US. https://doi.org/10.1007/978-1-4613-3397-5_5.
Singpurwalla, Nozer D., and Mark A. Youngren. 1993. “Multivariate Distributions Induced by Dynamic Environments.” Scandinavian Journal of Statistics 20 (3): 251–61. https://www.jstor.org/stable/4616280.
Steutel, Fred W., and Klaas van Harn. 2003. Infinite Divisibility of Probability Distributions on the Real Line. CRC Press.
Tankov, Peter, and Ekaterina Voltchkova. n.d. “Jump-Diﬀusion Models: A Practitioner’s Guide,” 24.
Veillette, Mark, and Murad S. Taqqu. 2010a. “Using Differential Equations to Obtain Joint Moments of First-Passage Times of Increasing Lévy Processes.” Statistics & Probability Letters 80 (7): 697–705. https://doi.org/10.1016/j.spl.2010.01.002.
———. 2010b. “Numerical Computation of First-Passage Times of Increasing Lévy Processes.” Methodology and Computing in Applied Probability 12 (4): 695–729. https://doi.org/10.1007/s11009-009-9158-y.
Weide, Hans van der. 1997. “Gamma Processes.” In Engineering Probabilistic Design and Maintenance for Flood Protection, edited by Roger Cooke, Max Mendel, and Han Vrijling, 77–83. Boston, MA: Springer US. https://doi.org/10.1007/978-1-4613-3397-5_6.
Wilson, Andrew Gordon, and Zoubin Ghahramani. 2011. “Generalised Wishart Processes.” In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, 736–44. UAI’11. Arlington, Virginia, United States: AUAI Press. http://dl.acm.org/citation.cfm?id=3020548.3020633.
Wolpert, R. 1998. “Poisson/Gamma Random Field Models for Spatial Statistics.” Biometrika 85 (2): 251–67. https://doi.org/10.1093/biomet/85.2.251.
Wolpert, Robert L. 2006. “Stationary Gamma Processes,” 13.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.