Lévy Gamma processes



\[\renewcommand{\var}{\operatorname{Var}} \renewcommand{\corr}{\operatorname{Corr}} \renewcommand{\dd}{\mathrm{d}} \renewcommand{\bb}[1]{\mathbb{#1}} \renewcommand{\vv}[1]{\boldsymbol{#1}} \renewcommand{\rv}[1]{\mathsf{#1}} \renewcommand{\vrv}[1]{\vv{\rv{#1}}} \renewcommand{\disteq}{\stackrel{d}{=}} \renewcommand{\gvn}{\mid} \renewcommand{\Ex}{\mathbb{E}} \renewcommand{\Pr}{\mathbb{P}}\]

Processes with Gamma marginals. Usually when we discuss Gamma processes we mean Gamma-Lévy processes. Such processes have independent Gamma increments, much like a Wiener process has independent Gaussian increments and a Poisson process has independent Poisson increments. Gamma processes provide the classic subordinator models, i.e. non-decreasing Lévy processes.

There are other processes with Gamma marginals.

Gamma processes are a natural model for spiky things

OK but if a process’s marginals are “Gamma-distributed”, what does that even mean? First, go an read about Gamma distributions. THEN go and read about Beta and Dirichlet distributions. We need both. And especially the Gamma-Dirichlet algebra.

For more, see the Gamma-Beta notebook.

The Lévy-Gamma process

Every divisible distribution induces an associated Lévy process by a standard procedure This works on the Gamma process too.

Ground zero for treating these processes specifically appears to be Ferguson and Klass (1972), and then the weaponisation of these processes to construct the Dirichlet process prior occurs in Ferguson (1974). Tutorial introductions to Gamma(-Lévy) processes can be found in (Applebaum 2009; Asmussen and Glynn 2007; Rubinstein and Kroese 2016; Kyprianou 2014). Existence proofs etc are deferred to those sources. You could also see Wikipedia, although that article was not particularly helpful for me.

The univariate Lévy-Gamma process \(\{\rv{g}(t;\alpha,\lambda)\}_t\) is an independent-increment process, with time index \(t\) and parameters by \(\alpha, \lambda.\) We assume it is started at \(\rv{g}(0)=0\).

The marginal density \(g(x;t,\alpha, \lambda )\) of the process at time \(t\) is a Gamma RV, specifically, \[ g(x;t, \alpha, \lambda) =\frac{ \lambda^{\alpha t} } { \Gamma (\alpha t) } x^{\alpha t\,-\,1}e^{-\lambda x}, x\geq 0. \] We can think of the Gamma distribution as the distribution at time 1 of a Gamma process.

That is, \(\rv{g}(t) \sim \operatorname{Gamma}(\alpha(t_{i+1}-t_{i}), \lambda)\). which corresponds to increments per unit time in terms of \(\bb E(\rv{g}(1))=\alpha/\lambda\) and \(\var(\rv{g}(1))=\alpha/\lambda^2.\)

Aside: Note that the useful special case that \(\alpha t=1,\) then we have that \(\rv{g}(t;1,\lambda )\sim \operatorname{Exp}(\lambda).\)

The increment distirbution leads to a method for simulating a path of a Gamma process at a sequence of increasing times, \(\{t_1, t_2, t_3, \dots, t_L\}.\) Given \(\rv{g}(t_1;\alpha, \lambda),\) we know that the increments are distributed as independent variates \(\rv{g}_i:=\rv{g}(t_{i+1})-\rv{g}(t_{i})\sim \operatorname{Gamma}(\alpha(t_{i+1}-t_{i}), \lambda)\). Presuming we may simulate from the Gamma distribution, it follows that \[\rv{g}(t_i)=\sum_{j < i}\left( \rv{g}(t_{i+1})-\rv{g}(t_{i})\right)=\sum_{j < i} \rv{g}_j.\]

Lévy characterisation

For arguments \(x, t>0\) and parameters \(\alpha, \lambda>0,\) we have the increment density as simply a Gamma density:

\[ p_{X}(t, x)=\frac{\lambda^{\alpha t} x^{\alpha t-1} \mathrm{e}^{-x \lambda}}{ \Gamma(\alpha t)}, \]

This gives us a spectrally positive Lévy measure \[ \pi_{\rv{x}}(x)=\frac{\alpha}{x} \mathrm{e}^{-\lambda x} \] and Laplace exponent \[ \Phi_{\rv{x}}(z)=\alpha \ln (1+ z/\lambda), z \geq 0. \]

That is, the Poisson rate, with respect to time \(t\) of jumps whose size is in the range \([x, x+dx)\), is \(\pi(x)dx.\) We think of this as an infinite superposition of Poisson processes driving different sized jumps, where the jumps are mostly tiny. This is how I think about Lévy process theory, at least.

Gamma bridge

A useful associated process. Consider a univariate Gamma-Lévy process, \(\rv{g}(t)\) with \(\rv{g}(0)=0.\) The Gamma bridge, analogous to the Brownian bridge, is that process conditionalised upon attaining a fixed the value \(S=\rv{g}(1)\) at terminal time \(1.\) We write \(\rv{g}_{S}:=\{\rv{g}(t)\mid \rv{g}(1)=S\}_{0< t < 1}\) for the paths of this process.

We can simulate from the Gamma bridge easily. Given the increments of the process are independent, if we have a Gamma process \(\rv{g}\) on the index set \([0,1]\) such that \(\rv{g}(1)=S\), then we can simulate from the bridge paths which connect these points at intermediate time \(t,\, 0<t<1\) by recalling that we have known distributions for the increments; in particular \(\rv{g}(t)\sim\operatorname{Gamma}(\alpha, \lambda)\) and \(\rv{g}(1)-\rv{g}(t)\sim\operatorname{Gamma}(\alpha (1-t), \lambda)\) and these increments, as increments over disjoints sets, are themselves independent. Then, by the Beta thinning, \[\frac{\rv{g}(t)}{\rv{g}(1)}\sim\operatorname{Beta}(\alpha t, \alpha(1-t))\] independent of \(\rv{g}(1).\) We can therefore sample from a path of the bridge \(\rv{g}_{S}(t)\) for some \(t< 1\) by simulating \(\rv{g}_{S}(t)=B S,\) where \(B\sim \operatorname{Beta}(\alpha (t),\alpha (1-t)).\)

For more on that theme, see Barndorff-Nielsen, Pedersen, and Sato (2001), Émery and Yor (2004) or Yor (2007).

Completely random measures

Random probability distributions induced by using Gamma-Lévy processes as a CDF. I laboriously reinvented these, bemused that no one seemed to use them, before discovering that they are called “completely random measures” and they are in fact pretty common. So that was a fun exercise.

Time-warped Lévy-Gamma process

Çinlar (1980) walks us through the mechanics of (deterministically) time-warping Gamma processes, which ends up being not too unpleasant. Predictable stochastic time-warps look like they should be OK. See N. Singpurwalla (1997) for an application. Why bother? Linear superpositions of Gamma processes can be hard work, and sometime the generalisation from time-warping can come out nicer supposedly. 🏗

References

Ahrens, J. H., and U. Dieter. 1974. Computer Methods for Sampling from Gamma, Beta, Poisson and Bionomial Distributions.” Computing 12 (3): 223–46.
———. 1982. Generating Gamma Variates by a Modified Rejection Technique.” Communications of the ACM 25 (1): 47–54.
Applebaum, David. 2004. Lévy Processes — from Probability to Finance and Quantum Groups.” Notices of the AMS 51 (11): 1336–47.
———. 2009. Lévy Processes and Stochastic Calculus. 2nd ed. Cambridge Studies in Advanced Mathematics 116. Cambridge ; New York: Cambridge University Press.
Asmussen, Søren, and Peter W. Glynn. 2007. Stochastic Simulation: Algorithms and Analysis. 2007 edition. New York: Springer.
Avramidis, Athanassios N., Pierre L’Ecuyer, and Pierre-Alexandre Tremblay. 2003. New Simulation Methodology for Finance: Efficient Simulation of Gamma and Variance-Gamma Processes.” In Proceedings of the 35th Conference on Winter Simulation: Driving Innovation, 319–26. WSC ’03. New Orleans, Louisiana: Winter Simulation Conference.
Barndorff-Nielsen, Ole E., Makoto Maejima, and Ken-Iti Sato. 2006. Some Classes of Multivariate Infinitely Divisible Distributions Admitting Stochastic Integral Representations.” Bernoulli 12 (1): 1–33.
Barndorff-Nielsen, Ole E., Jan Pedersen, and Ken-Iti Sato. 2001. Multivariate Subordination, Self-Decomposability and Stability.” Advances in Applied Probability 33 (1): 160–87.
Bertoin, Jean. 1996. Lévy Processes. Cambridge Tracts in Mathematics 121. Cambridge ; New York: Cambridge University Press.
———. 1999. Subordinators: Examples and Applications.” In Lectures on Probability Theory and Statistics: Ecole d’Eté de Probailités de Saint-Flour XXVII - 1997, edited by Jean Bertoin, Fabio Martinelli, Yuval Peres, and Pierre Bernard, 1717:1–91. Lecture Notes in Mathematics. Berlin, Heidelberg: Springer Berlin Heidelberg.
———. 2000. Subordinators, Lévy Processes with No Negative Jumps, and Branching Processes. University of Aarhus. Centre for Mathematical Physics and Stochastics ….
Bhattacharya, Rabi N., and Edward C. Waymire. 2009. Stochastic Processes with Applications. Society for Industrial and Applied Mathematics.
Bondesson, Lennart. 2012. Generalized Gamma Convolutions and Related Classes of Distributions and Densities. Lecture Notes in Statistics 76. New York: Springer Science & Business Media.
Buchmann, Boris, Benjamin Kaehler, Ross Maller, and Alexander Szimayer. 2015. Multivariate Subordination Using Generalised Gamma Convolutions with Applications to V.G. Processes and Option Pricing.” arXiv:1502.03901 [Math, q-Fin], February.
Chaumont, Loïc, and Marc Yor. 2012. Exercises in Probability: A Guided Tour from Measure Theory to Random Processes, Via Conditioning. Cambridge University Press.
Çinlar, Erhan. 1980. On a Generalization of Gamma Processes.” Journal of Applied Probability 17 (2): 467–80.
Connor, Robert J., and James E. Mosimann. 1969. “Concepts of Independence for Proportions with a Generalization of the Dirichlet Distribution.” Journal of the American Statistical Association 64 (325): 194–206.
Devroye, Luc. 1986. Non-uniform random variate generation. New York: Springer.
Dufresne, Daniel. 1998. Algebraic Properties of Beta and Gamma Distributions, and Applications.” Advances in Applied Mathematics 20 (3): 285–99.
Edwards, Matthew C., Renate Meyer, and Nelson Christensen. 2019. Bayesian Nonparametric Spectral Density Estimation Using B-Spline Priors.” Statistics and Computing 29 (1): 67–78.
Émery, Michel, and Marc Yor. 2004. A Parallel Between Brownian Bridges and Gamma Bridges.” Publications of the Research Institute for Mathematical Sciences 40 (3): 669–88.
Ferguson, Thomas S. 1974. Prior Distributions on Spaces of Probability Measures.” The Annals of Statistics 2 (4): 615–29.
Ferguson, Thomas S., and Michael J. Klass. 1972. A Representation of Independent Increment Processes Without Gaussian Components.” The Annals of Mathematical Statistics 43 (5): 1634–43.
Figueroa-López, José E. 2012. Jump-Diffusion Models Driven by Lévy Processes.” In Handbook of Computational Finance, edited by Jin-Chuan Duan, Wolfgang Karl Härdle, and James E. Gentle, 61–88. Berlin, Heidelberg: Springer Berlin Heidelberg.
Fink, Daniel. 1997. A Compendium of Conjugate Priors,” 46.
Foti, Nicholas, Joseph Futoma, Daniel Rockmore, and Sinead Williamson. 2013. A Unifying Representation for a Class of Dependent Random Measures.” In Artificial Intelligence and Statistics, 20–28.
Gaver, D. P., and P. a. W. Lewis. 1980. First-Order Autoregressive Gamma Sequences and Point Processes.” Advances in Applied Probability 12 (3): 727–45.
Gourieroux, Christian, and Joann Jasiak. 2006. Autoregressive Gamma Processes.” Journal of Forecasting 25 (2): 129–52.
Griffiths, Thomas L., and Zoubin Ghahramani. 2011. The Indian Buffet Process: An Introduction and Review.” Journal of Machine Learning Research 12 (32): 1185–1224.
Grigelionis, Bronius. 2013. Student’s t-Distribution and Related Stochastic Processes. SpringerBriefs in Statistics. Berlin, Heidelberg: Springer Berlin Heidelberg.
Gupta, Arjun K., and Saralees Nadarajah, eds. 2014. Handbook of Beta Distribution and Its Applications. Boca Raton: CRC Press.
Gusak, Dmytro, Alexander Kukush, Alexey Kulik, Yuliya Mishura, and Andrey Pilipenko. 2010. Theory of Stochastic Processes : With Applications to Financial Mathematics and Risk Theory. Problem Books in Mathematics. New York: Springer New York.
Hackmann, Daniel, and Alexey Kuznetsov. 2016. Approximating Lévy Processes with Completely Monotone Jumps.” The Annals of Applied Probability 26 (1): 328–59.
Hjort, Nils Lid. 1990. Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data.” The Annals of Statistics 18 (3): 1259–94.
Ishwaran, Hemant, and Mahmoud Zarepour. 2002. Exact and Approximate Sum Representations for the Dirichlet Process.” Canadian Journal of Statistics 30 (2): 269–83.
James, Lancelot F., Bernard Roynette, and Marc Yor. 2008. Generalized Gamma Convolutions, Dirichlet Means, Thorin Measures, with Explicit Examples.” Probability Surveys 5: 346–415.
Kingman, J. F. C. 1992. Poisson Processes. Clarendon Press.
Kirch, Claudia, Matthew C. Edwards, Alexander Meier, and Renate Meyer. 2019. Beyond Whittle: Nonparametric Correction of a Parametric Likelihood with a Focus on Bayesian Time Series Analysis.” Bayesian Analysis 14 (4): 1037–73.
Kyprianou, Andreas E. 2014. Fluctuations of Lévy Processes with Applications: Introductory Lectures. Second edition. Universitext. Heidelberg: Springer.
Lalley, Steven P. 2007. “Lévy Processes, Stable Processes, and Subordinators.”
Lawrance, A. J. 1982. The Innovation Distribution of a Gamma Distributed Autoregressive Process.” Scandinavian Journal of Statistics 9 (4): 234–36.
Lawrence, Neil D., and Raquel Urtasun. 2009. Non-Linear Matrix Factorization with Gaussian Processes.” In Proceedings of the 26th Annual International Conference on Machine Learning, 601–8. ICML ’09. New York, NY, USA: ACM.
Lefebvre, Mario. 2007. Applied Stochastic Processes. Universitext. Springer New York.
Lin, Jiayu. 2016. “On The Dirichlet Distribution,” 75.
Liou, Jun-Jih, Yuan-Fong Su, Jie-Lun Chiang, and Ke-Sheng Cheng. 2011. Gamma Random Field Simulation by a Covariance Matrix Transformation Method.” Stochastic Environmental Research and Risk Assessment 25 (2): 235–51.
Lo, Albert Y., and Chung-Sing Weng. 1989. On a Class of Bayesian Nonparametric Estimates: II. Hazard Rate Estimates.” Annals of the Institute of Statistical Mathematics 41 (2): 227–45.
Mathai, A. M. 1982. Storage Capacity of a Dam with Gamma Type Inputs.” Annals of the Institute of Statistical Mathematics 34 (3): 591–97.
Mathai, A. M., and P. G. Moschopoulos. 1991. On a Multivariate Gamma.” Journal of Multivariate Analysis 39 (1): 135–53.
Mathai, A. M., and Serge B. Provost. 2005. Some Complex Matrix-Variate Statistical Distributions on Rectangular Matrices.” Linear Algebra and Its Applications, Tenth Special Issue (Part 2) on Linear Algebra and Statistics, 410 (November): 198–216.
Mathal, A. M., and P. G. Moschopoulos. 1992. A Form of Multivariate Gamma Distribution.” Annals of the Institute of Statistical Mathematics 44 (1): 97–106.
Meier, Alexander. 2018. A matrix Gamma process and applications to Bayesian analysis of multivariate time series.”
Meier, Alexander, Claudia Kirch, Matthew C. Edwards, and Renate Meyer. 2019. beyondWhittle: Bayesian Spectral Inference for Stationary Time Series (version 1.1.1).
Meier, Alexander, Claudia Kirch, and Renate Meyer. 2020. Bayesian Nonparametric Analysis of Multivariate Time Series: A Matrix Gamma Process Approach.” Journal of Multivariate Analysis 175 (January): 104560.
Moschopoulos, P. G. 1985. The Distribution of the Sum of Independent Gamma Random Variables.” Annals of the Institute of Statistical Mathematics 37 (3): 541–44.
Olofsson, Peter. 2005. Probability, Statistics, and Stochastic Processes. Hoboken, N.J: Hoboken, N.J. : Wiley-Interscience.
Pérez-Abreu, Victor, and Robert Stelzer. 2014. Infinitely Divisible Multivariate and Matrix Gamma Distributions.” Journal of Multivariate Analysis 130 (September): 155–75.
Pfaffel, Oliver. 2012. Wishart Processes.” arXiv:1201.3256 [Math], January.
Polson, Nicholas G., James G. Scott, and Jesse Windle. 2013. Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables.” Journal of the American Statistical Association 108 (504): 1339–49.
Rao, Vinayak, and Yee Whye Teh. 2009. “Spatial Normalized Gamma Processes.” In Proceedings of the 22nd International Conference on Neural Information Processing Systems, 1554–62. NIPS’09. Red Hook, NY, USA: Curran Associates Inc.
Roychowdhury, Anirban, and Brian Kulis. 2015. Gamma Processes, Stick-Breaking, and Variational Inference.” In Artificial Intelligence and Statistics, 800–808. PMLR.
Rubinstein, Reuven Y., and Dirk P. Kroese. 2016. Simulation and the Monte Carlo Method. 3 edition. Wiley series in probability and statistics. Hoboken, New Jersey: Wiley.
Sato, Ken-iti. 1999. Lévy Processes and Infinitely Divisible Distributions. Cambridge University Press.
Semeraro, Patrizia. 2008. A Multivariate Variance Gamma Model for Financial Applications.” International Journal of Theoretical and Applied Finance 11 (01): 1–18.
Shah, Amar, Andrew Wilson, and Zoubin Ghahramani. 2014. Student-t Processes as Alternatives to Gaussian Processes.” In Artificial Intelligence and Statistics, 877–85. PMLR.
Shaked, Moshe, and J. George Shanthikumar. 1988. On the First-Passage Times of Pure Jump Processes.” Journal of Applied Probability 25 (3): 501–9.
Sim, C. H. 1990. First-Order Autoregressive Models for Gamma and Exponential Processes.” Journal of Applied Probability 27 (2): 325–32.
Singpurwalla, Nozer. 1997. Gamma Processes and Their Generalizations: An Overview.” In Engineering Probabilistic Design and Maintenance for Flood Protection, edited by Roger Cooke, Max Mendel, and Han Vrijling, 67–75. Boston, MA: Springer US.
Singpurwalla, Nozer D., and Mark A. Youngren. 1993. Multivariate Distributions Induced by Dynamic Environments.” Scandinavian Journal of Statistics 20 (3): 251–61.
Steutel, Fred W., and Klaas van Harn. 2003. Infinite Divisibility of Probability Distributions on the Real Line. Boca Raton: CRC Press.
Tankov, Peter, and Ekaterina Voltchkova. n.d. “Jump-Diffusion Models: A Practitioner’s Guide,” 24.
Thibaux, Romain, and Michael I. Jordan. 2007. Hierarchical Beta Processes and the Indian Buffet Process.” In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, 564–71. PMLR.
Thorin, Olof. 1977a. On the Infinite Divisbility of the Pareto Distribution.” Scandinavian Actuarial Journal 1977 (1): 31–40.
———. 1977b. On the Infinite Divisibility of the Lognormal Distribution.” Scandinavian Actuarial Journal 1977 (3): 121–48.
Tracey, Brendan D., and David H. Wolpert. 2018. Upgrading from Gaussian Processes to Student’s-T Processes.” 2018 AIAA Non-Deterministic Approaches Conference, January.
Veillette, Mark, and Murad S. Taqqu. 2010a. Using Differential Equations to Obtain Joint Moments of First-Passage Times of Increasing Lévy Processes.” Statistics & Probability Letters 80 (7): 697–705.
———. 2010b. Numerical Computation of First-Passage Times of Increasing Lévy Processes.” Methodology and Computing in Applied Probability 12 (4): 695–729.
Walker, S. G. 2000. A Note on the Innovation Distribution of a Gamma Distributed Autoregressive Process.” Scandinavian Journal of Statistics 27 (3): 575–76.
Weide, Hans van der. 1997. Gamma Processes.” In Engineering Probabilistic Design and Maintenance for Flood Protection, edited by Roger Cooke, Max Mendel, and Han Vrijling, 77–83. Boston, MA: Springer US.
Wilson, Andrew Gordon, and Zoubin Ghahramani. 2011. Generalised Wishart Processes.” In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, 736–44. UAI’11. Arlington, Virginia, United States: AUAI Press.
Wolpert, R., and Katja Ickstadt. 1998. Poisson/Gamma Random Field Models for Spatial Statistics.” Biometrika 85 (2): 251–67.
Wolpert, Robert L. 2021. Lecture Notes on Stationary Gamma Processes.” arXiv:2106.00087 [Math], May.
Wolpert, Robert L., and Lawrence D. Brown. 2021. Markov Infinitely-Divisible Stationary Time-Reversible Integer-Valued Processes.” arXiv:2105.14591 [Math], May.
Xuan, Junyu, Jie Lu, Guangquan Zhang, Richard Yi Da Xu, and Xiangfeng Luo. 2015. Nonparametric Relational Topic Models Through Dependent Gamma Processes.” arXiv:1503.08542 [Cs, Stat], March.
Yor, Marc. 2007. Some Remarkable Properties of Gamma Processes.” In Advances in Mathematical Finance, edited by Michael C. Fu, Robert A. Jarrow, Ju-Yi J. Yen, and Robert J. Elliott, 37–47. Applied and Numerical Harmonic Analysis. Birkhäuser Boston.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.