# Stationary Gamma processes


Processes with Gamma marginals. Usually when we discuss Gamma processes we mean Gamma-Lévy processes. Such processes have independent Gamma increments, much like a Wiener process has independent Gaussian increments and a Poisson process has independent Poisson increments. Gamma processes provide the classic subordinator models, i.e. non-decreasing Lévy processes.

There are other processes with Gamma marginals. Much like the Gaussian process family includes many processes with Gaussian marginals, so does the Gamma. It has a different set of natural algebraic relations to the Gaussian process. For example, the class of Gaussian processes is closed under addition and multiplication. The class of Gamma processes is closed under addition and thinning and some other weirder operations, all of which requires little more background knowledge to understand. It turns out there are complications with multivariate Gamma processes so those are handled separately.

Gamma processes are a natural model for spiky things

Gamma distributions and processes and such crop up all over the place. See also Pólya-Gamma distribution.

OK but if a process’s marginals are “Gamma-distributed”, what does that even mean? First, go and read about Gamma distributions. From that we can construct the Lévy Gamma process which is usually what we mean when we talk about Gamma processes. However, there are many more processes that we can construct with Gamma marginals; those others are here.

THEN go and read about Beta and Dirichlet distributions. and the Gamma-Beta notebook.

Now we are ready to look at stationary dependent Gamma processes.

There are Ornstein–Uhlenbeck-type constructions for Gamma processes . See R. L. Wolpert (2021) for a modern summary and overview of several popular alternatives.

For fixed $$\alpha, \lambda>0$$ these notes present six different stationary time series, each with Gamma $$\rv{g}(t) \sim \operatorname{Gamma}(\alpha, \lambda)$$ univariate marginal distributions and autocorrelation function $$\rho^{|s-t|}$$ for $$\rv{g}(s), \rv{g}(t)$$. Each will be defined on some time index set $$\mathcal{T}$$, either $$\mathcal{T}=\mathbb{\rv{z}}$$ or $$\mathcal{T}=\mathbb{R}$$.

Five of the six constructions can be applied to other Infinitely Divisible (ID) distributions as well, both continuous ones (normal, $$\alpha$$-stable, etc.) and discrete (Poisson, negative binomial, etc). For specifically the Poisson and Gaussian distributions, all but one of them (the Markov change-point construction) coincide — essentially, there is just one “AR(1)-like” Gaussian process (namely, the $$A R(1)$$ process in discrete time, or the Ornstein-Uhlenbeck process in continuous time), and there is just one $$A R(1)$$-like Poisson process. For other ID distributions, however, and in particular for the Gamma, each of these constructions yields a process with the same univariate marginal distributions and the same autocorrelation but with different joint distributions at three or more times.

## Thinned Autoregressive Gamma

To my mind the most natural one.

We let $\rv{g}(0) \sim \operatorname{Gamma}(\alpha, \lambda)$ and, for $$t \in \mathbb{N}$$ set $\rv{g}(t):=\xi(t)+\zeta(t)$ where \begin{aligned} &\xi(t):=\rv{b}(t) \cdot \rv{g}(t-1), \quad \rv{b}(t) \sim \operatorname{Beta}(\alpha \rho, \alpha \bar{\rho}) \\ &\zeta(t) \sim \operatorname{Gamma}(\alpha \bar{\rho}, \lambda) \end{aligned} where $$\bar{\rho}:=(1-\rho)$$ and all the $$\left\{\rv{b}(t)\right\}$$ and $$\left\{\zeta(t)\right\}$$ are independent. Then,$$\xi(t) \sim \operatorname{Gamma}(\alpha \rho, \lambda)$$ and $$\zeta(t) \sim \operatorname{Gamma}(\alpha \bar{\rho}, \lambda)$$ are independent, with sum $$\rv{g}(t) \sim$$ $$\operatorname{Gamma}(\alpha, \lambda)$$. Thus $$\left\{\rv{g}(t)\right\}$$ is a Markov process with Gamma univariate marginal distribution $$\rv{g}(t) \sim \operatorname{Gamma}(\alpha, \lambda)$$, now with joint characteristic function \begin{aligned} \chi(s, t) &=\mathbb{E}\exp\left(i s \rv{g}(0)+i t \rv{g}(1)\right) \\ &=\mathbb{E}\exp\left\{i s\left(\rv{g}(0)-\xi(1)\right)+i(s+t) \xi(1)+i t \zeta(1)\right\} \\ &=(1-i s / \lambda)^{-\alpha \bar{\rho}}(1-i(s+t) / \lambda)^{-\alpha \rho}(1-i t / \lambda)^{-\alpha \bar{\rho}} \end{aligned} Note that unlike the autoregressive construction, this characteristic function of this one is symmetric in the time arguments, and therefore the process is time-reversible. In some senses this is a “more natural” autoregressive process than the Zeta-innovation AR(1) process. For one, it is easy to imagine how to generalize this to vector autoregressive processes. For another, there is a natural generalization to continuous time using the Beta process in the sense of Hjort (1990) and Thibaux and Jordan (2007).

What does this look like in practice?

set.seed(105)
# generate a stationary thinned autoregressive Gamma series
gamp = function(T, alpha, lambda, rho) {
g = rgamma(1, alpha, rate=lambda)
b = rbeta(T, alpha*rho, alpha*(1-rho))
zeta = rgamma(T, alpha*(1-rho), rate=lambda)
gs = numeric(T)
for (i in 1:T) {
g = b[i] * g + zeta[i]
gs[i] = g
}
gs
}
T = 10000
ts = (1:T)/100
plot(ts, gamp(T, 1.0, 0.1, 0.999),
type = "l", col = 2,
ylim = c(0, 25), ylab="", xlab = "time")
lines(ts, gamp(T, 10, 1.0, 0.999),
type = "l", col = 3)
lines(ts, gamp(T, 100, 10.0, 0.999),
type = "l", col = 4)
legend("topright",
c("lambda=0.1", "lambda=1", "lambda=10"),
lty = 1, col = 2:4)

Fix $$0 \leq \rho<1$$. Let $$\rv{g}(0) \sim \operatorname{Gamma}(\alpha, \lambda)$$ and for $$t \in \mathbb{N}$$ define $$\rv{g}(t)$$ recursively by $\rv{g}(t):=\rho \rv{g}(t-1)+\zeta(t)$ for iid $$\left\{\zeta(t)\right\}$$ (see Zeta distribution). The process $$\left\{\rv{g}(t)\right\}$$ has Gamma univariate marginal distribution $$\rv{g}(t) \sim \operatorname{Gamma}(\alpha, \lambda)$$ for every $$t \in \mathbb{R}_{+}$$ and, at consecutive times $$s,t$$ joint characteristic function \begin{aligned} \chi(s, t) &=\operatorname{E} \exp \left(i s \rv{g}(0)+i t \rv{g}(1)\right) \\ &=\operatorname{E} \exp \left(i(s+\rho t) \rv{g}(0)+i t \zeta(1)\right) \\ &=\left[\frac{(1-i(s+\rho t) / \lambda)(1-i t / \lambda)}{1-i t \rho / \lambda}\right]^{-\alpha}. \end{aligned} Unlike Gaussian additive autoregressive processes, where the marginal and innovation processes are both Gaussian, in Gamma additive autoregressive processes the marginal is Gamma but the innovation is not . We can get a process that has a gamma innovation by the next construction instead.

Exercise: Generalise this to continuous time.

### Gamma-Zeta distribution

I don’t know a name for the distribution of the $$\zeta(t)$$ RVs from earlier. Let us go with Gamma-Zeta, because plain Zeta is taken.

It is easiest to describe that RV in terms of the characteristic function $$E e^{i \omega \zeta(t)}=(1-i \omega / \lambda)^{-\alpha}(1-i \rho \omega / \lambda)^{\alpha}=\left[\frac{\lambda-i \omega}{\lambda-i \rho \omega}\right]^{-\alpha}.$$

Simulating such RVs is easy via the algorithm of Walker (2000):

$\lambda(t) \sim \operatorname{Gamma}(\alpha, 1), \quad N(t)|\lambda(t) \sim \mathrm{Po}\left(\frac{1-\rho}{\rho} \lambda(t)\right), \quad \zeta(t)| N(t) \sim \operatorname{Gamma}\left(N(t), \frac{\lambda}{\rho}\right).$

However, this distribution does not seem to have an obvious density except as a Fourier transform. Let us set is aside for now, eh?

## Change-point gamma

Also from R. L. Wolpert (2021). What other marginals than Gamma can I construct with this?

Let $$\left\{\zeta_{n}: n \in \mathbb{Z}\right\} \stackrel{\text { iid }}{\sim} \mathrm{Ga}(\alpha, \beta)$$ be iid Gamma random variables and let $$N_{t}$$ be a standard Poisson process indexed by $$t \in \mathbb{R}$$ (so $$N_{0}=0$$ and $$\left(N_{t}-N_{s}\right) \sim \mathrm{Po}(t-s)$$ for all $$-\infty<s<$$ $$t<\infty$$, with independent increments), and set $X_{t}:=\zeta_{n}, \quad n=N_{\lambda t}$ Then each $$X_{t} \sim \mathrm{Ga}(\alpha, \beta)$$ and, for $$s, t \in \mathbb{R}, X_{s}$$ and $$X_{t}$$ are either identical (with probability $$\left.\rho^{|s-t|}\right)$$ or independent- reminiscent of a Metropolis MCMC chain. The chf is \begin{aligned} \chi(s, t) &=\mathrm{E} \exp \left(i s X_{0}+i t X_{1}\right) \\ &=\rho(1-i(s+t) / \beta)^{-\alpha}+\bar{\rho}(1-i s / \beta)^{-\alpha}(1-i t / \beta)^{-\alpha} \end{aligned} and once again the marginal distribution is $$X_{t} \sim \mathrm{Ga}(\alpha, \beta)$$ and the autocorrelation function is $$\operatorname{Corr}\left(X_{s}, X_{t}\right)=\rho^{|s-t|}$$.

## References

Ahrens, J. H., and U. Dieter. 1974. Computing 12 (3): 223–46.
———. 1982. Communications of the ACM 25 (1): 47–54.
Applebaum, David. 2004. Notices of the AMS 51 (11): 1336–47.
———. 2009. Lévy Processes and Stochastic Calculus. 2nd ed. Cambridge Studies in Advanced Mathematics 116. Cambridge ; New York: Cambridge University Press.
Asmussen, Søren, and Peter W. Glynn. 2007. Stochastic Simulation: Algorithms and Analysis. 2007 edition. New York: Springer.
Avramidis, Athanassios N., Pierre L’Ecuyer, and Pierre-Alexandre Tremblay. 2003. In Proceedings of the 35th Conference on Winter Simulation: Driving Innovation, 319–26. WSC ’03. New Orleans, Louisiana: Winter Simulation Conference.
Barndorff-Nielsen, Ole E., Makoto Maejima, and Ken-Iti Sato. 2006. Bernoulli 12 (1): 1–33.
Barndorff-Nielsen, Ole E., Jan Pedersen, and Ken-Iti Sato. 2001. Advances in Applied Probability 33 (1): 160–87.
Bertoin, Jean. 1996. Lévy Processes. Cambridge Tracts in Mathematics 121. Cambridge ; New York: Cambridge University Press.
———. 1999. In Lectures on Probability Theory and Statistics: Ecole d’Eté de Probailités de Saint-Flour XXVII - 1997, edited by Jean Bertoin, Fabio Martinelli, Yuval Peres, and Pierre Bernard, 1717:1–91. Lecture Notes in Mathematics. Berlin, Heidelberg: Springer Berlin Heidelberg.
———. 2000. Subordinators, Lévy Processes with No Negative Jumps, and Branching Processes. University of Aarhus. Centre for Mathematical Physics and Stochastics ….
Bhattacharya, Rabi N., and Edward C. Waymire. 2009. Stochastic Processes with Applications. Society for Industrial and Applied Mathematics.
Bondesson, Lennart. 2012. Generalized Gamma Convolutions and Related Classes of Distributions and Densities. Lecture Notes in Statistics 76. New York: Springer Science & Business Media.
Buchmann, Boris, Benjamin Kaehler, Ross Maller, and Alexander Szimayer. 2015. arXiv:1502.03901 [Math, q-Fin], February.
Chaumont, Loïc, and Marc Yor. 2012. Exercises in Probability: A Guided Tour from Measure Theory to Random Processes, Via Conditioning. Cambridge University Press.
Çinlar, Erhan. 1980. Journal of Applied Probability 17 (2): 467–80.
Connor, Robert J., and James E. Mosimann. 1969. “Concepts of Independence for Proportions with a Generalization of the Dirichlet Distribution.” Journal of the American Statistical Association 64 (325): 194–206.
Devroye, Luc. 1986. Non-uniform random variate generation. New York: Springer.
Dufresne, Daniel. 1998. Advances in Applied Mathematics 20 (3): 285–99.
Edwards, Matthew C., Renate Meyer, and Nelson Christensen. 2019. Statistics and Computing 29 (1): 67–78.
Émery, Michel, and Marc Yor. 2004. Publications of the Research Institute for Mathematical Sciences 40 (3): 669–88.
Ferguson, Thomas S. 1974. The Annals of Statistics 2 (4): 615–29.
Ferguson, Thomas S., and Michael J. Klass. 1972. The Annals of Mathematical Statistics 43 (5): 1634–43.
Figueroa-López, José E. 2012. In Handbook of Computational Finance, edited by Jin-Chuan Duan, Wolfgang Karl Härdle, and James E. Gentle, 61–88. Berlin, Heidelberg: Springer Berlin Heidelberg.
Fink, Daniel. 1997. 46.
Foti, Nicholas, Joseph Futoma, Daniel Rockmore, and Sinead Williamson. 2013. In Artificial Intelligence and Statistics, 20–28.
Gaver, D. P., and P. a. W. Lewis. 1980. Advances in Applied Probability 12 (3): 727–45.
Gourieroux, Christian, and Joann Jasiak. 2006. Journal of Forecasting 25 (2): 129–52.
Griffiths, Thomas L., and Zoubin Ghahramani. 2011. Journal of Machine Learning Research 12 (32): 1185–1224.
Grigelionis, Bronius. 2013. Student’s t-Distribution and Related Stochastic Processes. SpringerBriefs in Statistics. Berlin, Heidelberg: Springer Berlin Heidelberg.
Gupta, Arjun K., and Saralees Nadarajah, eds. 2014. Handbook of Beta Distribution and Its Applications. Boca Raton: CRC Press.
Gusak, Dmytro, Alexander Kukush, Alexey Kulik, Yuliya Mishura, and Andrey Pilipenko. 2010. Theory of Stochastic Processes : With Applications to Financial Mathematics and Risk Theory. Problem Books in Mathematics. New York: Springer New York.
Hackmann, Daniel, and Alexey Kuznetsov. 2016. The Annals of Applied Probability 26 (1): 328–59.
Hjort, Nils Lid. 1990. The Annals of Statistics 18 (3): 1259–94.
Ishwaran, Hemant, and Mahmoud Zarepour. 2002. Canadian Journal of Statistics 30 (2): 269–83.
James, Lancelot F., Bernard Roynette, and Marc Yor. 2008. Probability Surveys 5: 346–415.
Kingman, J. F. C. 1992. Poisson Processes. Clarendon Press.
Kirch, Claudia, Matthew C. Edwards, Alexander Meier, and Renate Meyer. 2019. Bayesian Analysis 14 (4): 1037–73.
Kyprianou, Andreas E. 2014. Fluctuations of Lévy Processes with Applications: Introductory Lectures. Second edition. Universitext. Heidelberg: Springer.
Lalley, Steven P. 2007. “Lévy Processes, Stable Processes, and Subordinators.”
Lawrance, A. J. 1982. Scandinavian Journal of Statistics 9 (4): 234–36.
Lawrence, Neil D., and Raquel Urtasun. 2009. In Proceedings of the 26th Annual International Conference on Machine Learning, 601–8. ICML ’09. New York, NY, USA: ACM.
Lefebvre, Mario. 2007. Applied Stochastic Processes. Universitext. Springer New York.
Lin, Jiayu. 2016. “On The Dirichlet Distribution,” 75.
Liou, Jun-Jih, Yuan-Fong Su, Jie-Lun Chiang, and Ke-Sheng Cheng. 2011. Stochastic Environmental Research and Risk Assessment 25 (2): 235–51.
Lo, Albert Y., and Chung-Sing Weng. 1989. Annals of the Institute of Statistical Mathematics 41 (2): 227–45.
Mathai, A. M. 1982. Annals of the Institute of Statistical Mathematics 34 (3): 591–97.
Mathai, A. M., and P. G. Moschopoulos. 1991. Journal of Multivariate Analysis 39 (1): 135–53.
Mathai, A. M., and Serge B. Provost. 2005. Linear Algebra and Its Applications, Tenth Special Issue (Part 2) on Linear Algebra and Statistics, 410 (November): 198–216.
Mathal, A. M., and P. G. Moschopoulos. 1992. Annals of the Institute of Statistical Mathematics 44 (1): 97–106.
Meier, Alexander. 2018.
Meier, Alexander, Claudia Kirch, Matthew C. Edwards, and Renate Meyer. 2019. beyondWhittle: Bayesian Spectral Inference for Stationary Time Series (version 1.1.1).
Meier, Alexander, Claudia Kirch, and Renate Meyer. 2020. Journal of Multivariate Analysis 175 (January): 104560.
Moschopoulos, P. G. 1985. Annals of the Institute of Statistical Mathematics 37 (3): 541–44.
Olofsson, Peter. 2005. Probability, Statistics, and Stochastic Processes. Hoboken, N.J: Hoboken, N.J. : Wiley-Interscience.
Pérez-Abreu, Victor, and Robert Stelzer. 2014. Journal of Multivariate Analysis 130 (September): 155–75.
Pfaffel, Oliver. 2012. arXiv:1201.3256 [Math], January.
Polson, Nicholas G., James G. Scott, and Jesse Windle. 2013. Journal of the American Statistical Association 108 (504): 1339–49.
Rao, Vinayak, and Yee Whye Teh. 2009. “Spatial Normalized Gamma Processes.” In Proceedings of the 22nd International Conference on Neural Information Processing Systems, 1554–62. NIPS’09. Red Hook, NY, USA: Curran Associates Inc.
Roychowdhury, Anirban, and Brian Kulis. 2015. In Artificial Intelligence and Statistics, 800–808. PMLR.
Rubinstein, Reuven Y., and Dirk P. Kroese. 2016. Simulation and the Monte Carlo Method. 3 edition. Wiley series in probability and statistics. Hoboken, New Jersey: Wiley.
Sato, Ken-iti. 1999. Lévy Processes and Infinitely Divisible Distributions. Cambridge University Press.
Semeraro, Patrizia. 2008. International Journal of Theoretical and Applied Finance 11 (01): 1–18.
Shah, Amar, Andrew Wilson, and Zoubin Ghahramani. 2014. In Artificial Intelligence and Statistics, 877–85. PMLR.
Shaked, Moshe, and J. George Shanthikumar. 1988. Journal of Applied Probability 25 (3): 501–9.
Sim, C. H. 1990. Journal of Applied Probability 27 (2): 325–32.
Singpurwalla, Nozer. 1997. In Engineering Probabilistic Design and Maintenance for Flood Protection, edited by Roger Cooke, Max Mendel, and Han Vrijling, 67–75. Boston, MA: Springer US.
Singpurwalla, Nozer D., and Mark A. Youngren. 1993. Scandinavian Journal of Statistics 20 (3): 251–61.
Steutel, Fred W., and Klaas van Harn. 2003. Infinite Divisibility of Probability Distributions on the Real Line. Boca Raton: CRC Press.
Tankov, Peter, and Ekaterina Voltchkova. n.d. “Jump-Diﬀusion Models: A Practitioner’s Guide,” 24.
Thibaux, Romain, and Michael I. Jordan. 2007. In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, 564–71. PMLR.
Thorin, Olof. 1977a. Scandinavian Actuarial Journal 1977 (1): 31–40.
———. 1977b. Scandinavian Actuarial Journal 1977 (3): 121–48.
Tracey, Brendan D., and David H. Wolpert. 2018. 2018 AIAA Non-Deterministic Approaches Conference, January.
Veillette, Mark, and Murad S. Taqqu. 2010a. Statistics & Probability Letters 80 (7): 697–705.
———. 2010b. Methodology and Computing in Applied Probability 12 (4): 695–729.
Walker, S. G. 2000. Scandinavian Journal of Statistics 27 (3): 575–76.
Weide, Hans van der. 1997. In Engineering Probabilistic Design and Maintenance for Flood Protection, edited by Roger Cooke, Max Mendel, and Han Vrijling, 77–83. Boston, MA: Springer US.
Wilson, Andrew Gordon, and Zoubin Ghahramani. 2011. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, 736–44. UAI’11. Arlington, Virginia, United States: AUAI Press.
Wolpert, R., and Katja Ickstadt. 1998. Biometrika 85 (2): 251–67.
Wolpert, Robert L. 2021. arXiv:2106.00087 [Math], May.
Wolpert, Robert L., and Lawrence D. Brown. 2021. arXiv:2105.14591 [Math], May.
Xuan, Junyu, Jie Lu, Guangquan Zhang, Richard Yi Da Xu, and Xiangfeng Luo. 2015. arXiv:1503.08542 [Cs, Stat], March.
Yor, Marc. 2007. In Advances in Mathematical Finance, edited by Michael C. Fu, Robert A. Jarrow, Ju-Yi J. Yen, and Robert J. Elliott, 37–47. Applied and Numerical Harmonic Analysis. Birkhäuser Boston.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.