# Models for count data

I have data and/or predictions made up of non-negative integers $${\mathbb N }\cup\{0\}$$. What probability distributions can I use to model it?

All the distributions I discuss here have support unbounded above. Bounded distributions (e.g. vanilla Binomial) are for some other time. The exception is the Bernoulli RV, a.k.a. the biased coin, which is simple enough to sneak in.

For some related ideas in a time series context, see

Should I also raid the document topic model literature for this. Surely they are implicitly count data? See string bags compare with Steyvers and Tenenbaum’s semantic network model .

A lot of this material is in , if you’d like a convenient 1-stop reference.

## Poisson

The Poisson is reminiscent of the Gaussian for count data, in terms of the number of places that it pops up, and the vast number of things that have a limiting Poisson distribution.

Conveniently, it has only one parameter, which means you don’t need to justify how you chose any other parameters, saving valuable time and thought. It’s useful as a “null model”, in that the number of particles in realisation of a point process without interaction will be Poisson-distributed, conditional upon the mean measure. Conversely, non-Poisson residuals are evidence that your model has failed to take out some kind of interaction or hidden variable.

Spelled
$$\operatorname{Poisson}(\lambda)$$
Pmf
$$\mathbb{P}(k;\lambda)={\frac {\lambda ^{k}}{k!}}e^{{-\lambda }}$$
Mean
$$\lambda$$
Variance
$$\lambda$$
Pgf
$$G(s;\lambda)=\exp(\lambda (s-1))$$

## Negative Binomial

Nearly a century old . A generic count data model which, unlike the Poisson, has both location-like and scale-like parameters, instead of only one parameter. This makes one feel better about using it than a Poisson, since 2 parameters seems like a good number. Has a traditional rationale in terms of drawing balls from urns and such, which is of little interest here. The key virtue here is that it is both flexible and uncontroversial.

It includes Geometric as special cases when $$r=1$$, and the Gamma and Poisson as limiting cases. More precisely, the Poisson is a limiting case of the Pólya distribution. for example, in the large-k limit, it approximates the Gamma distribution, and, when the mean is held constant, in the large $$r$$ limit, it approaches Poisson. For fixed r it is an exponential family

For all that, it’s still contrived, this model, and tedious to fit.

Spelled
$$\operatorname{NB}(p,r)$$
Pmf
$$\mathbb{P}(k;p,r) = {k+r-1 \choose k}\cdot (1-p)^{r}p^{k} = \frac{\Gamma(k+r)}{k!\,\Gamma(r)} (1-p)^rp^k$$
Mean
$${\frac {pr}{1-p}}$$
Variance
$${\frac {pr}{(1-p)^{2}}}$$
Pgf
$$G(s;p,r)=\left({\frac {1-p}{1-ps}}\right)^{{\!r}}{\text{ for }}|s|<{\frac 1p}$$

### Mean/dispersion parameterisation (Pólya)

Commonly, where the $$r$$ parameter is not required to be a non-negative integer, we call it a Pólya model, and use it for over-dispersed data i.e. data that looks like a Poisson process if we are drunk, but whose variance is too big in comparison to its mean for soberer minds.

To see how that works we reparameterise the model in terms of a “location” parameter $$\lambda$$ and “dispersion”/scale-ish parameter $$\alpha$$, such that we can rewrite it.

Spelled
$$\text{Pólya}(\lambda,\alpha)$$
Pmf
$$\mathbb{P}(k;\lambda,\alpha) = \frac{ \Gamma\left( k+\frac{1}{\alpha} \right) }{ \Gamma(k+1)\Gamma\left( \frac{1}{\alpha} \right) } \left( \frac{ \lambda }{ \lambda+\frac{1}{\alpha} }\right)^k\left( 1+\lambda\alpha \right)^{-\frac{1}{\alpha} }$$
Mean
$$\lambda$$
Variance
$$\lambda + \lambda^2\alpha$$
Pgf
$$G(s;\lambda,\alpha)=(\alpha (\lambda -\lambda s)+1)^{-1/\alpha }$$

The log Pmf is thus

$\mathbb{P}(k;\lambda,\alpha) =\begin{array}{l} \log\Gamma\left(k+\frac{1}{\alpha}\right)\\- \log\Gamma(k+1)\\- \log\Gamma\left( \frac{1}{\alpha} \right)\\+k\left( \log\lambda-\log\left( \lambda+\frac{1}{\alpha} \right) \right)\\- \frac{\log(1+\lambda\alpha)}{\alpha} \end{array}$

All these log-gamma differences are numerically unstable if calculated naïvely, so we need to use different numerical methods depending upon the combination of $$k,\lambda$$ and $$\alpha$$ values in play. Aieee! Tedious!

To bother with this nonsense, one would need a strong reason to believe the negative binomial was the true model; otherwise, for example, the Lagrangian Poisson-Paisson is easier to fit, in that the log Pmf is numerically tractable for all parameter combinations even with large count values, and it is typically no less natural for common data sets.

## Geometric

A discrete analogue of the exponential, i.e. the probability distribution of the number X of Bernoulli trials before the first success, supported on the set $$\{0, 1, 2, 3, …\}$$

Spelled
$$\text{Geom}(p)$$
Pmf
$$\mathbb{P}(k;p) = (1-p)^k\,p\!$$
Pgf
$$G(s;p)=\frac{p}{1-s+sp}.$$
Mean
$$\frac {1-p}{p}$$
Variance
$$\frac {1-p}{p^{2}}$$

Note that $${\operatorname{Geom}}(p)={\operatorname{NB}}(1,\,1-p).\,$$.

### Mean parameterisation

We can parameterise this in terms of the mean $$\lambda=\frac {1-p}{p}\Rightarrow p=\frac{1}{\lambda+1}$$

Spelled
$$\operatorname{MGeom}(\lambda)$$
Pmf
$$\mathbb{P}(k;\lambda) = \left(\frac{\lambda}{\lambda+1}\right)^k\frac{1}{\lambda+1}$$
Pgf
$$G(s;\lambda)=\frac{\lambda+1}{s\lambda+1}.$$
Mean
$$\lambda$$
Variance
$$\lambda^2+\lambda$$

## Lagrangian distributions

I like this family. Handled under cascade models. This family includes various others on this page; I will check which some day.

## Discrete Stable

Another generalisation of Poisson, with features such as a power-law tail.

By analogy with the continuous-stable distribution, a “stable” family for count data.

In particular, this is stable in the sense that it is a limiting distribution for sums of count random variables, analogous to the continuous stable family for real-valued RVs.

No (convenient) closed form for the Pmf in the general case, but the Pgf is simple, so that’s something.

Spelled
$$\operatorname{DS}(\nu,a)$$
Pmf
$$\mathbb{P}(k;\nu,a)= \left.\frac{1}{k!} \frac{d^kG(s;\nu,a)}{ds^k}\right|_{s=0}.$$ (which is simply the formula for extracting the Pmf from any Pgf.)
Pgf
$$G(s;\nu,a)=\exp(-a (1-s)^\nu).$$
Mean
$$\operatorname{E}(X) = G'(1^-) = a \nu e^{-a}.$$
Variance
$$\operatorname{Var}(X)=G''(1^-) + \operatorname{E}(X) - \operatorname{E}^2(X).$$

Here, $$a>0$$ is a scale parameter and $$0<\nu\leq 1$$ a dispersion parameter describing in particular a power-law tail such that when $$\nu< 1$$,

$\lim_{k \to \infty}\mathbb{P}(k;\nu,a) \simeq \frac{1}{k^{\nu+1}}.$

Question: the Pgf formulation implies this is a non-negative distribution. Does that mean that symmetric discrete RVs cannot be stable? Possibly-negative ones?

give approximate ML-estimators of the parameter. does some legwork:

This thesis considers the interplay between the continuous and discrete properties of random stochastic processes. It is shown that the special cases of the one-sided Lévy-stable distributions can be connected to the class of discrete-stable distributions through a doubly-stochastic Poisson transform. This facilitates the creation of a one-sided stable process for which the N-fold statistics can be factorised explicitly. […] Using the same Poisson transform interrelationship, an exact method for generating discrete-stable variates is found. It has already been shown that discrete-stable distributions occur in the crossing statistics of continuous processes whose autocorrelation exhibits fractal properties. The statistical properties of a nonlinear filter analogue of a phase-screen model are calculated, and the level crossings of the intensity analysed. […] The asymptotic properties of the inter-event density of the process are found to be accurately approximated by a function of the Fano factor and the mean of the crossings alone.

Brrr. That sounds gruelling. You can probably just read the shorter article version though .

## Zipf/Zeta models

The discrete version of the basic power-law models.

While we are here, the plainest explanation of the relation of Zips to Pareto distribution that I know is Lada Adamic’s Zipf, Power-laws, and Pareto - a ranking tutorial.

Spelled
$$\operatorname{Zipf}(s)$$
Pmf
$$\mathbb{P}(k;s)={\frac {1/k^{s}}{\zeta (s)}}$$
Mean
$${\frac {\zeta (s-1)}{\zeta (s)}}~{\text{ for }}~s>2$$
Variance
$${\frac {\zeta (s)\zeta (s-2)-\zeta (s-1)^{2}}{\zeta (s)^{2}}}~{\text{ for }}~s>3$$

This has unbounded support. In the bounded case, it becomes the Zipf—Mandelbrot law, which is too fiddly for me to discuss here unless I turn out to really need it, which would likely be for ranking statistics.

This is a heavy tailed distribution, in that it does not necessarily have finite higher moments.

John D. Cook wrote about these recently.

## Yule-Simon

Spelled
$$\operatorname{YS}(\rho)$$
Pmf
$$\mathbb{P}(k;\rho)=\rho \,{\mathrm {B}}(k,\rho +1)$$
Mean
$${\frac {\rho }{\rho -1}},\, \rho > 1$$
Variance
$${\frac {\rho^2}{(\rho-1)^2\;(\rho -2)}}, \rho > 2$$

where B is the beta function.

Zipf law in the tail. See also the two-parameter version, which replaces the beta function with an incomplete beta function, giving Pmf $$\mathbb{P}(k;\rho,\alpha )={\frac {\rho }{1-\alpha ^{{\rho }}}}\;{\mathrm {B}}_{{1-\alpha }}(k,\rho +1),\,$$

This is also a heavy tailed distribution, in that it does not necessarily have finite higher moments.

I’m bored with this one too.

## Conway-Maxwell-Poisson

Exponential family count model with free variance parameter. See .

## Decomposability properties

For background, see decomposability.

### Stability

By analogy with the continuous case we may construct a stability equation for count RVs:

$X(a) \triangleq W^{1/\alpha}\odot X(b),$

$$\odot$$ here is Steutel and van Harn’s discrete multiplication operator, which I won’t define here exhaustively because there are variously complex formulations of it, and I don’t care enough to wrangle them. In the simplest case it gives us a binomial thinning of the left operand by the right

$A\odot B \sim \mathrm{Binom}(A,B)$

### Self-divisibility

Poisson RVs are self-divisible, in the sense that

\begin{aligned} X_1&\sim \operatorname{Poisson}(\lambda_1),\\ X_2&\sim \operatorname{Poisson}(\lambda_2),\\ X_1&\perp X_2\\ &\Rightarrow \\ X_1+X_2&\sim \operatorname{Poisson}(\lambda_1+\lambda_2). \end{aligned}

Pólya RVs likewise are self-divisible:

\begin{aligned} X_1&\sim \operatorname{Pólya}(\lambda_1, \alpha_1),\,\\ X_2&\sim \operatorname{Pólya}(\lambda_1, \alpha_1),\,\\ X_1&\perp X_2 &\Rightarrow\\ X_1+X_2&\sim \operatorname{Pólya}\left(\lambda_1+\lambda_2,\, \frac{\alpha_1\lambda_1^2+\alpha_2\lambda_2^2}{(\lambda_1+\lambda_2)^2}\right)\\ \operatorname{E}(X_1+X_2)&=\operatorname{E}(X_1)+\operatorname{E}(X_2)\\ \operatorname{Var}(X_1+X_2)&=\operatorname{Var}(X_1)+\operatorname{Var}(X_2) \end{aligned}

So, AFAICS, are GPDs, in $$\lambda$$.

## References

Blaker, Helge. 2000. The Canadian Journal of Statistics / La Revue Canadienne de Statistique 28 (4): 783–98.
Burridge, James. 2013a. arXiv:1304.3741 [Math], April.
———. 2013b. Physical Review E 88 (3): 032124.
Cao, Yang, and Yao Xie. 2015. arXiv:1504.05229 [Cs, Math, Stat], April.
Chatla, Suneel Babu, and Galit Shmueli. 2016. arXiv:1610.08244 [Stat], October.
Consul, P. C. 1988. Generalized Poisson Distributions. New York: CRC Press.
Consul, P. C., and Felix Famoye. 1992. Communications in Statistics - Theory and Methods 21 (1): 89–109.
———. 2006. Lagrangian Probability Distributions. Boston: Birkhäuser.
Consul, P. C., and Famoye Felix. 1989. Statistics 20 (3): 407–15.
Consul, P. C., and G. C. Jain. 1973. Technometrics 15 (4): 791–99.
Consul, P. C., and L. R. Shenton. 1973. Communications in Statistics 2 (3): 263–72.
———. 1975. In A Modern Course on Statistical Distributions in Scientific Work, edited by G. P. Patil, S. Kotz, and J. K. Ord, 41–57. NATO Advanced Study Institutes Series 17. Springer Netherlands.
Consul, P.C., and M. M. Shoukri. 1984. Communications in Statistics - Theory and Methods 13 (12): 1533–47.
Consul, P.C., and M.M. Shoukri. 1988. American Journal of Mathematical and Management Sciences 8 (1-2): 181–202.
Consul, P., and L. Shenton. 1972. SIAM Journal on Applied Mathematics 23 (2): 239–48.
Cox, D. R. 1983. Biometrika 70 (1): 269–74.
Cui, Yunwei, and Robert Lund. 2009. Biometrika 96 (4): 781–92.
Doray, Louis G., Shu Mei Jiang, and Andrew Luong. 2009. Communications in Statistics - Simulation and Computation 38 (9): 2004–17.
Dwass, Meyer. 1969. Journal of Applied Probability 6 (3): 682–86.
Felix, Famoye, and Carl M.-S. Lee. 1992. Communications in Statistics - Simulation and Computation 21 (1): 173–88.
Goldstein, M. L., S. A. Morris, and G. G. Yen. 2004. The European Physical Journal B - Condensed Matter and Complex Systems 41 (2): 255–58.
Greenwood, Major, and G. Udny Yule. 1920. Journal of the Royal Statistical Society 83 (2): 255–79.
Haight, Frank A., and Melvin Allen Breuer. 1960. Biometrika 47 (1-2): 143–50.
Harn, K. van, and F. W. Steutel. 1993. Stochastic Processes and Their Applications 45 (2): 209–30.
Harn, K. van, F. W. Steutel, and W. Vervaat. 1982. Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete 61 (1): 97–118.
Imoto, Tomoaki. 2016. Communications in Statistics - Theory and Methods 45 (3): 712–21.
Janardan, K. 1984. SIAM Journal on Applied Mathematics 44 (4): 854–68.
Johnson, Norman Lloyd, Adrienne W. Kemp, and Samuel Kotz. 2005. Univariate Discrete Distributions. 3rd ed. Hoboken, N.J: Wiley.
Lee, W. H., K. I. Hopcraft, and E. Jakeman. 2008. Physical Review E 77 (1): 011109.
Lee, Wai Ha. 2010. PhD.
Li, S, F Famoye, and C Lee. 2010. “On the Generalized Lagrangian Probability Distributions.” Journal of Probability and Statistical Science 8 (1): 113–23.
Lloyd-Smith, James O. 2007. PLoS ONE 2 (2): e180.
Mutafchiev, Ljuben. 1995. Aequationes Mathematicae 49 (1): 57–85.
Neyman, Jerzy. 1965. Sankhyā: The Indian Journal of Statistics, Series A (1961-2002) 27 (2/4): 249–58.
Nolan, John P. 1997. Communications in Statistics. Stochastic Models 13 (4): 759–74.
———. 2001. In Lévy Processes, edited by Ole E. Barndorff-Nielsen, Sidney I. Resnick, and Thomas Mikosch, 379–400. Birkhäuser Boston.
Nolan, J.P. 1997. In, 1:443–47. IEEE Comput. Soc.
Pitman, Jim. 1998. In Microsurveys in Discrete Probability, edited by David Aldous and James Propp. Vol. 41. DIMACS Series in Discrete Mathematics and Theoretical Computer Science. Providence, Rhode Island: American Mathematical Society.
Saha, Krishna, and Sudhir Paul. 2005. Biometrics 61 (1): 179–85.
Schein, Aaron, Hanna Wallach, and Mingyuan Zhou. 2016. In Advances In Neural Information Processing Systems, 5006–14.
Shmueli, Galit, Thomas P. Minka, Joseph B. Kadane, Sharad Borle, and Peter Boatwright. 2005. Journal of the Royal Statistical Society. Series C (Applied Statistics) 54 (1): 127–42.
Shoukri, M. M., and P. C. Consul. 1987. In Biostatistics, edited by Ian B. MacNeill, Gary J. Umphrey, Allan Donner, and V. Krishna Jandhyala, 259–68. Dordrecht: Springer Netherlands.
Sibuya, Masaaki, Norihiko Miyawaki, and Ushio Sumita. 1994. Journal of Applied Probability 31: 185–97.
Soltani, A. R., A. Shirvani, and F. Alqallaf. 2009. Statistics & Probability Letters 79 (14): 1608–14.
Steutel, F. W., and K. van Harn. 1979. The Annals of Probability 7 (5): 893–99.
Steyvers, Mark, and Joshua B. Tenenbaum. 2005. Cognitive Science 29 (1): 41–78.
Tanner, J. C. 1961. Biometrika 48 (1-2): 222–24.
Tsou, Tsung-Shan. 2006. Journal of Statistical Planning and Inference 136 (9): 3173–86.
Tuenter, Hans J. H. 2000. Statistica Neerlandica 54 (3): 374–76.
Turkman, Kamil Feridun, Manuel González Scotto, and Patrícia de Zea Bermudez. 2014. “Models for Integer-Valued Time Series.” In Non-Linear Time Series, 199–244. Springer International Publishing.
Wedel, Michel, Ulf Böckenholt, and Wagner A. Kamakura. 2003. Journal of Multivariate Analysis 87 (2): 356–69.
Wolpert, Robert L., and Lawrence D. Brown. 2021. arXiv:2105.14591 [Math], May.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.