Stochastic processes which represent measures over the reals



Often I need to have a nonparametric representation for a measure over some non-finite index set. We might want to represent a probability, or mass, or a rate. I might want this representation to be something flexible and low-assumption, like a Gaussian process. If I want a nonparametric representation of functions this is not hard; I can simply use a Gaussian process. What can I use for measures?

If I am working directly with random distributions of mass then I might want other properties like conservation of mass.

Subordinators

A subordinator is an a.s. non-decreasing Lévy process. These can be used a random measures, which means they can be used as priors, which means they are handy for Bayesian nonparametrics. These priors include as a special case Dirichlet process priors in a way I will make precise maybe one day.

A rule of thumb is that while Gaussian process regression is good for giving you curves and more generally, fields, a subordinator prior can be interpreted as giving you non-negative measures. They are always kinda lumpy measures. It is easy to conserve mass with these things.

Other measure priors

Various transforms of Gaussian processes seem popular, e.g. squared or exponentiated. These always seem a little messy to me.

References

Barbour, A. D. “Stein’s Method and Poisson Process Convergence.” Journal of Applied Probability 25 (A): 175–84. https://doi.org/10.2307/3214155.
Barbour, A.D., and T.C. Brown. 1992. “Stein’s Method and Point Process Approximation.” Stochastic Processes and Their Applications 43 (1): 9–31. https://doi.org/10.1016/0304-4149(92)90073-Y.
Barndorff-Nielsen, O. E., and J. Schmiegel. 2004. “Lévy-Based Spatial-Temporal Modelling, with Applications to Turbulence.” Russian Mathematical Surveys 59 (1): 65. https://doi.org/10.1070/RM2004v059n01ABEH000701.
Çinlar, E. 1979. “On Increasing Continuous Processes.” Stochastic Processes and Their Applications 9 (2): 147–54. https://doi.org/10.1016/0304-4149(79)90026-7.
Higdon, Dave. 2002. “Space and Space-Time Modeling Using Process Convolutions.” In Quantitative Methods for Current Environmental Issues, edited by Clive W. Anderson, Vic Barnett, Philip C. Chatwin, and Abdel H. El-Shaarawi, 37–56. London: Springer. https://doi.org/10.1007/978-1-4471-0657-9_2.
James, Lancelot F. 2005. “Bayesian Poisson process partition calculus with an application to Bayesian Lévy moving averages.” Annals of Statistics 33 (4): 1771–99. https://doi.org/10.1214/009053605000000336.
Lijoi, Antonio, and Igor Prünster. 2010. “Models Beyond the Dirichlet Process.” In Bayesian Nonparametrics, edited by Nils Lid Hjort, Chris Holmes, Peter Müller, and Stephen G. Walker. Cambridge University Press. https://doi.org/10.2139/ssrn.1526505.
Nieto-Barajas, Luis E., Igor Prünster, and Stephen G. Walker. 2004. “Normalized random measures driven by increasing additive processes.” Annals of Statistics 32 (6): 2343–60. https://doi.org/10.1214/009053604000000625.
Ranganath, Rajesh, and David M. Blei. 2018. “Correlated Random Measures.” Journal of the American Statistical Association 113 (521): 417–30. https://doi.org/10.1080/01621459.2016.1260468.
Roychowdhury, Anirban, and Brian Kulis. 2015. “Gamma Processes, Stick-Breaking, and Variational Inference.” In Artificial Intelligence and Statistics, 800–808. PMLR. http://proceedings.mlr.press/v38/roychowdhury15.html.
Walker, Stephen G., Paul Damien, PuruShottam W. Laud, and Adrian F. M. Smith. 1999. “Bayesian Nonparametric Inference for Random Distributions and Related Functions.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61 (3): 485–527. https://doi.org/10.1111/1467-9868.00190.
Wolpert, R. 1998. “Poisson/Gamma Random Field Models for Spatial Statistics.” Biometrika 85 (2): 251–67. https://doi.org/10.1093/biomet/85.2.251.
Wolpert, Robert L., and Katja Ickstadt. 1998. “Simulation of Lévy Random Fields.” In Practical Nonparametric and Semiparametric Bayesian Statistics, edited by Dipak Dey, Peter Müller, and Debajyoti Sinha, 227–42. Lecture Notes in Statistics. New York, NY: Springer. https://doi.org/10.1007/978-1-4612-1732-9_12.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.