q-exponential process

April 29, 2025 — April 29, 2025

Gaussian
Hilbert space
kernel tricks
Lévy processes
nonparametric
regression
spatial
stochastic processes
time series
Figure 1

An interesting Bayesian functional regression trick based on the so-called q-exponential distribution: exp(12|u|q)$ which has a special relationship with Besov spaces and so connects to functional inverse problems. It seems to be in the same family as elliptical process as per Bånkestad et al. ().

NB, the q-exponential distribution is not the Tsallis q-exponential distribution but rather one developed by Dashti, Harris, and Stuart ().

Li, O’Connor, and Lan ():

Regularization is one of the most fundamental topics in optimization, statistics and machine learning. To get sparsity in estimating a parameter uRd, an q penalty term, uq, is usually added to the objective function. What is the probabilistic distribution corresponding to such q penalty? What is the correct stochastic process corresponding to uq when we model functions uLq ? This is important for statistically modeling high-dimensional objects such as images, with penalty to preserve certain properties, e.g. edges in the image. In this work, we generalize the q-exponential distribution (with density proportional to) exp(12|u|q) to a stochastic process named q-exponential ( QEP ) process that corresponds to the Lq regularization of functions. The key step is to specify consistent multivariate q-exponential distributions by choosing from a large family of elliptic contour distributions. The work is closely related to Besov process which is usually defined in terms of series. Q-EP can be regarded as a definition of Besov process with explicit probabilistic formulation, direct control on the correlation strength, and tractable prediction formula. From the Bayesian perspective, QEP provides a flexible prior on functions with sharper penalty (q<2) than the commonly used Gaussian process (GP, q=2 ). We compare GP, Besov and Q-EP in modeling functional data, reconstructing images and solving inverse problems and demonstrate the advantage of our proposed methodology.

1 References

Bånkestad, Sjölund, Taghia, et al. 2020. The Elliptical Processes: A Family of Fat-Tailed Stochastic Processes.”
Chang, Obite, Zhou, et al. 2024. Deep Q-Exponential Processes.”
Dashti, Harris, and Stuart. n.d. Besov Priors for Bayesian Inverse Problems.” Inverse Problems and Imaging.
Li, O’Connor, and Lan. 2023. Bayesian Learning via Q-Exponential Process.” Advances in Neural Information Processing Systems.