Gaussian processes

“Gaussian Processes” are stochastic processes/fields with joint Gaussian distributions over all finite sets of observation locations. The most familiar to finance and physics people is the Gauss-Markov process, a.k.a. the Wiener process, but there are many others. These processes are convenient due to certain useful properties of the multivariate Gaussian distribution, e.g., being uniquely specified by first and second moments, nice behaviour under various linear operations, and kernel tricks. Especially famous applications include Gaussian process regression and spatial statistics. Check out Ti’s Interactive visualization for some examples.

Gaussian processes are, specifically, probabilistic distributions over random functions \(\mathcal{T}\to \mathbb{C}\) for some index (or argument) set \(\mathcal{T}\), often taken to be \(\mathcal{T}:=\mathbb{R}^d\).

We typically work with a mean-zero process, meaning for every finite set \(\mathbf{f}:=\{f(t_k);k=1,\dots,K\}\) of observations of that process, the joint distribution is mean-zero Gaussian, \[\begin{aligned} \mathbf{f}(t) &\sim \operatorname{GP}\left(0, \kappa(t, t';\mathbf{\theta})\right) \\ &\Rightarrow\\ p(\mathbf{f}) &=(2\pi)^{-{\frac {K}{2}}}\det({\boldsymbol {\mathrm{K} }})^{-{\frac {1}{2}}}\,e^{-{\frac {1}{2}}\mathbf {f}^{\!{\mathsf {T}}}{\boldsymbol {\mathrm{K} }}^{-1}\mathbf {f}}\\ &=\mathcal{N}(\mathbf{f};0, \mathrm{K}). \end{aligned}\] where \(\mathrm{K}\) is the sample covariance matrix defined such that its entries are given by \(\mathrm{K}_{jk}=\kappa(t_j,t_k).\) This is the covariance kernel that maps from function argument — \(t\) — to the second moment of function values. In this case, we are specifying only the second moments, which gives all the remaining properties of the process.