A nonlinear way of transforming stationary kernels into non-stationary ones by transforming their inputs (Sampson and Guttorp 1992; Genton 2001; Genton and Perrin 2004; Perrin and Senoussi 1999, 2000).

This is of interest in the context of composing kernels to have known desirable properties by known transforms, and also learning (somewhat) arbitrary transforms to attain stationarity.

## Stationary reducible kernels

The main idea is to find a new feature space where stationarity (Sampson and Guttorp 1992) or local stationarity (Perrin and Senoussi 1999, 2000; Genton and Perrin 2004) can be achieved.

We say that a nonstationary kernel \(K(\mathbf{x}, \mathbf{z})\) is stationary reducible if there exist a bijective deformation \(\Phi\) such that: \[ K(\mathbf{x}, \mathbf{z})=K_{S}^{*}(\mathbf{\Phi}(\mathbf{x})-\mathbf{\Phi}(\mathbf{z})) \] where \(K_{S}^{*}\) is a stationary kernel.

## Classic deformations

### MacKay warping

### As a function of input

Invented apparently by Gibbs (1998) and generalised in Paciorek and Schervish (2003).

Let \(k_S\) be some stationary kernel on \(\mathbb{R}^D.\) Let \(\Sigma(\mathbf{x})\) be a \(D \times D\) matrix-valued function which is positive definite for all \(\mathbf{x},\) and let \(\Sigma_{i} \triangleq \Sigma\left(\mathbf{x}_{i}\right) .\) ) Then define \[ Q_{i j}=\left(\mathbf{x}_{i}-\mathbf{x}_{j}\right)^{\top}\left(\left(\Sigma_{i}+\Sigma_{j}\right) / 2\right)^{-1}\left(\mathbf{x}_{i}-\mathbf{x}_{j}\right) \] Then \[ k_{\mathrm{NS}}\left(\mathbf{x}_{i}, \mathbf{x}_{j}\right)=2^{D / 2}\left|\Sigma_{i}\right|^{1 / 4}\left|\Sigma_{j}\right|^{1 / 4}\left|\Sigma_{i}+\Sigma_{j}\right|^{-1 / 2} k_{\mathrm{S}}\left(\sqrt{Q_{i j}}\right) \] is a valid non-stationary covariance function.

Homework question: Is this a product of convolutional gaussian processes.

## Learning transforms

## References

*International Conference on Machine Learning*, 541–49. http://arxiv.org/abs/1802.01396.

*Environmetrics*12 (2): 161–78. https://doi.org/10.1002/1099-095X(200103)12:2<161::AID-ENV452>3.0.CO;2-G.

*Journal of Machine Learning Research*2 (December): 299–312. http://jmlr.org/papers/volume2/genton01a/genton01a.pdf.

*Journal of Applied Probability*41 (1, 1): 236–49. https://doi.org/10.1239/jap/1077134681.

*Advances in Neural Information Processing Systems 20*, edited by J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, 1249–56. Curran Associates, Inc. http://papers.nips.cc/paper/3211-using-deep-belief-nets-to-learn-covariance-kernels-for-gaussian-processes.pdf.

*Proceedings of the 16th International Conference on Neural Information Processing Systems*, 16:273–80. NIPS’03. Cambridge, MA, USA: MIT Press. https://papers.nips.cc/paper/2003/hash/326a8c055c0d04f5b06544665d8bb3ea-Abstract.html.

*Statistics & Probability Letters*43 (4): 393–97. https://doi.org/10.1016/S0167-7152(98)00278-8.

*Statistics & Probability Letters*48 (1): 23–32. https://doi.org/10.1016/S0167-7152(99)00188-1.

*Gaussian Processes for Machine Learning*. Adaptive Computation and Machine Learning. Cambridge, Mass: Max-Planck-Gesellschaft; MIT Press. http://www.gaussianprocess.org/gpml/.

*Journal of the American Statistical Association*87 (417): 108–19. https://doi.org/10.1080/01621459.1992.10475181.

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*65 (3): 743–58. https://doi.org/10.1111/1467-9868.00413.

*The Annals of Statistics*32 (2): 656–92. https://doi.org/10.1214/009053604000000139.

*Proceedings of the 31st International Conference on Machine Learning (ICML-14)*, 1674–82. http://www.jmlr.org/proceedings/papers/v32/snoek14.pdf.

*Proceedings of the AAAI Conference on Artificial Intelligence*32 (1, 1). https://ojs.aaai.org/index.php/AAAI/article/view/11696.

*Artificial Intelligence and Statistics*, 370–78. PMLR. http://proceedings.mlr.press/v51/wilson16.html.

## No comments yet!