Reconciliation of overlapping Gaussian processes

Combining Gaussian processes on the same domain; consistency; coherence; generalized phase retrieval

August 12, 2024 — August 12, 2024

functional analysis

Gaussian

generative

Hilbert space

kernel tricks

nonparametric

regression

spatial

stochastic processes

time series

Suspiciously similar content

Suppose I have two random functions $f_{1} (x)$ and $f_{2} (x)$ , each of which is a Gaussian process, i.e. a well-behaved random function. The two functions are defined on the same domain $x \in X$ , and I want to reconcile them in some way. This could be because they are noisy observations of the same underlying function or because they are two different models of the same phenomenon. This kind of thing arises often in the context of spatiotemporal modeling. There is also a close connection to Gaussian Belief Propagation. In either case, I want to find a way to combine them into a single random function $f (x)$ that captures the information in both. Ideally, it should behave like a standard update does, e.g. if they both agree with high certainty, then the combined function should also agree with high certainty. If they do not agree, or have low certainty, then the combined function should reflect that by having low certainty.

I am sure that this must be well-studied, but it is one of those things that is rather hard to google for and ends up being easier to work out by hand, which is what I do here.

1 Overlapping GPs

Suppose I have two GP priors $f_{1} (x) \sim GP (m_{1} (x), k_{1} (x, x^{'}))$ and $f_{2} (x) \sim GP (m_{2} (x), k_{2} (x, x^{'}))$ defined over the same index set. They could be two posterior likelihood updates, or two expert priors, or whatever.

How do we reconcile these two GPs into a single GP $f (x)$ ?

The standard answer for Gaussian processes is to find a new one whose density is the product of the density of the two components.

2 Enveloping overlapping GPs

3 Connection to classical methods

This idea bears a resemblance to Griffin-Lim iteration phase recovery, where we have two overlapping signals and we want to combine them into a single signal that is consistent with both. That case is somewhat different because it assumes

a point estimate will do; we do not talk about random functions, and
The covariance kernels are implicitly stationary, which we do not assume here (and as such, there is not necessarily a “phase” to “recover”).