Probabilistic graphical models over continuous index sets


Placeholder for my notes on probabilistic graphical models over a continuum, i.e. with possibly-uncountably many nodes in the graph; or put another way, where the random field has an uncountable index set (but some kind of structure — a metric space, say.)

There is much formalising to be done here, which I do not propose to attempt right now. Here’s a concrete example. Consider Gaussian process whose covariance kerne \(K\) is continuous and of bounded support. Let it be over index space \(\mathcal{T}:=\mathbb{R}^n\) for the sake of argument. It implicitly defines an undirected graphical model where for any given observation index \(t_0\in\mathcal{T}\), the value \(x_0\) is influenced by the values of the field at \(\operatorname{supp}\{K(\cdot, t_0)\}\); (or a really a continuum of different strengths of influence depending on the magnitude of the kernel).

Does this kind of factoring buy us anything? Does the standard finite dimensional distribution argument get us anywhere in this setting if we can introduce some conditional independence?

I suspect that (Lauritzen 1996) is sufficiently general to cover this, but TBH I haven’t read it for long enough that I can’t remember. (Eichler, Dahlhaus, and Dueck 2016) is probably an example of what I mean; they construct a continuous index directed graphical model for point process fields, based on limiting cases of a discrete field, which seems like the obvious method of attack.

(Hansen and Sokol 2014; Schulam and Saria 2017) tackle this by considering SDE influence via limits of discretizations of SDEs which is, now I think of it, an intuitive way to approach this problem.

Bishop, Christopher M. 2006. Pattern Recognition and Machine Learning. Information Science and Statistics. New York: Springer.

Blom, Tineke, Stephan Bongers, and Joris M. Mooij. 2020. “Beyond Structural Causal Models: Causal Constraints Models.” In Uncertainty in Artificial Intelligence, 585–94. PMLR. http://proceedings.mlr.press/v115/blom20a.html.

Bongers, Stephan, and Joris M. Mooij. 2018. “From Random Differential Equations to Structural Causal Models: The Stochastic Case.” March 27, 2018. http://arxiv.org/abs/1803.08784.

Eichler, Michael, Rainer Dahlhaus, and Johannes Dueck. 2016. “Graphical Modeling for Multivariate Hawkes Processes with Nonparametric Link Functions.” Journal of Time Series Analysis, January, n/a–n/a. https://doi.org/10.1111/jtsa.12213.

Hansen, Niels, and Alexander Sokol. 2014. “Causal Interpretation of Stochastic Differential Equations.” Electronic Journal of Probability 19. https://doi.org/10.1214/EJP.v19-2891.

Lauritzen, Steffen L. 1996. Graphical Models. Clarendon Press.

Mogensen, Søren Wengel, Daniel Malinsky, and Niels Richard Hansen. 2018. “Causal Learning for Partially Observed Stochastic Dynamical Systems.” In UAI2018, 17. http://auai.org/uai2018/proceedings/papers/142.pdf.

Rubenstein, Paul K., Stephan Bongers, Bernhard Schölkopf, and Joris M. Mooij. 2018. “From Deterministic ODEs to Dynamic Structural Causal Models.” July 9, 2018. http://arxiv.org/abs/1608.08028.

Schulam, Peter, and Suchi Saria. 2017. “Reliable Decision Support Using Counterfactual Models.” In Proceedings of the 31st International Conference on Neural Information Processing Systems, 1696–1706. NIPS’17. Red Hook, NY, USA: Curran Associates Inc. http://papers.nips.cc/paper/6767-reliable-decision-support-using-counterfactual-models.pdf.