Continuous and equilibrium probabilistic graphical models

August 5, 2014 — October 10, 2022

graphical models
machine learning
Figure 1

Placeholder for my notes on probabilistic graphical models over a continuum, i.e. with possibly-uncountably many nodes in the graph; or put another way, where the random field has an uncountable index set (but some kind of structure — a metric space, say.) There is much formalising to be done, which I do not propose to attempt right now, in lieu, here are some notes.

Normally when we discuss graphical models it is in terms of a finitely indexed set of rvs \(\{X_i;i=1,\ldots,n\}\). If that index \(i\) ranges instead over a continuum then what does the gaphical model formalism look like?

Here’s a concrete example. Consider Gaussian process whose covariance kernel \(K\) is continuous and smoothly decays. Let it be over index space \(\mathcal{T}:=\mathbb{R}^n\) for the sake of argument. It implicitly defines an undirected graphical model where for any given observation index \(t_0\in\mathcal{T}\), the value \(x_0\) is influenced by the values of the field at \(\operatorname{supp}\{K(\cdot, t_0)\}\); (or a really a continuum of different strengths of influence depending on the magnitude of the kernel). This kind of setup is very important in spatiotemporal modelling.

Does the standard finite dimensional distribution argument get us anywhere in this setting if we can introduce some conditional independence?

I suspect that (Lauritzen 1996) is sufficiently general to cover such cases, but TBH I haven’t read it for long enough that I can’t remember.

1 Handling continuous index spaces by pretending a continuous field is discrete

(Eichler, Dahlhaus, and Dueck 2016) is probably an example of what I mean; they construct a continuous index directed graphical model for point process fields, based on limiting cases of a discrete field, which seems like the obvious method of attack.

(Hansen and Sokol 2014; Schulam and Saria 2017) tackle this by considering SDE influence via limits of discretizations of SDEs which is, now I think of it, an intuitive way to approach this problem.

2 The central european school

There is a strand of research in this area that I am just starting to notice across Tübingen, Amsterdam, and Zürich. Causality on continuous index spaces, and, which turns out to be related, equilibrium/feedback dynamics.

Figure 2

Maybe start from Schölkopf et al. (2012)? Bongers and Mooij (2018) gives the flavour of a more recent result.

Uncertainty and random fluctuations are a very common feature of real dynamical systems. For example, most physical, financial, biochemical and engineering systems are subjected to time-varying external or internal random disturbances. These complex disturbances and their associated responses are most naturally described in terms of stochastic processes. A more realistic formulation of a dynamical system in terms of differential equations should involve such stochastic processes. This led to the fields of stochastic and random differential equations, where the latter deals with processes that are sufficiently regular. Random differential equations (RDEs) provide the most natural extension of ordinary differential equations to the stochastic setting and have been widely accepted as an important mathematical tool in modeling…

Over the years, several attempts have been made to interpret these structural causal models that include cyclic causal relationships. They can be derived from an underlying discrete-time or continuous-time dynamical system. All these methods assume that the dynamical system under consideration converges to a single static equilibrium… These assumptions give rise to a more parsimonious description of the causal relationships of the equilibrium states and ignore the complicated but decaying transient dynamics of the dynamical system. The assumption that the system has to equilibrate to a single static equilibrium is rather strong and limits the applicability of the theory, as many dynamical systems have multiple equilibrium states.

In this paper, we relax this condition and capture, under certain convergence assumptions, every random equilibrium state of the RDE in an SCM. Conversely, we show that under suitable conditions, every solution of the SCM corresponds to a sample-path solution of the RDE. Intuitively, the idea is that in the limit when time tends to infinity the random differential equations converge exactly to the structural equations of the SCM.

“RDEs” seem to be stochastic differential equations with differentiable sample paths.

3 Incoming

4 References

Aalen, Røysland, Gran, et al. 2012. Causality, Mediation and Time: A Dynamic Viewpoint.” Journal of the Royal Statistical Society: Series A (Statistics in Society).
Akbari, Winter, and Tomko. 2023. Spatial Causality: A Systematic Review on Spatial Causal Inference.” Geographical Analysis.
Bishop. 2006. Pattern Recognition and Machine Learning. Information Science and Statistics.
Blom, Bongers, and Mooij. 2020. Beyond Structural Causal Models: Causal Constraints Models.” In Uncertainty in Artificial Intelligence.
Bongers, Forré, Peters, et al. 2020. Foundations of Structural Causal Models with Cycles and Latent Variables.” arXiv:1611.06221 [Cs, Stat].
Bongers, and Mooij. 2018. From Random Differential Equations to Structural Causal Models: The Stochastic Case.” arXiv:1803.08784 [Cs, Stat].
Bongers, Peters, Schölkopf, et al. 2016. Structural Causal Models: Cycles, Marginalizations, Exogenous Reparametrizations and Reductions.” arXiv:1611.06221 [Cs, Stat].
Dash. 2003. Caveats For Causal Reasoning With Equilibrium Models.”
Dash, and Druzdzel. 2001. Caveats For Causal Reasoning With Equilibrium Models.” In Symbolic and Quantitative Approaches to Reasoning with Uncertainty.
Eichler, Dahlhaus, and Dueck. 2016. Graphical Modeling for Multivariate Hawkes Processes with Nonparametric Link Functions.” Journal of Time Series Analysis.
Glymour. 2007. When Is a Brain Like the Planet? Philosophy of Science.
Hansen, and Sokol. 2014. Causal Interpretation of Stochastic Differential Equations.” Electronic Journal of Probability.
Lauritzen. 1996. Graphical Models. Oxford Statistical Science Series.
Lopez-Paz, Nishihara, Chintala, et al. 2016. Discovering Causal Signals in Images.” arXiv:1605.08179 [Cs, Stat].
Mogensen, Malinsky, and Hansen. 2018. Causal Learning for Partially Observed Stochastic Dynamical Systems.” In UAI2018.
Peters, Janzing, and Schölkopf. 2017. Elements of Causal Inference: Foundations and Learning Algorithms. Adaptive Computation and Machine Learning Series.
Peters, Mooij, Janzing, et al. 2012. Identifiability of Causal Graphs Using Functional Models.” arXiv:1202.3757 [Cs, Stat].
Peters, Mooij, Janzing, et al. 2014. “Causal Discovery with Continuous Additive Noise Models.” The Journal of Machine Learning Research.
Rubenstein, Bongers, Schölkopf, et al. 2018. From Deterministic ODEs to Dynamic Structural Causal Models.” In Uncertainty in Artificial Intelligence.
Rubenstein, Weichwald, Bongers, et al. 2017. Causal Consistency of Structural Equation Models.” arXiv:1707.00819 [Cs, Stat].
Runge, Bathiany, Bollt, et al. 2019. Inferring Causation from Time Series in Earth System Sciences.” Nature Communications.
Schölkopf. 2022. Causality for Machine Learning.” In Probabilistic and Causal Inference: The Works of Judea Pearl.
Schölkopf, Janzing, Peters, et al. 2012. On Causal and Anticausal Learning.” In ICML 2012.
Schulam, and Saria. 2017. Reliable Decision Support Using Counterfactual Models.” In Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS’17.
Wang, Sankaran, and Perdikaris. 2022. Respecting Causality Is All You Need for Training Physics-Informed Neural Networks.”