Continuous and equilibrium probabilistic graphical models



Placeholder for my notes on probabilistic graphical models over a continuum, i.e. with possibly-uncountably many nodes in the graph; or put another way, where the random field has an uncountable index set (but some kind of structure — a metric space, say.) There is much formalising to be done, which I do not propose to attempt right now, in lieu, here are some notes.

Normally when we discuss graphical models it is in terms of a finitely indexed set of rvs \(\{X_i;i=1,\ldots,n\}\). If that index \(i\) ranges instead over a continuum then what does the gaphical model formalism look like?

Here’s a concrete example. Consider Gaussian process whose covariance kernel \(K\) is continuous and smoothly decays. Let it be over index space \(\mathcal{T}:=\mathbb{R}^n\) for the sake of argument. It implicitly defines an undirected graphical model where for any given observation index \(t_0\in\mathcal{T}\), the value \(x_0\) is influenced by the values of the field at \(\operatorname{supp}\{K(\cdot, t_0)\}\); (or a really a continuum of different strengths of influence depending on the magnitude of the kernel). This kind of setup is very important in spatiotemporal modelling.

Does the standard finite dimensional distribution argument get us anywhere in this setting if we can introduce some conditional independence?

I suspect that (Lauritzen 1996) is sufficiently general to cover such cases, but TBH I haven’t read it for long enough that I can’t remember.

Handling continuous index spaces by pretending a continuous field is discrete

(Eichler, Dahlhaus, and Dueck 2016) is probably an example of what I mean; they construct a continuous index directed graphical model for point process fields, based on limiting cases of a discrete field, which seems like the obvious method of attack.

(Hansen and Sokol 2014; Schulam and Saria 2017) tackle this by considering SDE influence via limits of discretizations of SDEs which is, now I think of it, an intuitive way to approach this problem.

The central european school

There is a strand of research in this area that I am just starting to notice across Tübingen, Amsterdam, and Zürich. Causality on continuous index spaces, and, which turns out to be related, equilibrium/feedback dynamics.

Maybe start from Schölkopf, Bernhard et al. (2012)? Bongers and Mooij (2018) gives the flavour of a more recent result.

Uncertainty and random fluctuations are a very common feature of real dynamical systems. For example, most physical, financial, biochemical and engineering systems are subjected to time-varying external or internal random disturbances. These complex disturbances and their associated responses are most naturally described in terms of stochastic processes. A more realistic formulation of a dynamical system in terms of differential equations should involve such stochastic processes. This led to the fields of stochastic and random differential equations, where the latter deals with processes that are sufficiently regular. Random differential equations (RDEs) provide the most natural extension of ordinary differential equations to the stochastic setting and have been widely accepted as an important mathematical tool in modeling…

Over the years, several attempts have been made to interpret these structural causal models that include cyclic causal relationships. They can be derived from an underlying discrete-time or continuous-time dynamical system. All these methods assume that the dynamical system under consideration converges to a single static equilibrium… These assumptions give rise to a more parsimonious description of the causal relationships of the equilibrium states and ignore the complicated but decaying transient dynamics of the dynamical system. The assumption that the system has to equilibrate to a single static equilibrium is rather strong and limits the applicability of the theory, as many dynamical systems have multiple equilibrium states.

In this paper, we relax this condition and capture, under certain convergence assumptions, every random equilibrium state of the RDE in an SCM. Conversely, we show that under suitable conditions, every solution of the SCM corresponds to a sample-path solution of the RDE. Intuitively, the idea is that in the limit when time tends to infinity the random differential equations converge exactly to the structural equations of the SCM.

“RDEs” seem to be stochastic differential equations with differentiable sample paths.

References

Aalen, Odd O., Kjetil Røysland, Jon Michael Gran, and Bruno Ledergerber. 2012. Causality, Mediation and Time: A Dynamic Viewpoint.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 175 (4): 831–61.
Akbari, Kamal, Stephan Winter, and Martin Tomko. n.d. Spatial Causality: A Systematic Review on Spatial Causal Inference.” Geographical Analysis n/a (n/a).
Bishop, Christopher M. 2006. Pattern Recognition and Machine Learning. Information Science and Statistics. New York: Springer.
Blom, Tineke, Stephan Bongers, and Joris M. Mooij. 2020. Beyond Structural Causal Models: Causal Constraints Models.” In Uncertainty in Artificial Intelligence, 585–94. PMLR.
Bongers, Stephan, Patrick Forré, Jonas Peters, Bernhard Schölkopf, and Joris M. Mooij. 2020. Foundations of Structural Causal Models with Cycles and Latent Variables.” arXiv:1611.06221 [Cs, Stat], October.
Bongers, Stephan, and Joris M. Mooij. 2018. From Random Differential Equations to Structural Causal Models: The Stochastic Case.” arXiv:1803.08784 [Cs, Stat], March.
Bongers, Stephan, Jonas Peters, Bernhard Schölkopf, and Joris M. Mooij. 2016. Structural Causal Models: Cycles, Marginalizations, Exogenous Reparametrizations and Reductions.” arXiv:1611.06221 [Cs, Stat], November.
Dash, Denver. 2003. Caveats For Causal Reasoning With Equilibrium Models.”
Dash, Denver, and Marek Druzdzel. 2001. Caveats For Causal Reasoning With Equilibrium Models.” In Symbolic and Quantitative Approaches to Reasoning with Uncertainty, edited by Salem Benferhat and Philippe Besnard, 2143:192–203. Berlin, Heidelberg: Springer Berlin Heidelberg.
Eichler, Michael, Rainer Dahlhaus, and Johannes Dueck. 2016. Graphical Modeling for Multivariate Hawkes Processes with Nonparametric Link Functions.” Journal of Time Series Analysis, January, n/a–.
Glymour, Clark. 2007. When Is a Brain Like the Planet? Philosophy of Science 74 (3): 330–46.
Hansen, Niels, and Alexander Sokol. 2014. Causal Interpretation of Stochastic Differential Equations.” Electronic Journal of Probability 19.
Lauritzen, Steffen L. 1996. Graphical Models. Oxford Statistical Science Series. Clarendon Press.
Lopez-Paz, David, Robert Nishihara, Soumith Chintala, Bernhard Schölkopf, and Léon Bottou. 2016. Discovering Causal Signals in Images.” arXiv:1605.08179 [Cs, Stat], May.
Mogensen, Søren Wengel, Daniel Malinsky, and Niels Richard Hansen. 2018. Causal Learning for Partially Observed Stochastic Dynamical Systems.” In UAI2018, 17.
Peters, Jonas, Dominik Janzing, and Bernhard Schölkopf. 2017. Elements of Causal Inference: Foundations and Learning Algorithms. Adaptive Computation and Machine Learning Series. Cambridge, Massachuestts: The MIT Press.
Peters, Jonas, Joris M. Mooij, Dominik Janzing, and Bernhard Schölkopf. 2014. “Causal Discovery with Continuous Additive Noise Models.” The Journal of Machine Learning Research 15 (1): 2009–53.
Peters, Jonas, Joris Mooij, Dominik Janzing, and Bernhard Schoelkopf. 2012. Identifiability of Causal Graphs Using Functional Models.” arXiv:1202.3757 [Cs, Stat], February.
Rubenstein, Paul K., Stephan Bongers, Bernhard Schölkopf, and Joris M. Mooij. 2018. From Deterministic ODEs to Dynamic Structural Causal Models.” In Uncertainty in Artificial Intelligence.
Rubenstein, Paul K., Sebastian Weichwald, Stephan Bongers, Joris M. Mooij, Dominik Janzing, Moritz Grosse-Wentrup, and Bernhard Schölkopf. 2017. Causal Consistency of Structural Equation Models.” arXiv:1707.00819 [Cs, Stat], July.
Runge, Jakob, Sebastian Bathiany, Erik Bollt, Gustau Camps-Valls, Dim Coumou, Ethan Deyle, Clark Glymour, et al. 2019. Inferring Causation from Time Series in Earth System Sciences.” Nature Communications 10 (1): 2553.
Schölkopf, Bernhard. 2019. Causality for Machine Learning.” arXiv:1911.10500 [Cs, Stat], December.
Schölkopf, Bernhard, Bernhard, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. 2012. On Causal and Anticausal Learning.” In ICML 2012.
Schulam, Peter, and Suchi Saria. 2017. Reliable Decision Support Using Counterfactual Models.” In Proceedings of the 31st International Conference on Neural Information Processing Systems, 1696–706. NIPS’17. Red Hook, NY, USA: Curran Associates Inc.
Wang, Sifan, Shyam Sankaran, and Paris Perdikaris. 2022. Respecting Causality Is All You Need for Training Physics-Informed Neural Networks.” arXiv.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.