Causal inference on DAGs

Making valid statistical inference, in the sense of making inference that is compatible with our understanding of the causal relationships that exist in the world (not just the correlations in our data). Graphical models and related techniques for doing it. Avoiding the danger of folk statistics. Observational studies, confounding, adjustment criteria, d-separation, identifiability, interventions, moral equivalence… Avoidance of Ecological fallacy/ Simpson’s paradox.

The gold standard, of course, is to work out if A causes B by doing an experiment where no input but A changes, then observing B, which is what a controlled trial is. In practice this is unattainable because it would usually require cloning the entire state of the universe and running multiple copies in parallel. Statistically it can be nearly as good to do the experiment where we change A and all other influences apart from A are at least uncorrelated with A, which is more usually what we do — a randomised controlled trial. In many circumstances though, (budget restrictions, ethical constraints, bad experimental design…) we cannot do these ideal experiments, and a mathematical crutch is needed to get us the next-best outcome, which is to control for the things that we must (and not control for the things we must not).

In classic-flavoured causal inference, we use graphical models with the additional assumption that \(A\rightarrow B\) may be read as “A causes a change in B”. This is what you end up with if you use a Structural Equation Model (a.k.a. hierarchical models) to impose a causal structure on the observations. The result is a particular type of graph, a Directed Acyclic Graph (DAG) which, informally put, summarises what can possibly affect what in the model. Slightly more formally, it summarises what cannot (conditionally) affect what. C&C conditional treatment effect estimation by potential outcomes.

With this tool in hand I can answer the question of when I can use my crappy observational data, collected without a good experimental design for whatever reason, to do interventional inference? There is a lot of research in this area; I should summarize the salient bits for myself.

1 What can go wrong?

What can I actually identify? For a start, if we are resorting to this more difficult methodology that already suggests that we might be trying to use data which was collected with no regard to our actual statistical needs, and thus we might really stretch to imagine that we can actually find appropriate instruments in the data. Here is an essay on that theme.

• OMFG Exogenous Variation! Or, Can You Find Good Nails When You Find an Indonesian Politics Hammer quotes Angus Deaton

  we have at least some control over the light but choose to let it fall where it may and then proclaim that whatever it illuminates is what we were looking for all along

2 Teaching

Yanir Seroussi’s Causal inference resources recommends

• Causal Diagrams: Draw Your Assumptions Before Your Conclusions. A high-level introduction to causal diagrams by Miguel Hernán. Highly recommended for those who want to get a conceptual overview of how causal diagrams work and why they’re useful.
• A/B Testing by Google: Online Experiment Design and Analysis. Experimentation is key to causal inference, with the online world offering an accessible ground for running experiments. This short course is worth doing if you’re involved in online experiments in any way.

Miguel Hernán and Jamie Robins’ causal inference book (Miguel A. Hernán and Robins 2020) is available in free draft form online. See Yanir Seroussi’s review.

Jonas Peters’ notes from his teaching in 2015 (I think I took this course).

• An Introduction to Bayesian Network Theory and Usage
• pgmpy_notebook/2. Bayesian Networks.ipynb at master · pgmpy/pgmpy_notebook
• cs228 notes

Samantha Kleinberg wrote two classes, introductory and advanced. The latter is notable for handling time-dependent causality.

Tutorial: David Sontag and Uri Shalit, Causal inference from observational studies. Mastering Metrics: The Path from Cause to Effect A resource list for causality in statistics, data science and physics.

Brady Neal’s Introduction to Causal Inference includes his draft textbook.

Felix Elwert’s summary is good (Elwert 2013).

Chapter 3 of (some edition of) Pearl’s book is available as an author’s preprint: Part 1, 2, 3, 4, 5, 6.

Stanford encyclopaedia of philosophy entry.

Various classic introductions (Pearl 2012, 1998; Elwert 2013; Morgan and Winship 2015; Rohrer 2018). Notably not recommended as a pedagogic experience (Koller and Friedman 2009) (although as a reference text it is great and will make you smarter).

The dagitty intro is an interactive guide via visualizations. Likewise, the ggdag bias structure vignette shows off the useful explanation diagrams available in ggdag and is also a good introduction to selection bias and causal DAGs themselves.

Amit Sharma’s tutorial at KDD. See also Emily Riederer’s Causal design patterns for data analysts Spurious correlation induced by sampling bias.

Still confused? Overwhelmed? I am. How about a diagram?