UNSW Stats reading group 2016 - Causal DAGs

DAGs

DAG: Directed (probabilistic) graphical model. Graph defined, as usual, defined by a set of vertexes and edges.

\[ \mathcal{G}=(\mathbf{V},E) \]

We show the directs of edges by writing them as arrows.

For nodes \(X,Y\in V\) we write (XY) to mean there is a directed edged joining them.

Familiar from, e.g., Structural equation models, hierarchical models, expert systems. General graph theory…

A graph with directed edges, and no cycles. (you cannot return to the same starting node traveling only forward along the arrows)

We need some terminology.

Parents

The parents of a node \(X\) in a graph are all nodes joined to it by an incoming arrow, \(\operatorname{parents}(X)=\{Y\in V:Y\rightarrow X\}.\)

Children

similarly, \(\operatorname{parents}(X)=\{Y\in V:X\rightarrow Y\}.\)

Co-parent

\(\operatorname{coparents}(X)=\{Y\in V:\exists Z\in V \text{ s.t. } X\rightarrow Z\text{ and }Y\rightarrow Z\}.\)

Ancestors and descendants should be clear as well. For convenience, we define \(X\in\operatorname{parents}(X)\)

Random variables

I will deal with finite collections of random variables \(\mathbf{V}\).

For simplicity of exposition, each of the RVs will be supported on \(\mathcal{X}_i\subset\mathbb{Z}\), so that we may work with pmfs, and write \(p(X_i|X_j)\) for the pmf. I may write \(p(x_i|x_j)\) to mean \(p(X_i=x_i|X_j=x_j)\).

Also we are working with sets of random variables rather than sets of events and the discrete state space reduces the need to discuss sets of events.

Extension to continuous RVs, or arbitrary RVs is trivial for everything I discuss here. (A challenge is if the probabilities are not all strictly positive.)

Motivation in terms of structural models.

\[\begin{aligned} X_6 &= f_6(X_4, X_3, \varepsilon_6) \\ X_5 &= f_5(X_4, X_3, \varepsilon_5) \\ X_4 &= f_4(X_3, X_2, X_1, \varepsilon_4) \\ X_3 &= f_3(\varepsilon_3) \\ X_2 &= f_2(X_1, \varepsilon_2) \\ X_1 &= f_3(\varepsilon_1) \\ \end{aligned}\]

Without further information about the forms of \(f_i\) or \(\varepsilon_i\), our assumptions have constrained our conditional independence relations to permit a particular factorization of the mass function:

\[ p(x_6, x_5, x_4, x_3, x_2, x_1) = p(x_1) p(x_2|x_1) p(x_3) p(x_4|x_1, x_2, x_3) p(x_5|x_3,x_4) p(x_6|x_3,x_4) \]

We are “nonparametric” in the sense that working with this conditional factorization does not require any further parametric assumptions on the model.

However, we would like to proceed from this factorization to conditional independence, which is non-trivial. Specifically, we would like to know which variables are conditionally independent of others, given such an (assumed) factorization.

More notation: We write

\[ X \perp Y|Z \]

for \(X\) independent of \(Y\) given \(Z\).

We also use this notation for sets of random variables, and will bold them when it is necessary to emphasis this.

\[ \mathbf{X} \perp \mathbf{Y}|\mathbf{Z} \]

Questions:

1. \(X_2\perp X_3\)?
2. \(X_2\perp X_3|X_1\)?
3. \(X_2\perp X_3|X_4\)?

However, this product notation is not illuminating; we use a graph formalism. That’s where the DAGs come in.

This will proceed in 3 steps

• The graphs will describe conditional factorization relations.

• We do some work to construct from these relations some conditional independence relations, which may be read off the graph.

• From these relations plus a causal interpretation we derive rules for identification of causal relations

• If we get further than that, it will be all about coffee

Anyway, a joint distribution \(p(\mathbb{X})\) decomposes according to a directed graph \(G\) if we may factor it

\[ p(X_1,X_2,\dots,X_v)=\prod_{X=1}^v p(X_i|\operatorname{parents}(X_i)) \]

Uniqueness?

It would be tempting to suppose that a node is independent of its children given its parents or somesuch. But things are not quite so simple.

Questions:

• \(\text{Sprinkler}\perp \text{Rain}\)?
• \(\text{Sprinkler}\perp \text{Rain}|\text{Wet season}\)?
• \(\text{Sprinkler}\perp \text{Rain}|\text{Wet pavement}\)?
• \(\text{Sprinkler}\perp \text{Rain}|\text{Wet season}, \text{Wet pavement}\)?

To make precise statements about conditional independence relations we do more work.

We need new graph vocabulary and conditional independence vocabulary.

Axiomatic characterisation of conditional independence. (Pearl 2008; Steffen L. Lauritzen 1996).

Theorem: ((Pearl 2008)) For disjoint subsets \(\mathbf{W},\mathbf{X},\mathbf{Y},\mathbf{Z}\subseteq\mathbf{V}.\)

Then the relation \(\cdot\perp\cdot|\cdot\) satisfies the following relations:

\[\begin{aligned} \mathbf{X} \perp \mathbf{Z} |\mathbf{Y} & \Leftrightarrow & \mathbf{Z}\perp \mathbf{X} | \mathbf{Y} && \text{ Symmetry }&\\ \mathbf{X} \perp \mathbf{Y}\cup \mathbf{W} |\mathbf{Z} & \Rightarrow & \mathbf{X} \perp \mathbf{Y} \text{ and } \mathbf{X} \perp \mathbf{W} && \text{ Decomposition }&\\ \mathbf{X} \perp \mathbf{Y}\cup \mathbf{W} |\mathbf{Z} & \Rightarrow & \mathbf{X} \perp \mathbf{Y}|\mathbf{Z}\cup\mathbf{W} && \text{ Weak Union }&\\ \mathbf{X} \perp \mathbf{Y} |\mathbf{Z} \text{ and } \mathbf{X} \perp \mathbf{W}|\mathbf{Z}\cup \mathbf{Y} |\mathbf{Z} & \Rightarrow & \mathbf{X} \perp \mathbf{Y}\cup \mathbf{W}|\mathbf{Z} && \text{ Contraction }&\\ \mathbf{X} \perp \mathbf{Y} |\mathbf{Z}\cup \mathbf{W} \text{ and } \mathbf{X} \perp \mathbf{W} |\mathbf{Z}\cup \mathbf{Y} & \Rightarrow & \mathbf{X}\perp \mathbf{W}\cup\mathbf{Y} | \mathbf{Z} && \text{ Intersection } & (*)\\ \end{aligned}\]

(*) The Intersection axiom only holds for strictly positive distributions.

How can we relate this to the topology of the graph?

The flow of conditional information does not correspond exactly to the marginal factorization, but it relates. (mention UG connections?)

Definition: A set \(\mathbf{S}\) blocks a path \(\pi\) from X to Y in a DAG \(\mathcal{G}\) if either

1. There a node \(a\in\pi\) which is not a collider on \(\pi\) such that \(a\in\mathbf{S}\)

2. There a node \(b\in\pi\) which is a collider on \(\pi\) and \(\operatorname{descendants}(b)\cap\mathbf{S}=\emptyset\)

If a path is not blocked, it is active.

Definition: A set \(\mathbf{S}\) d-separates two subsets of nodes \(\mathbf{X},\mathbf{X}\subseteq\mathcal{G}\) if it blocks every path between any every pair of nodes \((A,B)\) such that \(A\in\mathbf{X},\, B\in\mathbf{Y}.\)

This looks ghastly and unintuitive, but we have to live with it because it is the shortest path to making simple statements about conditional independence DAGs without horrible circumlocutions, or starting from undirected graphs, which is tedious.

Theorem: (Pearl 2008; Steffen L. Lauritzen 1996). If the joint distribution of \(\mathbf{V}\) factorises according to the DAG \(\mathbf{G}\) then for two subsets of variables \(\mathbf{X}\perp\mathbf{Y}|\mathbf{S}\) iff \(\mathbf{S}\) d-separates \(\mathbf{X}\) and \(\mathbf{Y}\).

This puts us in a position to make non-awful, more intuitive statements about the conditional independence relationships that we may read off the DAG.

Corollary: The DAG Markov property.

\[ X \perp \operatorname{descendants}(X)^C|\operatorname{parents}(X) \]

Corollary: The DAG Markov blanket.

Define

\[ \operatorname{blanket}(X):= \operatorname{parents}(X)\cup \operatorname{children}(X)\cup \operatorname{coparents}(X) \]

Then

\[ X\perp \operatorname{blanket}(X)^C|\operatorname{blanket}(X) \]