Judea Pearl, Dan Geiger, and Thomas Verma, computer scientists at UCLA working on the problem of storing and processing uncertain information efficiently in artificially intelligent agents, solved this mathematical problem in the mid 1980s. Pearl and his colleagues realized that uncertain information could be stored much more efficiently by taking advantage of conditional independence, and they used directed acyclic graphs (graphs with no loops from a variable back to itself) to encode probabilities and the conditional independence relations among them. D-separation was the algorithm they invented to compute all the conditional independence relations entailed by their graphs (see Pearl, 1988). Peter Spirtes, Clark Glymour, and Richard Scheines, working on the problem of causal inference at the Philosopy Department at Carnegie Mellon University in the late 1980s and early 1990s, connected the artificial intelligence work of Pearl and his colleagues to the problem of testing and discovering causal structure in behavioral sciences (see Spirtes, Glymour, and Scheines, 1993). The work didn’t stop there, however. Pearl and his colleagues proved many more interesting results about graphical models, what they entail, and algorithms to discover them (see Judea Pearl’s home page). In 1994, Spirtes proved that d-separation correctly computes the conditional independence relations entailed by cyclic directed graphs interepred as linear statistical models (Spirtes, 1994), and in the same year Richardson (1994) developed an efficient procedure to determine when two linear models, cyclic or not, are d-separation equivalent. In 1996, Pearl proved that d-separation correctly encodes the independencies entailed by directed graphs with or without cycles in a special class of discrete causal models (Pearl, 1996). Also in 1996, Spirtes Richardson, Meek, Scheines, and Glymour (1996) proved that d-separation works for linear statistical models with correlated errors. So it should be obvious that d-separation is a central idea in the theory of graphical causal models. In the rest of this module, we try to explain the ideas behind the definition and then give the definition formally. At the end of the module you can run a few Java applets which provide interactive tutorials for these ideas.

So in short, d-separation is a criterion for deciding, from a given a causal graph, whether a set X of variables is independent of another set Y, given a third set Z. To illustrate the concept, I will follow Judea Pearl’s 3 rule description (http://bayes.cs.ucla.edu/BOOK-2K/d-sep.html).

Rule 1: Unconditional Separation
Two nodes are d-connected if there is an unblocked path between them. By path we mean edges without regard to directionality and by unblocked we mean that there are no head-to-head arrows on some path. Here’s a picture:

In the figure above, there is one collider at t, x-r-s-t is unblocked, and so x and t are d-connected. The path t-u-v-y is unblocked, so t and y are also d-connected. So too are all the pairs, x-r, x-s, r-s, t-u, etc. However, x and y are not d-connected since we can’t trace a path without hitting the collider; hence they are d-separated. So too are x-u, x-v, r-u, etc.

Rule 2: Blocking by Conditioning
Two nodes x and y are d-connected, conditioned on a set Z, if there is a collider-free path between x and y that traverses no member of Z. If no such path exists, we say that x and y are d-separated by Z; we also say then that every path between x and y is “blocked” by Z. Here’s a picture:

Let Z be the set {r,v}. By Rule 2, x and y are d-separated by Z, along with x-s, u-y, s-u, etc. The path x-r-s is blocked by Z, along with u-v-y and s-t-u. Only s-t and u-t remain d-connected conditioned on Z. The path s-t-u is also blocked Z since t is a collider, and is blocked by Rule 1.

Rule 3: Conditioning on Colliders
If a collider is a member of the conditioning set Z, or has a descendant in Z, then it no longer blocks any path that traces this collider. This is called the common effect of two independent causes explaining away one. Pearl gave an example with two independent causes of your car refusing to start: having no gas and having a dead battery (both arrows point to “car won’t start”.
Telling you that the battery is charged tells you nothing about whether there is gas, but telling you that the battery is charged after I have told you that the car won’t start tells me that the gas tank must be empty. So independent causes are made dependent by conditioning on a common effect, which in the directed graph representing the causal structure is the same as conditioning on a collider. (Text from Scheines).

Let’s look at a picture for rule 3:

Let Z be the set {r, p}. By Rule 3 s and y are d-connected by Z: The collider at t has a descendant (p) in Z, which unblocks the path s-t-u-v-y. However, x and u are still d-separated by Z; the linkage at t is unblocked but the one at r is blocked by Rule 2 (since r is in Z).

So that’s d-separation in a nutshell. I recommend Pearl’s and Scheines’ sites.

Slides (ppt)

Causality Seminar Slides (ppt)

Bayesian Network Example (ppt)

Bayesian Networks in Use (ppt)

Talk 5 (pdf) – Long talk on parameter learning–we got through page 8.

Talk 6 (pdf) – Short talk finishing the previous talk and completing parameter learning.

Talk 6 Handout (pdf) – This handout summarizes many of the definitions, theorems and lemmas from parameter learning. The reference numbers and notation correspond to Neapolitan’s text.

Talk 7 (ppt) – This presentation gave an overview of structure learning via local search. It is lacking in examples and explicit theory, but examples will be shown more as we come to causality, and the theory is based on the theory presented in parameter learning.

Hopefully after the semester I’ll get one big Latex version of all of these.

Next week I will give it another shot.  While my background knowledge must be extensive and airtight, my talk should be just an overview of important concepts.  I look forward very much to posting notes from the talk that don’t suck so much.

So far in our lab seminar series (http://sysbio.fulton.asu.edu) I’ve covered the background in probability theory and in graph theory (Markov chains/blankets/equivalence, faithfulness, d-separation, etc.), and have begun discussing how to actually learn Bayesian networks from data. The last thing discussed was an intro on leaning posterior probability distributions, which will lead into learning network topology, and finally learning both.

I am posting my slides/notes from the four talks I’ve given, but take them only as an illustration of my progress and not as a good resource. For good resources, check out David Heckerman, Nir Friedman, Dana Pe’er, Eran Segal, Marco Ramoni, Andrew Moore, Jose Pena, and Daphne Koller, Richard Neapolitan, and of course the seminal text by Judea Pearl. All of those are who I am learning from.

Talk 1 (ppt) (horrible introduction, I didn’t know anything)

Talk 2 (ppt) (getting there, decent probability discussion)

Talk 3 (ppt) (getting better, decent graph discussion)

Talk 4 (pdf) (not so great, mostly review, notes only, no slides used)