So for example, you could read the first statement as saying, for every assignment
little X and little Y to the variables x and y, we have that p of the event x
comma y is equal to p of x * p of y. So you can think of it as a conjunction
of lots and lots of independent statements of the form [SOUND] over here.
That's the first interpretation. The second interpretation is as an
expression over factors, that is, this one tells me that the factor over here
which is the joint distribution over XY is actually a product of two lower
dimensional factors one which a factor whose scope is X, and one is a factor
whose scope is Y. These are all equivalent definitions but
each of them has a slightly different intuition so it's useful to recognize all
of them. So let's think of examples of
independence, here a, A fragment of, our student network, it
has, three rend variabled intelligence, difficulty and course grade, and this is
a, probability distribution whose, who, who, that has a scope over three
variables, but we can go ahead and marginalize that, to get a probability
distribution over the scope, which is a factor over the scope ID as it happens,
this is the marginal distribution which you can confirm for yourselfs by just
adding up, the appropriate entries, so just as a reminder to get I0, D0 we're
going to add up this one. This one, and that one.
And that's going to give us this factor. And it's not difficult to test that.
If we then go ahead and marginalize p of I, d to get p of I and p of d.
That p of I, d is the product of these two factors.
Here is a good example of a distribution that satisfies an independence property.
And here is the graphical model and when you look at it you can see that there's
no, direct connections between I and V, and, well, and we'll talk later bout how
that tells us that there is no the detour action independence of this distribution.
Now independence by itself is not a particularly powerful notion because it
happens only very rarely. That is only in very few cases are you
going to have prob, random variables that are truly independent of each other, at
least few interesting cases, you can always construct examples.
So now we're going to define a much broader notion of much greater usefulness
which is the notion of conditional independence.
Conditional independence which applies equally well to random variables or to
set of random variables is written like this so here we have once again the P
satisfies. Here we have, again, the independent
sign, but here we have a conditioning sign.
And this is red as p is p satisfies x is independent of y given z,
okay? And once again, we have three identical,
not identical, sorry. Three equivalent definitions of this of
this property. The first says that probability of X, Y
given Z is equal to the product of P of X given Z times the probability of Y given
Z. Once again, you can view this as a
universally quantified statement over all possible values of X, Y and Z or as a
product of factors. Definition number two, is a definition of
information flow given Z, Y gives me no additional information that changes my
probability in X, or, given Z, X gives me no additional information that changes my
probability in Y. Once again, this is a, this is, you can
view this as an expression involving factors.
Notice that this is very analagous to the definitions that we had to just plain old
independence, Z effectively never moves, it always sits there on the right hand
side of the conditioning bar and never moves.
And so if you find yourself having a hard time remembering conditional independence
just remember that the thing your conditioning on just sits there on the
right hand side of the conditioning bar, all the time.