[MUSIC] So far we discussed notion of independence for two events. However, when we have three or more events things becomes a little bit more difficult, and we have different notions of independence in this case. Let us discuss them. As usual, let us begin with example. Let us assume that we toss a coin three times. And we assume that it is a fair coin. So probability of head and tail is equal to one half. How many outcomes we have in this experiment? It is 2 to the power of 3, which equals to 8. Example of outcome is head, head, head or head, head, tail. Now, let us denote by Rj the result of j's tossing. It can be either head or tail. And let us introduce three events, A, it is event that R1 = R2, which means that the result of the first tossing is the same as the result of the second tossing. So we had head, head or tail, tail. B is R2 = R3. And C is R1 = R3. Now, let us test is it true that for example A and B are independent? To just solve as usual, they have to find the probability of A probability of B and probability of their intersection. By the way, what is probability of A? Let us find it. Probability of A equals to probability of the event that consists of the following outcomes. If A occurred then the result of first tossing is the same. So, we have head head tail, head head tail, tail tail, tail, tail, head. We have four outcomes, that corresponds to this event, out of eight outcomes. So their probability equals to 4/8 and it is 1/2. In a similar way, we can show that probability of B equals to 1/2 as well. Now, to test is it true that A and B are independent. They have to find probability on their intersection. If both events A and B occur, it means that the result of the first tossing is the same as the result of the second tossing. And the result of the second tossing is the same as the result of the third tossing. It means that we have only two outcomes. Either it is head head head. Or tail tail tail. The probability equals to 2/8, which is 1/4. Is it true that A and B are independent? We see that probability of intersection equals to product of probability of A and probability of B. It means that A and B are independent. In the same way, we can show that A and C are also independent, as well as B and C. I leave it as an exercise. However, we can consider a formula like this one but for three events. Let us check. Is it true that probability of intersection of all three events equals to probability of A times probability of B times the probability of C? To answer this question, let us consider this intersection. Note that if A and B, occur simultaneously, it means that R1 equals to R2 and R2 equals to R3. It follows automatically that R1 equals to R3, which is event C. It means that if we have all three events occur simultaneously, it is the same as only the first two events occur simultaneously. So we have that this thing is equal to A intersection with B. Now, we can say that this probability equals to probability of A intersection with B. And as we discussed before, this equals to probability of A times probability of B, and it is 1/4. Unfortunately, all three factors here are equal to 1/2. So this product is 1/8. We see that the answer to this question is no. We can interpret this result as a kind of dependence between C and both events A and B. In this example, we have so called pairwise independence, meaning that all these three events pairwise independent. A independent of C, B independent of C, A independent of B. But here, we can see there are all three runs as a system. We see some dependencies between them. In this case, this dependence can be understood as the effect that if A and B occur, then C occur probability 1. In this case we say that we don't have mutual independence between these three events. Let us give a definition. If we have three runs A1, A2 and A3. They are mutually independent. If first, they are pairwise independent, it means that probability of, Intersections like this equals to product of probabilities. However, this is not enough. We also need the following property. In this case if both properties hold, then these events are mutually independent. It means that there are no any dependents like between A3 and some combination of A1 and A2. If we have more events, then we have to test this kind of equality for any subset of our series of event. In marginalizing it is possible that we're interested in some variable we are going to predict. And we can use different features of our objects to predict it. And it is possible that we don't see any dependents between each of these features and the outcome. But if we unite all the features, we can use them to predict the outcome efficiently. This is the case which is similar to the case we discussed here. When there is no pairwise dependents, but only have dependents when we consider that all events as a system. [MUSIC]