The famous Monty Hall problem is named for the host of the long running US television game show Let's Make A Deal. A valuable prize was hidden behind one of three identical doors. If my doors don't look identical, excuse me. Imagine they are identical. A contestant can win the prize if they pick the correct door. And at the beginning, the contestant is told pick any of the doors and just for convenience, we'll say that the contestant picks door number one. With no basis for choosing between the doors, it's reasonable to assume that the probability of the prize being behind any one door is simply one-third. So the contestant has a one in three chance of winning. Monty Hall then opens one of the other doors, so he eliminates it as a possibility. He then offers the contestant the option of switching doors before he reveals the location of the price. So the mathematical question known as the Monty Hall problem is should the contestant switch doors? Should they stay with door number or should they go to what we're labeling door number two? Or to put the problem a little bit more mathematically, what is the new probability distribution now that door three has been eliminated that the prize is behind each of doors one and two? And is the probability now greater for one door than the other? The common intuition is that the probability would now be split 50, 50 between the 2 doors. However, this is not correct. In fact, the probability that the prize is behind door one remains at one-third just like it was when that door was first chosen. And all the probability that was formerly associated with door three migrates over to door two, so that the probability of the prize being behind door two is now two thirds. So the correct answer is that the contestant should switch doors, doubling their chance of winning. The key to understanding how this could be is understanding that Monty Hall is not opening one of the three doors at random. If one of the three doors swung open at random say, because of an earthquake and that just happened to be door number three. Then at that point, there might be a 50, 50 chance that the prize was behind either of the remaining 2 doors. But instead, Monty Hall is very constrained in what he can do. He cannot open the door that has been chosen by the contestant and you cannot open the door that's got the valuable prize behind it, which might be that door, might be that door. But in any case, it cannot be the door that he opened. So Monty Hall is making a decision to open a door in full knowledge of where the prize is. One-third of the time, the prize will be behind the door already chosen, door number one. And in that case, Monty Hall can just randomly choose which of the other two doors to open, but two-thirds of the time the prize is behind one of the doors the contestant did not choose. In those cases, Monty Hall must open the other door. He has no choice. So two-thirds of the time Monty is signaling where the prize is, we just don't know which two-thirds. So that's a problem that I always thought was really, really interesting. And it's fine as far as it goes and we'll look at some variations of it, like when there are more than three doors or you open more than one door and so on. We'll look at that later on, but the question that always bothered, or I really wanted to know the answer to is okay, I accept that we're getting communicated some information by Monty Hall when he opens that door, but I'd like to know how much information exactly. What percentage of the information required for total certainty is Monty providing? Monty could just tell us where the prize is and then that would be 100% of the information and we'd be done. But instead, he's eliminating some uncertainty, but how much? 10%? 20%? 5%? So we can actually find the answer by calculating the entropy of the original probability distribution and then comparing it to the entropy of the probability distribution after. So we'll just work this simple example, so just work this with me. Initially, we have probabilities of one-third, one-third, one-third and our entropy. So let's say, this is our original probability distribution. Our entropy is going to be one-third log2 of 3 + one-third log2 of 3 + one-third log2 of 3, which is gonna be equal to log2 of 3. So that is H(X). So if we label as Y, what Monty told us? H(X given Y) is the entropy of the new probability distribution one-third, two-thirds, which is equal to one third log2 of 3 + two-thirds log2 of three-halves. This happens to be 1.585 bits and this happens to be 0.918 bits. So, using our mutual information formula. Equals 1.585-0.918=0.667. So Monty has provided us with 0.667 bits of information by opening one door. If you'd like to know the percentage information gain or PIG., that would be 0.667/1.585=0.421. So Monty Hall has provided us with 42.1% of all the information we would need to have certainty about where the prize is.