Hi, my name is Brian Caffo and this is mathematical bio statistics boot camp, lecture seven on Fisher's exact test. So in this lecture we're going to talk about Fisher's exact test. We're going to talk about the hypergeometric distribution which plays a central role in Fisher's exact test. And we'll talk about some practical implementations and then we'll talk about how you can execute Fisher's exact test using Monte Carlo. Fisher's exact test is a historically very famous test, and it's going to be of the, you know, one of the first instances where we'll able to test. Equality of say binomial proportions using a formal exact test rather than relying on asymptotics. So what does exact mean? So Fisher's exact test is exact because it guarantees the alpha rate. So when you do a asymptotic test and you use the nominal type one error rate, say a 5%. If you say for example calculate a 95% confidence interval for the risk difference and declare the differences in the proportions as as being significant if the confidence interval for the difference doesn't include zero, that's a nice valid testing procedure. However, that doesn't guarantee you a 5% err rate, it only guarantees you a 5% limit as the sample sizes go to infinity. Fisher's exact test and contrast guarantees you the 5% limit provided the IID assumptions are met for each of the two groups. And the,the background on Fisher's exact test is that the um,the famous example is that of the so called Lady tasting tea. And, in this case, Fisher was at a party and there was a wager about whether or not a lady at the party could determine whether the cream had been put in place first or the milk had been put in place first into her, her tea. And Fisher devised a blinded experiment where he put tea in first for a couple of cups and put the milk in first for a couple of, of examples and then had her declare which she thought was the case. So you know here is a kind of more um,um medically oriented version of that so here we have chemically toxicant with ten mice and we treated um,um, ten of them I'm sorry, we treated five of them with the toxicant, and five with the control. And then here we have the counts of, in each group, of tumours versus in each, group of treated versus control, what number had tumours. So here just looking at this four exposed to the toxicant received tumors, two controls received tumors out of five in both cases. So there's maybe some amount of indication that there's a difference that the toxicant may have some association with the tumors. But we'd like to test that formally. there's a lot of ways you, there's actually a surprising number of ways you can, wind up at Fisher's exact test. and so, we're going to go through a particular development. So, the way we're going to develop Fisher's exact test, is we're going to assume that we have two binomials, and we want to test equality of the proportions. When Fisher originally developed this test, it was quite interesting. He, so in our case were going to margins these in the five and the five fix, so we have five treated and five controlled and we are going to model the probability of random mouse from this population of the treated mice has a tumor as being having binomial probably p1 and simialriy p2 for the controls. Fisher thought about this problem differently, he said I know that I random, let's just assume this was the tea tasting experiment and he had randomized five cups to have the tea put in first and five cups to have the milk put in first, and he says I know that these margins should be fixed. Then, then it would have made sense if the, given that the, the, the lady guessing would also know that this margin was fixed. She would, it would have made sense for her to fix the second margin, instead of six and four, she would force five and five. So we wanted to create a procedure that would force both margins. Right? And it turns out the Fisher's exact test constrains the margins, right? It looks at all tables matching the margins, and, and, that, that, that gave him the motivation for designing the test this way. So he looked at so-called hyper-geometric distribution, which is, is a way in which we can characterize this table. in, in, in terms of having fixed margins. Now when we actually implement the test now we don't have to have five and five on both margins the way Fisher was thinking about it. But he was analyzing a particular experiment with in a particular way but it, it, it the, the, the result of Fisher's exact test is constraining both of these margins and then looking at what tables satisfy the margins. And we'll go through the formal mathematical development of the way in which we're thinking about Fischer's exact test. There's other ways and maybe I'll ellude to some more of them um,later on, okay. Okay on this slide so imagine that we want to test that p1 equals p2 where p1 is the probability that a mouse a treated mouse, had, has a tumor, and then p2 is the probability the control mouse has a tumor. And so they, we're going to want the null hypothesis that these are equal which let's just call the common proportion p. and you know just as a matter of practicality, we can't use a normal distribution or a chi square test because the sample size is small. But also, we don't have a specific value of p which is taken care of in the chi square and the Z test. They figure out a way to do that using the asymptotics, but we want to use small sample distribution. But making the small sample distribution using a small sample distribution is hard because we don't actually know this this probability p. Okay so one way wha, one way to think about Fisher's exact what's testing is imagine if we were to create the observed data. List out the observed data as the individual data points. So mouse one was treated and got a tumor, mouse two was treated and got a tumor, mouse three was treated and got a tumor, mouse four was treated and got a tumor. Mouse five was treated and had no tumor, six was a control and got a tumor, and so on,okay. now, one, so, so what we, what I'm going to do right now is go through another way to kind of develop Fisher's exact test. and it winds up with the same test. All these different developments yield the exact same test. so, in this case imagine if the treatment and control were randomised. And we wanted to explicitly use the randomization process, in analyzing the data. So treatment and control are were randomized. Treatment and control status were randomized. Well then, if the null hypothesis is true, And it should be exchangeable for any mouse, whether or not it got a tumor. As to whether or not it was from the treated group or the control group, right? The, the, there should be nothing outrageous about the particular collection of treated and control mice, that we see how they line up with the tumor and non-tumor status that we saw. So what we could do is take this top row and permute the treat, treatment and control labels. Right? Permute the Ts and the Cs. And we see the second row here is one example of a permutation Okay, and the result of that is that the total number of treated, and the total number of controls remained fix, and the total number to tumors and total numbers of non tumors also remained fixed. And thats exactly, maintaining the margins of the, of the table. This seems like kind of a reasonable null distribution to investigate right, this that in other words that treatment in control statistics is exchangeable relative to tumor status so we'll look at some test statistic relative to this distribution. And, the consequence of this is, every time we permute treatment and control labels, if we were to reform the little two by two table we had, it would have the same margins, it would have five and five on the row margins and six and four on the column margins. And this is exactly one way to develop the null distribution for Fisher's exact test. Test and this is an interesting way to develop it because it explicitly uses the idea of randomization. There's more than one way to think about it. We can think about this rather than explicitly using the randomization, we can say well maybe treatment and control status weren't randomized. Maybe just the first five mouse mice got the treatment, the latter five mouse mice got the control. And we believe that there is no ordering effect, like there is no light in the lab for the first five mice, causing tumors; and not for the controls. So in that case we might think of not the relabeling as a process of re-implementing the randomization scheme, but instead it's a process of, well we think, if the null hypothesis is true, relative to tumor status, treatment and control status are a bunch of exchangeable, permutable labels. And you could do the same procedure to coming up with the null distribution and that's another way to think of it. Yet another way to think about it is the development we going to go through next which is a Mathematical treatment. But it is interesting that these very different methods of thinking about the problem results in the same task and you see this very often in Statistics where you wind up with procedurally the same approach but that the interpretation differs quite a bit. I would argue that explicitly using the randomization saying well, this treatment and control was randomized and so we're going to explicitly work that into our analysis. It's a very fundamentally different process than what we're about to do. Which is to assume that the data are binomally distributed, I impose a model, a super population model for the mice, and to work with that lined up with the same procedure, but the interpretation I think is vastly different, and I hope you think so too after taking this class.
