[MUSIC] Hi, in this module, I'm going to talk about experimental designs in social science research. In the natural sciences, experiments have a long history, and they're relatively straightforward. In the natural sciences, generally, the approach to an experiment is, if we have a theory about a relationship between X and Y, where changes in X are hypothesized to lead to changes in Y. In the natural sciences, people design an experiment such that the subjects of the experiment could be particles, could be chemicals are isolated from the outside environment. So that in the system, the only thing left are really X and Y. Then the experimenter manipulates X to see if it leads to changes in Y. So, perhaps, if they're studying some chemical reaction, they may heat the chemical to see what happens to it. Now, since the system is isolated, any changes in Y that occur while X is being manipulated must be the result of those changes in X that were induced by the experimenter. Again, the system has been isolated from the outside, you don't have to worry about other factors intervening and affecting Y. Now, this is not so easy to achieve in the social sciences. In particular, it's very difficult to isolate our subjects from outside influences when we're conducting social science research. It's nearly impossible. Everything that we, as social scientists, want to study for interconnected people, families, firms, regions or countries. They all interact with each other in very complex ways, and they are subject to multiple interacting influences. And then, even if we can isolate our study subjects, another problem is that our subjects, again, they might be people, they could be firms, they could be countries, they have a history. And then they do things even when nothing else is going on around them. Because people, countries, firms, they're in a constant state of flux. They're always changing. People are always making choices, making decisions about what to do. So, if we just watch some people and then apply some stimulus. Perhaps give them some money to see what happens. And then we look at them later, conducting a pre and post-comparison to try to assess whether giving them money has any effect, we actually can't prove that it was the money that had the effect. Because it's possible that these people might have behaved in certain ways, anyway, or have undergone other changes without us doing anything at all. So to help clarify this difficulty with pre and post-comparisons as the basis for an experiment in social science, let's take a hypothetical example. The important thing to remember is that if we have a sample of people, represented here as the white figures, social entities may change on their own. Again, they decide to do things even in the, perhaps, absence of immediate external stimuli. So ,whether or not somebody gives you some money, you may decide one day to do something, right? So, even in the absence of any external stimuli, people may change, again, for no apparent reason. So, here, two people turned orange. We don't even know why they just did. So, this leads to the problem that if we are trying to do a pre-imposed comparison, where, again, we start out with some subjects, do something to them and then observe the change that takes place. We don't really know if when we apply a stimulus, and then see some change take place, in this case, two people have turned orange. We can't rule out the possibility that perhaps this change would have taken place anyway without us doing anything at all. To address these problems in social science research, we incorporate in our experimental designs what we refer to as control groups. We take inspiration from the approaches of life scientists who conduct research on living organisms, mollusks, mice, spiders. All of which experience change on their own, even in the absence of anything going on in the outside environment. And in the life sciences, the way that people address the difficulties of a simple pre and post-comparison is that they make use of control and treatment groups. These groups are supposed to be identical to each other at the outset of the study with no systematic differences. The control group is left alone, nothing is done to it. And then what we referred to is the treatment group is subjected to some sort of stimulus. In the case of the social science research, it might be the experiment's target is to give people money and see how they respond. Any change that occurs naturally, without the effect of the stimulus, should be apparent in both groups, control, and treatment. Any difference that emerges between the two groups over the course of the experiment must be the result of the stimulus that was applied to the treatment group. So let's walk-through a rather abstract examples. So if we start with a set of subjects, these white figures indicate people. We divide them into control and treatment group. And then, once they're divided, we apply the stimulus to the treatment group. Again, it might be giving them money to see what happens. I'd like to be part of an experiment like that. And then, when it's all over, we see what differences have emerged between the control and the treatment group. In the control group, two people turned orange. In the treatment group, a total of six people turned orange. So, through this comparison of the control and the treatment groups, which, again, were identical at the outset of the experiment, we see that there's a difference. And because they differ, we conclude that the treatment actually had some sort of effect. When we divide our subjects into control and treatment groups, there's a few things that we need to pay special attention to. Importantly, when we divide our subjects into these two groups, to really be sure that the treatment that we apply has an effect, and was responsible for the differences between the two groups. We have to be sure that the two groups were similar at the outset of the experiment, that they were not all ready different in some way. If the control and treatment groups are similar at the outset then any differences in their outcomes must be the result of the application of the treatment to the one group. And then the other group, not being affected, because they were left as the control. The ideal way of ensuring the similarity of the two groups, although it maybe counterintuitive, is what we call random assignment. Where if we start with a group of subjects that we're going to conduct our experiment on, we actually assign them at random to control or to treatment, essentially through coin flips or some other process. This actually ensures that the distributions within the two groups, the control group and the treatment group, the distributions of all of the variables that we can measure and, indeed, all of the variables that we can't measure, are going to be similar between the two groups. That there will not be one systematic bias according to which the control or the treatment group will be expected to be higher on some measure than on some other measure. And we can also achieve some further improvements in the similarity of the two groups if we match treatments and controls. This is a common approach, especially for smaller studies. Essentially, when people are dividing into control and treatment, we can pick out pairs of subjects that are identical on measured characteristics, perhaps subjects that are the same age, same sex, same level of education. And then, from each of these pairs, one is assigned to control, one is assigned to treatment. This maximizes the similarity between control and treatment on the variables that we can observe. And this reduces the chances that the effect of our treatment, or the apparent effect of our treatment, is actually the result of chance variation in terms of the preexisting differences between the two groups. So there are opportunities to conduct experiments in social science. Now when we back away from social science and we think about research on human beings more broadly, there are lots of examples of experiments in clinical trials, that is medicine. People want to check whether a drug is effective or not, they follow the process that I just described. Divide the subjects into control treatment, give one of them the new drug and the other one is treated as a control group and left alone. They are watched for the emergence of differences. Psychologists also routinely conduct experiments in very controlled settings. Opportunities to conduct experiments in the social sciences, however, are actually quite rare. Experiments are expensive and difficult. And indeed, many of the most interesting experiments would probably be unethical. For example, if we wanted to study the effects of unemployment on health, we couldn't simply recruit a large number of people and then randomly select a subset of those people and cause them to lose their jobs. And then see what happens, and then compare them with the people who kept their jobs, it would be unethical. And also, the settings in which we actually have the opportunity to conduct the experiments may be so isolated or unusual that it may be hard to generalize from the results of those experiments to the larger social processes in which we're interested. That said, social science researchers do press ahead and conduct experiments in certain settings. Let me talk about some examples, one of the most common examples of experimental research in social science, is actually in education. So education, as you might imagine, it's actually fairly straightforward to think about dividing clusters of people. That is, classrooms or schools into control and treatment group. And then doing something different in the treatment group and then comparing the outcomes with the control group. So in educational research, for example, on the effects of the introduction of a new curriculum, researchers might select 20 schools. And then randomly select ten of those schools and introduce a new curriculum with perhaps training for the teachers. And the other ten schools, they're left alone. The whole set of schools is followed over time. After a year, maybe two years, the outcomes of the students are measured and compared between control and treatment. The treatment, again, being in schools where the new curriculum has been introduced. If a difference emerges between the two sets of schools, that's probably going to be the result of the treatment, the introduction of the new curriculum. Another area in which increasingly people are conducting experiments is in the effects of certain policy interventions. This is especially the case in less developed countries. A common approach in this situation is that researchers who are hypothesizing that perhaps some policy intervention, perhaps the introduction of a new nutritional assistance program for expectant mothers. Or the introduction of new rules for allowing time off under certain circumstances. Where they're trying to figure out this has an effect or not, they may select, again, a selection of communities or villages. Maybe 20, 30, 50, if they have the resources. Randomly divide them in the two groups. And then in the treatment group, the new policy is introduced. Control group, again, is left alone. Then these two sets of villages or communities are followed over time. A year or two later, they're compared to see if any differences have emerged. If differences emerge, then the result is likely to be the product of the treatment. Finally, there are laboratory experiments in certain areas of the social sciences, in particular political science and economics. People conduct laboratory experiments in very controlled settings to test the workings on auction mechanisms, voting behaviors and so forth. So, overall, they're indeed opportunities to conduct experiments in social science. Nevertheless, in a lot of other settings, we really can't conduct an experiment. And we have to rely on observational studies. And, in the following modules, I'm going to talk about how, with observational studies, we try to prove cause and effect.
