How can we know if the differences in wages between men and women are caused by discrimination or differences in background characteristics? In this PhD-level course we look at causal effects as opposed to spurious relationships. We will discuss how they can be identified in the social sciences using quantitative data, and describe how this can help us understand social mechanisms.
Lecture 5 - Difference in Difference
Loading...
Do curso por University of Copenhagen
Measuring Causal Effects in the Social Sciences
101 classificações
Na lição
Difference in Difference
The final module of the course deals with the difference in difference. We hope you enjoyed the course and have learned something that you can use in your future work and research.
Conheça os instrutores
Anders HolmProfessor
Department of Sociology
[MUSIC]
In this final lecture, I'll go through an alternative method to
infer causal effects in the absence of data from a randomized control trial
or in lottery like variation in relocation to the dependent variables unavailable.
This alternative method is called difference in difference and
generally requires longitudinal data.
All this will be explained in this lecture.
The idea behind the difference in difference method is ac very simple,
and it's illustrated on the figure in the slide.
Imagine, as with a randomized control trial,
that our independent variable, referred to as x in the previous lectures,
only has two values indicating whether the individual is treated or not treated.
We have data on relevant outcome for a group of treated individuals, before and
after treatment, and similarly for control group.
Then you can estimate the average difference in outcomes before treatment
across the two groups.
Imagine that we measure the amount of unemployment experienced for
some individuals.
Some of them are selected to participate in a labor
market training program and some are not.
If the two groups have experienced different amounts of unemployment
before treatment this is an indication of selection into treatment.
Imagine for instance, that the average amount of unemployment is lower for
those selected to participate in the subsequent training program
compared to those selected through the control group.
This could be because individuals who volunteered for training, or
more motivated, or has better skills than those who do not choose to sign up.
Obviously, when comparing average unemployment between three of
the controls after treatment,
do not in this case estimate the causal effect from the training program,
because differences also include pre treatment differences.
In order to obtain a better estimate of the causal effect of the training program
we'd like to deduce the pre program differences from
the post program differences.
Before inferring causal effect of the training program.
Therefore in order to calculate the causal effect of the training program
we estimate the difference in difference across treated and controlled.
Hence the name of the method, the difference and difference method.
This is all there is to it.
The following slides will just formalize the method in terms of
regression model and finally offer some empirical samples.
In what follows, we use the capital letter T to indicate treatment.
However, you should just think of it as x, the independent variable that might be
correlated with the error term in a regression, on outcomes and treated and controls.
It is this correlation that
induced pretreatment differences between treatment,
treated and controlled.
We now write a regression model, with two independent variables.
The instrument indicator capital T and a time period indicator, lower-case t.
Both has two values.
The treatment indicator is one if the individuals is in the treatment group and
zero otherwise.
And the time indicator has a value zero in the pre program period and
one in the post program period.
We now postulate that the interaction term between the treatment
indicator and the period indicator,
is equal to the causal effect of the treatment, the difference in difference.
The regression coefficient, for
the treatment indicator captures any preprogrammed differences
and the period indicator captures any shared differences across the pre and
post program period.
In the case of unemployment we could imaging that if there's is
a period of economic recovery between two time periods,
both the treatments and
control groups might experience a decrease in unemployment across two periods.
This would be captured by the pure indicator.
The following slides will show that
this regression model will actually estimate the difference in differences.
First we have that both the control and
the treatment group can experience a change across periods
for instance due to changes in economic conditions in the unemployment example.
That is, the difference in the average outcome across periods could be non-zero.
We also have that the change across periods in groups could be non-zero.
Therefore, these are the changes that could occur in the data, and
that to be captured accordingly by the model.
So now we investigate how our proposed regression model
captured those differences.
First we state our regression model again to remember the difference
of coefficients.
Next we see how the model captures
the difference in average outcomes in the pre-program period,
that is when the time indicator is zero and the treatment indicator varies.
Concerning the appropriate values of the treatment indicator,
which varies across groups
and the time indicator, which is zero, as we only look at the pre-program period
into the regression model, yields the regression model equivalent of
the program, preprogram average differences.
In addition, as can be seen from the slide,
this is equal to the regression coefficient for the treatment indicator.
This regression coefficient captures preprogrammed differences across groups.
If it's different from zero, there are selection into the treatment.
That is, those who are selected or select themselves into treatment, on average
different from those in the control group.
Had data been generated from a successful randomized control trial,
this coefficient should be zero.
Now I want to see how the model captures change across time for
the treatment group.
This change is due to both share changes in the environment.
For instance, economic conditions in the unemployment example as well
as any potential effect of the treatment.
We write the regression model
once again to keep track of all the coefficients,
again inserting the appropriate values of the treatment indicator
(which is now one
As you only look at the treatment group) and the time indicator (which is one in
the last period and zero in the first period)
we get that the average difference across time for the treatment group is
a coefficient for the time indicator plus the indicator for the interaction term.
For the third time, we restate our aggression model in order to
see how the model kept its changes across time for the control group.
Again, inserting the appropriate values of the treatment indicator,
which is now zero
as we only look at the control group and the time indicator, which is one (in
the last period and zero in the first period)
we get the average difference across time for
the control group as a coefficient for the time indicator.
Now finally we want to see how the model captures differences in
differences between the control- and treatment group across time.
The data is the average difference across time for
the treatment group minus the average difference for the control group.
For the regression model, we found from the previous slides that the difference in
the average outcome for the treatment group, was a coefficient for
time indicator plus the coefficient for the interaction term
while different for
the control group, was just the coefficient for the time indicator.
And the difference between the two groups is just the interaction term.
Therefore, the interaction term captures the difference in time across the two
groups, or the differences in differences.
It seems as a difference, the difference model is a magic tool.
It removes all selection effects
and therefore allows interpretation of the interaction term as
the average causal effect of the treatment, without the need for
randomized data or instrumental variables.
All we need is average outcomes for individuals in the treatment group and
control group before and after the treatment.
We do not even need data for the same persons across time.
Therefore, the data for the treatment and
control groups does not need to be the same across periods.
This is neat, if for
instance, you want to study the effect of a smoking policy in one state or region,
all you need is health outcomes in the region and
a control group in a neighbouring region, for instance
before and after the implementation of this modern policy.
Any difference and difference across the two regions are interpreted as a causal
effect of this smoking policy.
This might sound too good to be true, and it is.
In all derivations of the difference and
differences from an aggression model, the average value of the residual term E
is the same across all four groups, treated in both periods and
controlled in both periods.
If this is true, the difference in difference estimator is
estimating the average causal effect of the treatment.
If that's not true, however, then obviously the difference in
difference estimator is estimating something else
and the interpretation of the interaction terms as causal effect may be
biased interpretations or estimates.
Therefore, the interesting question is when it
would be likely that it's not true
and the average zero term is not the same across groups.
It turned out that the interesting question is when the change in
the error term
is not the same across time for the two groups.
That is, if the treatment group would have had counterfactual change across time
compared to the control group
in the case of the smoking policy sample.
If other policies that related to health were invoked in both regions,
then changes in health outcomes could be due to a wide range of factors
and not just the invoking of the smoking policy.
Hence the validity of the difference in difference approach to causal inference is
therefore depending on how comparable change is
across groups in the absence of the treatment.
As an example of the application of the difference in difference approach,
we now turn to this thigh example again.
We use the difference in difference approach to see if
attending a small class
in first grade, improved math achievement. We saw earlier that attending a small
class in Kindergarten had a positive affect on math achievement.
Here we rely on the randomized control trial to confer causality.
Now I want you to use a different approach to see if a small class in first grade
also improved math achievement.
We cannot completely rely on randomization to infer the effect of
a small class in first grade.
As some non-random drop-offs and
crossovers took place between kindergarten and first grade.
We now run from regressions in kindergarten and
first grade math outcomes.
In our first regression we only include a constant term.
This is just to get a feel for the data.
We find a constant term of 566.
This means that an average student in the two grades score 566
on the math outcome scale.
Now we rerun the regression model.
Including a time indicator,
grade equal to one in the first grade and zero in kindergarten.
A treatment indicator, small equal to one if the student attends a small class in
first grade, and zero otherwise
and an interaction term, called inter, between the treatment indicator and
time indicator.
From the estimates we find,
that on average students in small classes scored 8.7 points higher in
math achievement after kindergarten, our pre-treatment period.
That is students in a small class in first grade, on average better in math
than students in ordinary classes in first grade, when they enter first grade.
This is because students in small classes in first grade
typically also attended a small class in kindergarten
and this had a costly effect on the achievement in kindergarten.
Therefore the two groups are not equal before treatment.
A small class in first grade.
We also find that an average student improved 44 points
on the math scale between kindergarten and first grade.
We finally find that the interaction term between treatment and
the time indicator is negative.
Hence students who are in a small class have a lower gain in
math achievement during first grade than students in ordinary classes.
The difference, is however small.
But never the less the difference and difference approach indicate a negative
albeit small effect of attending a small class in first grade.
This seems somewhat counterintuitive.
It could be expected that the effect was zero, but
it seems unlikely that it should be negative.
However, if you remember the Hawthorn effect from the last lecture
it could be thought that students and
teachers in small classes in kindergarten showed an extra effort
because they thought this was appropriate as
they had been selected into the treatment.
Therefore, the negative effect in first grade, could be a kind of
burnout effect that teachers and students in small classes in first grade
were resting on the previous huge effort in math achievements.
And this effect was apparently not due
to the small class itself, but rather an effect of being assigned to the treatment.
This concludes the course on inferring cause and effects in the social sciences.
Hopefully you've enjoyed the course and appreciated
that its indeed possible to conduct field work that enables estimation of
cause and effect.
But that you also have realized that its far from an easy task, and
that many starters and data sets are not suited to infer causality.
[MUSIC]
Explore nosso catálogo
Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.
O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.
© 2019 Coursera Inc. Todos os direitos reservados.
