📊Causal Inference Unit 8 – Difference-in-Differences & Panel Data Methods

What's the Big Idea?

• Difference-in-Differences (DiD) is a quasi-experimental design used to estimate the causal effect of a treatment or intervention on an outcome of interest
• Compares the change in outcomes over time between a treatment group and a control group
• Assumes that in the absence of the treatment, the average change in the outcome would have been the same for both the treatment and control groups (parallel trends assumption)
• Allows for the control of both observed and unobserved time-invariant confounding factors
• Can be extended to multiple time periods and multiple treatment groups using panel data methods
• Provides a powerful tool for causal inference when randomized controlled trials are not feasible or ethical
• Widely used in economics, public policy, and social sciences to evaluate the impact of policies, programs, and interventions

Key Concepts to Grasp

• Treatment group: The group that receives the intervention or is exposed to the policy change
• Control group: The group that does not receive the intervention and serves as a comparison
• Pre-treatment period: The time period before the intervention or policy change occurs
• Post-treatment period: The time period after the intervention or policy change occurs
• Parallel trends assumption: The key identifying assumption in DiD, which states that in the absence of the treatment, the average change in the outcome would have been the same for both the treatment and control groups
  • Implies that any differences in the outcome between the two groups can be attributed to the treatment
  • Can be assessed by examining pre-treatment trends in the outcome variable
• Time-invariant confounding factors: Factors that affect the outcome and are constant over time (individual fixed effects)
• Time-varying confounding factors: Factors that affect the outcome and change over time (time fixed effects)

The Math Behind It

• Let $Y_{it}$ be the outcome for individual $i$ at time $t$, $D_{it}$ be a binary treatment indicator, and $Post_t$ be a binary indicator for the post-treatment period
• The basic DiD estimator can be expressed as:

$\delta = (\bar{Y}_{Treatment, Post} - \bar{Y}_{Treatment, Pre}) - (\bar{Y}_{Control, Post} - \bar{Y}_{Control, Pre})$

• Can also be estimated using a regression framework:

$Y_{it} = \beta_0 + \beta_1 D_{it} + \beta_2 Post_t + \delta (D_{it} \times Post_t) + \varepsilon_{it}$

• $\delta$ captures the causal effect of the treatment (DiD estimate)
• $\beta_1$ captures the time-invariant differences between the treatment and control groups
• $\beta_2$ captures the time trend common to both groups
• Panel data methods extend the basic DiD framework by including individual fixed effects ($\alpha_i$) and time fixed effects ($\lambda_t$):

$Y_{it} = \alpha_i + \lambda_t + \delta D_{it} + \varepsilon_{it}$

• Controls for both time-invariant and time-varying confounding factors

Real-World Examples

• Card and Krueger (1994) used DiD to study the impact of a minimum wage increase on employment in the fast-food industry in New Jersey and Pennsylvania
  • Treatment group: Fast-food restaurants in New Jersey
  • Control group: Fast-food restaurants in Pennsylvania
  • Found no evidence of a negative impact on employment
• Duflo (2001) used DiD to evaluate the impact of a school construction program on education and labor market outcomes in Indonesia
  • Treatment group: Regions that received more schools
  • Control group: Regions that received fewer schools
  • Found positive effects on educational attainment and earnings
• Wolfers (2006) used DiD to analyze the impact of unilateral divorce laws on divorce rates in the United States
  • Treatment group: States that adopted unilateral divorce laws
  • Control group: States that did not adopt unilateral divorce laws
  • Found a short-term increase in divorce rates followed by a long-term decline

Common Pitfalls to Avoid

• Violation of the parallel trends assumption
  • If pre-treatment trends in the outcome variable differ between the treatment and control groups, the DiD estimate may be biased
  • Can be assessed by plotting pre-treatment trends or using placebo tests
• Endogenous treatment assignment
  • If the treatment is not randomly assigned and is correlated with unobserved factors that affect the outcome, the DiD estimate may be biased
  • Can be addressed by using instrumental variables or other methods to control for selection bias
• Anticipation effects
  • If individuals or entities change their behavior in anticipation of the treatment, the DiD estimate may capture both the effect of the treatment and the anticipation effect
  • Can be mitigated by excluding periods immediately before the treatment or using a dynamic DiD specification
• Spillover effects
  • If the treatment affects the control group through spillovers or general equilibrium effects, the DiD estimate may be biased
  • Can be addressed by choosing a control group that is less likely to be affected by spillovers or using a triple-differences design

How to Apply This Stuff

• Identify a research question that involves estimating the causal effect of a treatment or intervention on an outcome of interest
• Determine whether DiD is an appropriate method based on the availability of data and the plausibility of the parallel trends assumption
  • Requires data on the outcome variable for both the treatment and control groups before and after the treatment
  • Assess the parallel trends assumption by examining pre-treatment trends in the outcome variable
• Define the treatment and control groups, as well as the pre-treatment and post-treatment periods
• Estimate the DiD model using either the basic estimator or a regression framework with individual and time fixed effects
• Interpret the DiD estimate as the causal effect of the treatment on the outcome, assuming the parallel trends assumption holds
• Conduct robustness checks and sensitivity analyses to assess the validity of the DiD estimate
  • Test for parallel trends using placebo tests or event study designs
  • Control for potential confounding factors using additional covariates or matching methods
  • Explore heterogeneous treatment effects across subgroups or time periods

Advanced Topics

• Staggered adoption designs: When the treatment is adopted at different times across units, a staggered DiD design can be used to estimate the average treatment effect
  • Requires additional assumptions about the homogeneity of treatment effects across adoption cohorts
  • Can be estimated using a two-way fixed effects model with unit and time fixed effects
• Dynamic treatment effects: When the effect of the treatment varies over time, a dynamic DiD specification can be used to estimate the treatment effect at different points in time relative to the treatment
  • Allows for the investigation of anticipation effects, short-term effects, and long-term effects
  • Can be estimated by interacting the treatment indicator with indicators for different time periods relative to the treatment
• Synthetic control methods: When there are few treated units and many potential control units, synthetic control methods can be used to construct a more appropriate counterfactual
  • Involves creating a weighted average of control units that closely matches the pre-treatment characteristics of the treated unit
  • Provides a data-driven approach to selecting the control group and can improve the credibility of the DiD estimate

Wrapping It Up

• Difference-in-Differences is a powerful quasi-experimental design for estimating causal effects when randomized controlled trials are not feasible
• Relies on the parallel trends assumption, which states that in the absence of the treatment, the average change in the outcome would have been the same for both the treatment and control groups
• Can be extended to multiple time periods and multiple treatment groups using panel data methods with individual and time fixed effects
• Has been widely applied in economics, public policy, and social sciences to evaluate the impact of policies, programs, and interventions
• Requires careful consideration of potential pitfalls, such as violation of the parallel trends assumption, endogenous treatment assignment, anticipation effects, and spillover effects
• Advanced topics include staggered adoption designs, dynamic treatment effects, and synthetic control methods
• When applied appropriately, DiD can provide credible estimates of causal effects and contribute to evidence-based decision-making in various domains