1

Paper

The Immigration Issue and the 2010 House Elections: A Research Design Monogan, Jamie Uploaded 11-02-2010 Keywords causal inference propensity score elections immigration Abstract This paper proposes a research design for evaluating the effect of Republican candidates' immigration stances on House election outcomes. It develops a measure of immigration stance which is based on the text of each candidate's issue statement. With this as the treatment, propensities to support a harsh line on immigration are calculated for each candidate based on a variety of covariates that also may influence election outcomes. In this way, a research design is developed before election outcomes are observed. Thus, this project clearly reflects the advice of Rubin, who argues that the research design ought to be set before the outcome is even observed.

2

Paper

Covariate Balancing Propensity Score Imai, Kosuke Ratkovic, Marc Uploaded 07-13-2012 Keywords causal inference instrumental variables inverse propensity score weighting marginal structural models observational studies propensity score matching randomized experiments Abstract The propensity score plays a central role in a variety of settings for causal inference. In particular, matching and weighting methods based on the estimated propensity score have become increasingly common in observational studies. Despite their popularity and theoretical appeal, the main practical difficulty of these methods is that the propensity score must be estimated. Researchers have found that slight misspecification of the propensity score model can result in substantial bias of estimated treatment effects. In this paper, we introduce covariate balancing propensity score (CBPS) estimation, which simultaneously optimizes the covariate balance and the prediction of treatment assignment. We exploit the dual characteristics of the propensity score as a covariate balancing score and the conditional probability of treatment assignment and estimate the CBPS within the generalized method of moments or empirical likelihood framework. We find that the CBPS dramatically improves the poor empirical performance of propensity score matching and weighting methods reported in the literature. We also show that the CBPS can be extended to a number of other important settings, including the estimation of generalized propensity score for non-binary treatments, causal inference in longitudinal settings, and the generalization of experimental and instrumental variable estimates to a target population.

3

Paper

On the Use of Linear Fixed Eects Regression Models for Causal Inference Imai, Kosuke Kim, In Song Uploaded 07-23-2012 Keywords difference-in-differences first difference matching observational data panel data propensity score randomized experiments stratification Abstract Linear fixed effects regression models are a primary workhorse for causal inference among applied researchers. And yet, it has been shown that even when the treatment is exogenous within each unit, the linear regression models with unit-specific fixed effects may not consistently estimate the average treatment effect. In this paper, we offer a simple solution. Specifically, we show that weighted linear fixed effects regression models can accomodate a number of identification strategies including matching, stratification, first difference, propensity score weighting, and difference-in-differences. We prove the results by establishing finite sample equivalence relationships between weighted fixed effects and these estimators. Our analysis identifies the information implicitly used by standard fixed effects models to estimate counterfactual outcomes necessary for causal inference, highlighting the potential sources of their bias and inefficiency. In addition, we develop efficient computation strategies, model-based standard errors, and a specification test for weighted fixed effects estimators. Finally, we illustrate the proposed methodology by revisiting the controversy concerning the effects of the General Agreement on Tariffs and Trade (GATT) membership on international trade. Open-source software is available for fitting the proposed weighted linear fixed effects estimators.

4

Paper

The Varying Role of Voter Information across Democratic Societies Sekhon, Jasjeet Uploaded 07-26-2004 Keywords Voter Information Elections Causal Inference Matching Propensity Score Matching Robust Estimation Democratization Survey Data Abstract Using new robust matching methods for making causal inferences from survey data, I demonstrate that there are profound differences between how voters behave in mature democracies versus how they behave in new ones. The problems of voter ignorance and inattentiveness are not as serious in mature democracies as many analysts have suggested but are of grave concern in new democracies. Citizens in mature democracies are able to accomplish something that citizens in fledgling democracies are not: inattentive and poorly informed citizens are able to vote like their better informed compatriots and hence need to pay little attention to political events such as election campaigns in order to vote as if they were attentive. The results from the U.S. (which rely on various National Election Studies) and Mexico (2000 Panel Study) are reported in detail. Results from other countries are briefly reported.

5

Paper

Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies Diamond, Alexis Sekhon, Jasjeet Uploaded 07-19-2005 Keywords Matching Propensity Score Causal Inference Genetic Algorithm Evolutionary Programming Optimization Program Evaluation Abstract Genetic matching is a new method for performing multivariate matching which uses an evolutionary search algorithm to determine the weight each covariate is given. The method utilizes an evolutionary algorithm developed by Mebane and Sekhon (1998; Sekhon and Mebane 1998) that maximizes the balance of observed potential confounders across matched treated and control units. The method is nonparametric and does not depend on knowing or estimating the propensity score, but the method is greatly improved when a known or estimated propensity score is incorporated. Genetic matching reliably reduces both the bias and the mean square error of the estimated causal effect even when the property of equal percent bias reduction (EPBR) does not hold. When this property does not hold, matching methods---such as Mahalanobis distance and propensity score matching---often perform poorly. Even if the EPBR property does hold and the propensity score is correctly specified, in finite samples, estimates based on genetic matching have lower mean square error than those based on the usual matching methods. We present a reanalysis of the LaLonde (1986) job training dataset which demonstrates the benefits of genetic matching and which helps to resolve a longstanding debate between Dehejia and Wahba (1999, 2002); Dehejia (2005) and Smith and Todd (2001, 2005a,b) over the ability of matching to overcome LaLonde's critique of nonexperimental estimators. Monte Carlos are also presented to demonstrate the properties of our method.

6

Paper

The Dangers of Extreme Counterfactuals King, Gary Zeng, Langche Uploaded 07-18-2005 Keywords propensity score extrapolation counterfactual convex hull distance model dependence Abstract We address the problem that occurs when inferences about counterfactuals -- predictions, ``what if'' questions, and causal effects -- are attempted far from the available data. The danger of these extreme counterfactuals is that substantive conclusions drawn from statistical models that fit the data well turn out to be based largely on speculation hidden in convenient modeling assumptions that few would be willing to defend. Yet existing statistical strategies provide few reliable means of identifying extreme counterfactuals. We offer a proof that inferences farther from the data are more model-dependent, and then develop easy-to-apply methods to evaluate how model-dependent our answers would be to specified counterfactuals. These methods require neither sensitivity testing over specified classes of models nor evaluating any specific modeling assumptions. If an analysis fails the simple tests we offer, then we know that substantive results are sensitive to at least some modeling choices that are not based on empirical evidence. The most recent version of this paper and software that implements the methods described is available at http://gking.harvard.edu.

7

Paper

Causal Inference of Repeated Observations: A Synthesis of the Propensity Score Methods and Multilevel Modeling Su, Yu-Sung Uploaded 07-03-2008 Keywords causal inference balancing score multilevel modeling propensity score time-series-cross-sectional data Abstract The fundamental problem of causal inference is that an individual cannot be simultaneously observed in both the treatment and control states (Holland 1986). The propensity score methods that compare the treatment and control groups by discarding the unmatched units are now widely used to deal with this problem. In some situations, however, it is possible to observe the same individual or unit of observation in the treatment and control states at different points in time. The data has the structure that is often refer to as time-series-cross-sectional (TSCS) data. While multilevel modeling is often applied to analyze TSCS data, this paper proposes that synthesizing the propensity score methods and multilevel modeling is preferable. The paper conducts a Monte Carlo simulation with 36 different scenarios to test the performance of the two combined methods. The result shows that synthesizing the propensity score matching with multilevel modeling performs better in that such method yields less biased and more efficient estimates. An empirical case study that reexamine the model of Przeworksi et al (2000) on democratization and development also shows the advantage of this synthesis.

8

Poster

Optimally Selecting Matched Samples Nielsen, Rich Uploaded 07-20-2010 Keywords Matching CEM Propensity Score Calipers Abstract We apply a new and simple graphical method (the ``space graph"; Iacus, King, and Porro, 2010) for evaluating many matched samples and selecting the best one(s). We then use this technique to reveal patterns in the relative performance of matching methods across data sets. We also identify an important and previously unnoticed problem that causes propensity score matching with calipers to fail in precisely the applications for which it was designed.