1

Paper

Unpacking the Black Box: Learning about Causal Mechanisms from Experimental and Observational Studies Imai, Kosuke Keele, Luke Tingley, Dustin Yamamoto, Teppei Uploaded 07-01-2010 Keywords causal inference direct and indirect effects mediation moderation potential outcomes sensitivity analysis media cues incumbency effects Abstract Understanding causal mechanisms is a fundamental goal of social science research. Demonstrating whether one variable causes a change in another is often insufficient, and researchers seek to explain why such a causal relationship arises. Nevertheless, little is understood about how to identify causal mechanisms in empirical research. Many researchers either informally talk about possible causal mechanisms or attempt to quantify them without explicitly stating the required assumptions. Often, some assert that process tracing in detailed case studies is the only way to evaluate causal mechanisms. Others contend the search for causal mechanisms is so elusive that we should instead focus on causal effects alone. In this paper, we show how to learn about causal mechanisms from experimental and observational studies. Using the potential outcomes framework of causal inference, we formally define causal mechanisms, present general identification and estimation strategies, and provide a method to assess the sensitivity of one's conclusions to the possible violations of key identification assumptions. We also propose several alternative research designs for both experimental and observational studies that may help identify causal mechanisms under less stringent assumptions. The proposed methodology is illustrated using media framing experiments and observational studies of incumbency advantage.

2

Paper

Using Qualitative Information to Improve Causal Inference Glynn, Adam Ichino, Nahomi Uploaded 09-23-2012 Keywords case study causal inference qualitative mixed methods sensitivity analysis process-tracing observational study Abstract We demonstrate four techniques that utilize case studies to improve causal inference within the Rosenbaum [2002, 2009] approach to observational studies. This approach accommodates small to medium sample sizes in a nonparametric framework and does not require the elicitation of Bayesian priors. First, we show that this approach allows case studies to ameliorate the effects of poorly measured outcomes, sometimes reducing p-values. Second, we show that qualitative information can be incorporated in an analysis and presented as qualitative confidence intervals. Third, we demonstrate that a standard technique of comparative case studies can improve sensitivity analysis within this framework, sometimes reducing the sensitivity of p-values to unmeasured confounders. Finally, we demonstrate that qualitative information on the heterogeneity of treatments can be used to check the robustness of p-values. We illustrate these methods by examining the effect of not having a runoff provision on opposition harassment in transitional presidential elections in 1990s sub-Saharan Africa.

3

Paper

Statistical Analysis of Randomized Experiments with Nonignorable Missing Binary Outcomes Imai, Kosuke Uploaded 07-24-2006 Keywords Causal Inference Instrumental Variables Intention-to-Treat Effect Latent Ignorability Noncompliance Treatment Effect Sensitivity Analysis Abstract Missing data are frequently encountered in the statistical analysis of randomized experiments. In this article, I propose statistical methods that can be used to analyze randomized experiments with a nonignorable missing binary outcome where the missing-data mechanism may depend on the unobserved values of the outcome variable itself. I first introduce an identification strategy for the average treatment effect and compare it with the existing alternative approaches in the literature. I then derive the maximum likelihood estimator and its asymptotic properties, and discuss possible estimation methods. Furthermore, since the proposed identification assumption is not directly verifiable from the data, I show how to conduct a sensitivity analysis based on the parameterization that links the key identification assumption with the causal quantities of interest. Then, the proposed methodology is extended to the analysis of randomized experiments with noncompliance. Although the method introduced in this article may not directly apply to randomized experiments with non-binary outcomes, I briefly discuss possible identification strategies in more general situations. Finally, I apply the proposed methodology to analyze data from the German election experiment and the influenza vaccination study, which originally motivated the methodological problems addressed in this article.

4

Paper

Learning from the Campaign Context: Multivariate Matching with Exposure Christenson, Dino Uploaded 07-14-2008 Keywords multivariate matching non-bipartite matching signed rank test sensitivity analysis political information presidential campaigns Abstract PolMeth XXV poster.

5

Paper

What Can Be Learned from a Simple Table? Bayesian Inference and Sensitivity Analysis for Causal Effects from 2x2 and 2x2xK Tables in the Presence of Unmeasured Confounding Quinn, Kevin Uploaded 09-07-2008 Keywords causal inference bayesian inference sensitivity analysis unmeasured confounding Abstract What, if anything, should one infer about the causal effect of a binary treatment on a binary outcome from a $2 imes 2$ cross-tabulation of non-experimental data? Many researchers would answer ``nothing'' because of the likelihood of severe bias due to the lack of adjustment for key confounding variables. This paper shows that such a conclusion is unduly pessimistic. Because the complete data likelihood under arbitrary patterns of confounding factorizes in a particularly convenient way, it is possible to parameterize this general situation with four easily interpretable parameters. Subjective beliefs regarding these parameters are easily elicited and subjective statements of uncertainty become possible. This paper also develops a novel graphical display called the confounding plot that quickly and efficiently communicates all patterns of confounding that would leave a particular causal inference relatively unchanged.

6

Paper

Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analyses of a Field Experiment on Democratic Deliberations Imai, Kosuke Yamamoto, Teppei Uploaded 06-30-2008 Keywords differential misclassification nonparametric bounds retrospective studies sensitivity analysis survey measurements Abstract Political scientists have long been concerned about the validity of survey measurements. Although many have studied classical measurement error in linear regression models where the error is assumed to arise completely at random, in a number of situations the error may be correlated with the outcome. We analyze the impact of differential measurement error on causal estimation. The proposed nonparametric identification analysis avoids arbitrary modeling decisions and formally characterizes the roles of additional assumptions. We show the serious consequences of differential misclassification and offer a new sensitivity analysis that allows researchers to evaluate the robustness of their conclusions. Our methods are motivated by a field experiment on democratic deliberations, in which one set of estimates potentially suffers from differential misclassification. We show that an analysis ignoring differential measurement error may considerably overestimate the causal effects. This finding contrasts with the case of classical measurement error which always yields attenuation bias.

7

Paper

A General Approach to Causal Mediation Analysis Imai, Kosuke Keele, Luke Tingley, Dustin Uploaded 07-20-2009 Keywords causal inference causal mechanisms sensitivity analysis sequential ignorability structural equation modeling unobserved confounder Abstract In a highly influential paper, Baron and Kenny (1986) proposed a statistical procedure to conduct a causal mediation analysis and identify possible causal mechanisms. This procedure has been widely used across many branches of the social and medical sciences and especially in psychology and epidemiology. However, one major limitation of this approach is that it is based on a set of linear regressions and cannot be easily extended to more complex situations that are frequently encountered in applied research. In this paper, we propose an approach that generalizes the Baron-Kenny procedure. Our method can accommodate linear and nonlinear relationships, parametric and nonparametric models, continuous and discrete mediators, and various types of outcome variables. We also provide a formal statistical justification for the proposed generalization of the Baron-Kenny procedure by placing causal mediation analysis within the widely-accepted counterfactual framework of causal inference. Finally, we develop a set of sensitivity analyses that allow applied researchers to quantify the robustness of their empirical conclusions. Such sensitivity analysis is important because as we show the Baron-Kenny procedure and our generalization of it rest on a strong and untestable assumption even in randomized experiments. We illustrate the proposed methods by applying them to a randomized field experiment, the Job Search Intervention Study (JOBS II). We also offer easy-to-use software that implements all of our proposed methods.

8

Poster

Hookworm Eradication as an Instrument for Schooling in the American South Henderson, John Uploaded 08-01-2011 Keywords hookworm education participation rockefeller sanitary commission instrumental variables matching permutation inference sensitivity analysis Abstract I exploit an historical natural experiment to assess whether more schooling causes greater vote participation. Specifically, I leverage the Rockefeller Sanitary Commission’s campaign to eradicate hookworm infection in the early-20th century American South as a plausibly-exogenous instrument for primary and secondary education. I evaluate two county-level interventions from the public health campaign: (a) exposure to the campaign and (b) pre-campaign hookworm incidence. Due to the presence of possible confounders, I use pair (genetic) and dose (optimal) matching techniques to strengthen the exogeneity of both instruments. I then use Rosenbaum permutation inference to assess the inclusion strength of the campaign exposure instrument, and I employ a simultaneous sensitivity analysis to evaluate robustness to remaining bias. Throughout, I find a robust and positive effect of education on participation.