logoPicV1 logoTextV1

Search Results

Below results based on the criteria 'experiments'
Total number of records returned: 35

Is There a Gender Gap in Fiscal Political Preferences
Alvarez, R. Michael
McCaffery, Edward J.

Uploaded 08-12-2000
Keywords Gender gap
fiscal politics
budget surplus
multinomial logit
missing data
survey experiments
Abstract This paper examines the relationship between attitudes on potential uses of the budget surplus and gender. Survey results show relatively weak support overall for using a projected surplus to reduce taxes, with respondents much likelier to prefer increased social spending on education or social security. There is a significant gender gap with men being far more likely than women to support tax cuts or paying down the national debt. Given a menu of particular types of tax cuts, women are marginally more likely to favor child-care relief or working poor tax credits whereas men are marginally more likely to favor capital gains reduction or tax rate cuts. When primed that the tax laws are biased against two-worker families, men significantly change their preferences, moving from support for general tax rate cuts to support for working poor tax relief, but not to child-care relief. One of the strongest results to emerge is that women are far more likely than men not to express an opinion or to confess ignorance about fiscal matters. Both genders increase their ``no opinion'' answer in the face of priming, but men more so than women. Further research will explore this no opinion/uncertainty aspect.

On the Use of Linear Fixed E ects Regression Models for Causal Inference
Imai, Kosuke
Kim, In Song

Uploaded 07-23-2012
Keywords difference-in-differences
first difference
observational data
panel data
propensity score
randomized experiments
Abstract Linear fixed effects regression models are a primary workhorse for causal inference among applied researchers. And yet, it has been shown that even when the treatment is exogenous within each unit, the linear regression models with unit-specific fixed effects may not consistently estimate the average treatment effect. In this paper, we offer a simple solution. Specifically, we show that weighted linear fixed effects regression models can accomodate a number of identification strategies including matching, stratification, first difference, propensity score weighting, and difference-in-differences. We prove the results by establishing finite sample equivalence relationships between weighted fixed effects and these estimators. Our analysis identifies the information implicitly used by standard fixed effects models to estimate counterfactual outcomes necessary for causal inference, highlighting the potential sources of their bias and inefficiency. In addition, we develop efficient computation strategies, model-based standard errors, and a specification test for weighted fixed effects estimators. Finally, we illustrate the proposed methodology by revisiting the controversy concerning the effects of the General Agreement on Tariffs and Trade (GATT) membership on international trade. Open-source software is available for fitting the proposed weighted linear fixed effects estimators.

A Theory of Nonseparable Preferences in Survey Responses
Lacy, Dean

Uploaded 07-11-1997
Keywords nonseparable-preferences
Abstract This paper presents a model of individual-level responses to issue questions in public opinion surveys when respondents have nonseparable preferences. The model implies two results: responses will change depending on the order of questions and vary over time. Each of these conclusions is consistent with empirical findings that are often cited to support the argument that people are irrational or lack fixed and well-formed preferences. Results from an experiment reveal that question-order effects occur on issues for which people have nonseparable preferences, and order effects do not occur on issues for which most people have separable preferences.

Causal Inference in Conjoint Analysis: Understanding Multi-Dimensional Choices via Stated Preference Experiments
Hainmueller, Jens
Hopkins, Daniel
Yamamoto, Teppei

Uploaded 12-12-2012
Keywords potential outcomes
average marginal component effects
fractional factorial design
orthogonal design
randomized design
survey experiments
public opinion
vote choice
Abstract For decades, market researchers have used conjoint analysis to understand how consumers make decisions when faced with multi-dimensional choices. In such analyses, respondents are asked to score or rank a set of alternatives, where each alternative is defined by multiple attributes which are varied randomly or intentionally. Political scientists are frequently interested in parallel questions about decision-making, yet to date conjoint analysis has seen little use within the field. In this manuscript, we demonstrate the potential value of conjoint analysis in political science, using examples about vote choice and immigrant admission to the United States. In doing so, we develop a set of statistical tools for drawing causal conclusions from stated preference data based on the potential outcomes framework of causal inference. We discuss the causal estimands of interest and provide a formal analysis of the assumptions required for identifying those quantities. Prior conjoint analyses have typically used designs which limit the number of unique conjoint profiles. We employ a survey experiment to compare this approach to a fully randomized approach. Both our formal analysis of the causal estimands and our empirical results highlight the potential biases of common approaches to conjoint analysis which restrict the number of profiles.

Treatment Spillover Effects Across Survey Experiments
Lee, Daniel
Transue, John
Aldrich, John

Uploaded 04-05-2005
Keywords survey experiments
survey methods
Abstract Embedding experiments within surveys has reinvigorated survey research in general and especially in political science. These designs use random assignment to create true experiments within (typically nationally) representative sample surveys. Thus, they combine the internal validity of experiments with the external validity of national surveys. We investigate whether experimental treatments spill over and effect later experiments in an unintended manner. Using the 1991 Race and Politics survey, we find evidence of experimental spillover. Specifically we find that experiments at the beginning of a survey influence later experiments. We also find (much less) evidence of adjacent experiments affecting subsequent experiments. The paper concludes with a discussion of designs for future research that could aid our understanding of experimental spillover.

Definition and Diagnosis of Problematic Attrition in Randomized Controlled Experiments
Martel García, Fernando

Uploaded 04-25-2013
Keywords attrition
randomized controlled experiments
field experiments
causal diagrams
directed acyclic graphs
average treatment effect
Abstract Attrition is the Achilles' Heel of the randomized experiment: It is fairly common, and it can completely unravel the benefits of randomization. Using the structural language of causal diagrams I demonstrate that attrition is problematic for identification of the average treatment effect (ATE) if -- and only if -- it is a common effect of the treatment and the outcome (or a cause of the outcome other than the treatment). I also demonstrate that whether the ATE is identified and estimable for the full population of units in the experiment, or only for those units with observed outcomes, depends on two d-separation conditions. One of these is testable ex-post under standard experimental assumptions. The other is testable ex-ante so long as adequate measurement protocols are adopted. Missing at Random (MAR) assumptions are neither necessary nor sufficient for identification of the ATE.

Attributing Effects to A Cluster Randomized Get-Out-The-Vote Campaign: An Application of Randomization Inference Using Full Matching
Bowers, Jake
Hansen, Ben

Uploaded 07-18-2005
Keywords causal inference
randomization inference
attributable effects
full matching
instrumental variables
missing data
field experiments
Abstract Statistical analysis requires a probability model: commonly, a model for the dependence of outcomes $Y$ on confounders $X$ and a potentially causal variable $Z$. When the goal of the analysis is to infer $Z$'s effects on $Y$, this requirement introduces an element of circularity: in order to decide how $Z$ affects $Y$, the analyst first determines, speculatively, the manner of $Y$'s dependence on $Z$ and other variables. This paper takes a statistical perspective that avoids such circles, permitting analysis of $Z$'s effects on $Y$ even as the statistician remains entirely agnostic about the conditional distribution of $Y$ given $X$ and $Z$, or perhaps even denies that such a distribution exists. Our assumptions instead pertain to the conditional distribution $Z vert X$, and the role of speculation in settling them is reduced by the existence of random assignment of $Z$ in a field experiment as well as by poststratification, testing for overt bias before accepting a poststratification, and optimal full matching. Such beginnings pave the way for ``randomization inference'', an approach which, despite a long history in the analysis of designed experiments, is relatively new to political science and to other fields in which experimental data are rarely available. The approach applies to both experiments and observational studies. We illustrate this by applying it to analyze A. Gerber and D. Green's New Haven Vote 98 campaign. Conceived as both a get-out-the-vote campaign and a field experiment in political participation, the study assigned households to treatment and desired to estimate the effect of treatment on the individuals nested within the households. We estimate the number of voters who would not have voted had the campaign not prompted them to --- that is, the total number of votes attributable to the interventions of the campaigners --- while taking into account the non-independence of observations within households, non-random compliance, and missing responses. Both our statistical inferences about these attributable effects and the stratification and matching that precede them rely on quite recent developments from statistics; our matching, in particular, has novel features of potentially wide applicability. Our broad findings resemble those of the original analysis by citet{gerbergreen00}.

A Unified Approach to Generalized Causal Inference
Martel Garcia, Fernando

Uploaded 08-01-2013
Keywords External validity
causal diagrams
Abstract Randomized controlled trials and natural experiments have been criticized for their lack of generalizability (external validity), questioning their usefulness to social science and policy. Here I show how three common approaches to generalizability - the heuristic, statistical, and structural approaches --, are each incomplete on their own, and how generalized causal diagrams, or g-dags, can achieve a complete representation of the problem. G-dags combine theory and evidence to (1) make inferences from a study to a population, or subgroup; (2) combine two or more studies that are not generalizable on their own, into a generalized inference; (3) encode and test generalizable knowledge; and (4) provide a link to boosting algorithms as generalized additive models. Just as important, g-dags make make explicit what is being assumed, or questioned, in discussing the generalizability of experiments. This allows for constructive discourse and informed research agendas.

The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation
Imai, Kosuke
King, Gary
Nall, Clayton

Uploaded 07-17-2007
Keywords causal inference
community intervention trials
field experiments
group-randomized trials
health policy
matched-pair design
Abstract A basic feature of many field experiments is that investigators are only able to randomize clusters of individuals -- such as households, communities, firms, medical practices, schools, or classrooms -- even when the individual is the unit of interest. To recoup some of the resulting efficiency loss, many studies pair similar clusters and randomize treatment within pairs. Other studies (including almost all published political science field experiments) avoid pairing, in part because some prominent methodological articles claim to have identified serious problems with this 'matched-pair cluster-randomized' design. We prove that all such claims about problems with this design are unfounded. We then show that the estimator for matched-pair designs favored in the literature is appropriate only in situations where matching is not needed. To address this problem without modeling assumptions, we generalize Neyman's (1923) approach and propose a simple new estimator with much improved statistical properties. We also introduce methods to cope with individual-level noncompliance, which most existing approaches incorrectly assume away. We show that from the perspective of, among other things, bias, efficiency, or power, pairing should be used in cluster-randomized experiments whenever feasible; failing to do so is equivalent to discarding a considerable fraction of one's data. We develop these techniques in the context of a randomized evaluation we are conducting of the Mexican Universal Health Insurance Program.

Unresponsive, Unpersuaded: The Unintended Consequences of Voter Persuasion Efforts
Bailey, Michael
Hopkins, Daniel
Rogers, Todd

Uploaded 08-09-2013
Keywords causal inference
field experiments
multiple imputation
Approximate Bayesian Bootstrap
Abstract Can randomized experiments at the individual level help assess the persuasive effects of campaign tactics? In the contemporary U.S., vote choice is not observable, so one promising research design to assess persuasion involves randomizing appeals and then using a survey to measure vote intentions. Here, we analyze one such field experiment conducted during the 2008 presidential election in which 56,000 registered voters were assigned to persuasion in person, by phone, and/or by mail. Persuasive appeals by canvassers had two unintended consequences. First, they reduced responsiveness to the follow-up survey, lowering the response rate sharply among infrequent voters. Second, various statistical methods to address the resulting biases converge on a counter-intuitive conclusion: the persuasive canvassing reduced candidate support. Our results allow us to rule out even small effects in the intended direction, and illustrate the backlash that persuasion can engender.

Misunderstandings among Experimentalists and Observationalists about Causal Inference
Imai, Kosuke
King, Gary
Stuart, Elizabeth

Uploaded 09-16-2007
Keywords matching
causal inference
experimental design
observational studies
average treatment effects
covariate balance
field experiments
survey experiments
Abstract We attempt to clarify, and suggest how to avoid, several serious misunderstandings about and fallacies of causal inference in experimental and observational research. These issues concern some of the most basic advantages and disadvantages of each basic research design. Problems include improper use of hypothesis tests for covariate balance between the treated and control groups, and the consequences of using randomization, blocking before randomization, and matching after treatment assignment to achieve covariate balance. Applied researchers in a wide range of scientific disciplines seem to fall prey to one or more of these fallacies, and as a result make suboptimal design or analysis choices. To clarify these points, we derive a new four-part decomposition of the key estimation errors in making causal inferences. We then show how this decomposition can help scholars from different experimental and observational research traditions better understand each other's inferential problems and attempted solutions. (This paper is forthcoming in the Journal of the Royal Statistical Society, but we have some time for revisions and would value any comments anyone might have. This is a revised and much more general version of an earlier paper, "The Balance Test Fallacy in Causal Inference".)

Voter Persuasion in Compulsory Electorates: Evidence from a Field Experiment in Australia
Lam, Patrick
Peyton, Kyle

Uploaded 12-14-2013
Keywords voter persuasion
field experiments
causal inference
missing data
Abstract Most of the literature on grassroots campaigning focuses on mobilizing potential sup- porters to turn out to vote. The actual ability of partisan campaigns to boost support by changing voter preferences is unclear. We present the results of a field experiment the Australian Council of Trade Unions (ACTU) ran during the 2013 Australian Federal Election. The experiments were designed to minimize the conservative (the Coalition) vote as part of one of the largest and most extensively documented voter persuasion campaigns in Australian history. Union members who were identified as undecided voters in over 30 electorates were targeted with appeals by direct mail and phone banks. Because of the presence of compulsory voting in Australia, we are able to identify the effects of voter persuasion independently of voter turnout. We find that direct mail, the most extensively used campaign strategy in Australia, has little effect of voter persuasion. Direct human contact, on the other hand, seems to be an effective tool for voter persuasion. Among undecided voters who actually receive direct contact via phone call, we find a ten percentage point decrease in the Coalition vote. From a methodological standpoint, we use various methods to account for multiple treatment arms, measured treatment noncompliance in one of the treatments, and missing outcome and covariate data. The field experiment also provides a good lesson in conducting and saving broken experiments in the presence of planning uncertainty and implementation failures.

Measurement Error as a Threat to Causal Inference: Acquiescence Bias and Deliberative Polling
Weiksner, G. Michael

Uploaded 06-29-2008
Keywords Causal inference
acquiescence bias
deliberative polling
measurement error
questionnaire design
Abstract Experiments, unlike observational studies, are rarely criticized for yielding invalid causal inferences. However, I identify measurement error as a threat to causal inference of an experiment. In particular, acquiescence bias, a common and substantial source of measurement error within surveys, may be correlated with experimental manipulations. Using data from a survey experiment embedded in a Deliberative Poll, I find that acquiescence bias causes significant measurement error and that the bias differs before and after deliberation. I conclude that even experimental researchers should heed the recommendation by questionnaire design researchers to refrain from asking agree/disagree questions completely and instead ask only construct-specific questions to avoid this threat to validity.

Adjusting Experimental Data
Keele, Luke
McConnaughy, Corrine
White, Ismail

Uploaded 07-06-2008
Keywords Experiments
Abstract Randomization in experiments allows researchers to assume that the treatment and control groups are balanced with respect to all characteristics except the treatment. Randomization, however, only makes balance probable, and accidental covariate imbalance can occur for any specific randomization. As such, statistical adjustments for accidental imbalance are common with experimental data. The most common method of adjustment for accidental imbalance is to use least squares to estimate the analysis of covariance (ANCOVA) model. ANCOVA, however, is a poor choice for the adjustment of experimental data. It has a strong functional form assumption, and the least squares estimator is notably biased in sample sizes of less than 500 when applied to the analysis of treatment effects. We evaluate alternative methods of adjusting experimental data. We compare ANCOVA to two different techniques. The first technique is a modified version of ANCOVA that relaxes the strong functional form assumption of this model. The second technique is matching, and we test the differences between two matching methods. For the first, we match subjects and then randomize treatment across pairs. For the second, we randomize the treatment and match prior to the estimation of treatment effects. We use all three techniques with data from a series of experiments on racial priming. We find that matching substantially increases the efficiency of experimental designs.

Research Opportunities - The 2009/10 British Election Study
Clarke, Harold
Sanders, David
Stewart, Marianne
Whiteley, Paul

Uploaded 07-07-2008
Keywords electons
public opinion
Abstract The 2009/10 British Election Study (BES) will include significant research opportunities for students of voting, elections and public opinion. The BES will have three major components: (a) in-person pre-post election surveys; (b) rolling campaign internet panel survey (RCPS); (c) 48 inter-election monthly continuous monitoring surveys (CMS) with annual panel components. Each CMS survey will offer researchers opportunities to include question batteries including experiments. Participation is free and data release is very fast. Proposals for research modules reviewed by BES Advisory Board and P.I.s. Proposals also entertained for research modules on core and RCPS components.

Regression Adjustments to Experimental Data: Do David Freedman??s Concerns Apply to Political Science?
Green, Donald

Uploaded 07-15-2009
Keywords Experiments
Analysis of Covariance
Abstract Abstract: One of David Freedman's important legacies was to raise awareness of the assumptions that underlie everyday statistical practice, such as regression analysis. His recent papers (Freedman 2008a, 2008b) offer stern warnings to those who offer regression analysis as an appropriate way to analyze experimental results. In particular, Freedman demonstrates that including pre-treatment covariates as controls leads to bias in finite samples and inaccurate standard errors. Freedman advises researchers against using regression adjustments for experiments involving fewer than 500 observations (2008a, p.191), a recommendation that has gained increasing attention and acceptance among social scientists. This paper argues that the ever-cautious Freedman was probably too cautious in his recommendations. After explicating the special features of Freedman's model, I use a combination of simulated and actual examples to show that as a practical matter the biases that Freedman pointed out tend to be negligible for N > 20. Pathological cases that could generate biases for larger experiments involve extreme outliers that would be readily detected through visual inspection.

Treatment effects in before-after data
Gelman, Andrew

Uploaded 04-27-2004
Keywords correlation
hierarchical models
observational studies
variance components
Abstract In experiments and observations with before-after data, the correlation between "before" and "after" measurements is typically higher among the controls than among the treated units, violating the usual assumptions of equal variance and a constant treatment effect. We illustrate with three applied examples and then discuss models that could be used to fit this phenomenon, which we argue is related to the

From Nature to the Lab: The Methodology of Experimental Political Science and the Study of Causality
Morton, Rebecca
Williams, Kenneth

Uploaded 09-18-2009
Keywords experiments
Abstract In this manuscript we review the methodology of experimental political science and the study of causality.

Causal Interaction in High Dimension
Imai, Kosuke
Egami, Naoki

Uploaded 07-17-2015
Keywords causal inference
conjoint analysis
factorial experiments
heterogeneous treatment effects
randomized experiments
variable selection
Abstract Estimating causal interaction effects is essential for the exploration of heterogeneous treatment effects. In the presence of multiple treatment variables with each having several levels, researchers are often interested in identifying the combinations of treatments that induce large additional causal effects beyond the sum of separate effects attributable to each treatment. We show, however, the standard definition of causal interaction effect, typically estimated with the standard linear regression or ANOVA, suffers from the lack of invariance to the choice of baseline condition and the difficulty of interpretation beyond two-way interaction. We propose an alternative definition of causal interaction effect, called the marginal treatment interaction effect, whose relative magnitude does not depend on the choice of baseline condition while maintaining an intuitive interpretation even for higher-order interaction. The proposed approach enables researchers to effectively summarize the structure of causal interaction in high-dimension by decomposing the total effect of any treatment combination into the marginal effects and the interaction effects. We also establish the identification condition and develop an estimation strategy for the proposed marginal treatment interaction effects. Our motivating example is conjoint analysis where the existing literature largely assumes the absence of causal interaction. Given a large number of interaction effects, we apply a variable selection method to identify significant causal interaction. Our exploratory analysis of a survey experiment on immigration preferences reveals substantive insights the standard conjoint analysis fails to discover.

Lagging the Dog?: The Robustness of Panel Corrected Standard Errors in the Presence of Serial Correlation and Observation Specific Effects
Kristensen, Ida
Wawro, Gregory

Uploaded 07-13-2003
Keywords time-series cross-section data
serial correlation
fixed effects
panel data
lag models
Monte Carlo experiments
Abstract This paper examines the performance of the method of panel corrected standard errors (PCSEs) for time-series cross-section data when a lag of the dependent variable is included as a regressor. The lag specification can be problematic if observation-specific effects are not properly accounted for, leading to biased and inconsistent estimates of coefficients and standard errors. We conduct Monte Carlo studies to assess how problematic the lag specification is, and find that, although the method of PCSEs is robust when there is little to no correlation between unit effects and explanatory variables, the method's performance declines as that correlation increases. A fixed effects estimator with robust standard errors appears to do better in these situations.

Statistical Inference for the Item Count Technique
Imai, Kosuke

Uploaded 07-19-2010
Keywords list experiments
sensitive questions
survey experiments
unmatched count technique
Abstract The item count technique is a survey methodology that is designed to elicit respondents' truthful answers to sensitive questions such as racial prejudice and drug use. The method is also known as the list experiment or the unmatched count technique and is an alternative to the commonly used randomized response method. In this paper, I propose new nonlinear least squares (NLS) and maximum likelihood (ML) estimators for a multivariate analysis with the item count technique. The two-step estimation procedure and the Expectation Maximization algorithm are developed to facilitate the computation. Enabling a multivariate statistical analysis is essential because the item count technique provides respondents with privacy at the expense of statistical efficiency. As an empirical illustration, the proposed methodology is applied to the 1991 National Race and Politics survey where the investigators used the item count technique to measure the degree of racial hatred in the United States. A small-scale simulation study suggests that the ML estimator can be substantially more efficient than the NLS estimator. The software package is made available to implement the proposed methodology.

Identification and Estimation of Joint Treatment Effects with Instrumental Variables
Blackwell, Matthew

Uploaded 07-21-2015
Keywords causal inference
instrumental variables
direct effects
Abstract Over the last twenty years, a literature spanning several fields of applied statistics has analyzed how to identify and estimate causal effects of a nonrandomized treatment when an instrumental variable (IV) is available. But researchers often have multiple treatments that might interact with one another and want to estimate either the direct or joint effect of these treatments. This paper introduces a set of novel estimands for instrumental variables with multiple treatments and multiple instruments. These estimands are similar to previous IV estimands as they are ``local'' to strata defined by the joint compliance status across the treatments. Furthermore, I show that these estimands are nonparametrically identified under standard instrumental variable assumptions. The paper further develops nonparametric estimators for these quantities and assesses their performance relative to classic parametric approaches like two-stage least squares. Finally, I demonstrate the method through an empirical application to a voter mobilization field experiment with (1) a telephone treatment and (2) an in-person canvassing treatment.

Negotiated Compliance: Social Solutions to the 'Principal's Problem'
Whitford, Andrew B.
Miller, Gary J.
Bottom, William P.

Uploaded 07-11-2003
Keywords principal-agency theory
hierarchical logit
Abstract Principal-agency theory has typically analyzed the principal's problem1: how to write a contract with incentives that will induce an agent to provide the principal with the maximum feasible expected gain. In practice, principal-agent contracts are typically negotiated, not imposed. Experiments indicate that agent compliance is determined less by the negotiated terms of the contract than by expectations created by the negotiation process itself. We interpret this as justification for a renewed interest in the politics of negotiation and bureaucratic politics.

Agnostic Notes on Regression Adjustments to Experimental Data: Reexamining Freedman's Critique
Lin, Winston

Uploaded 09-02-2011
Keywords Covariate adjustment
Randomization inference
Neyman's repeated sampling approach
Sandwich estimator
Social experiments
Abstract Freedman [Adv. in Appl. Math. 40 (2008a) 180–193; Ann. Appl. Stat. (2008b) 2 176–196] critiqued OLS regression adjustment of estimated treatment effects in randomized experiments, using Neyman’s model for randomization inference. This paper argues that in sufficiently large samples, the statistical problems he raised are either minor or easily fixed. OLS adjustment improves or does not hurt asymptotic precision when the regression includes a full set of treatment-covariate interactions. Asymptotically valid confidence intervals can be constructed with the Huber-White sandwich standard error estimator. Even the traditional OLS adjustment has benign large-sample properties when subjects are randomly assigned to two groups of equal size. The strongest reasons to support Freedman’s preference for unadjusted estimates are transparency and the dangers of specification search.

Political Preference Formation: Competition, Deliberation, and the (Ir)relevance of Framing Effects
Druckman, Jamie

Uploaded 07-09-2003
Keywords framing effects
rational choice theory
political psychology
Abstract A framing effect occurs when different, but logically equivalent, words or phrases such as 95% employment or 5% unemployment cause individuals to alter their preferences. Framing effects challenge the foundational assumptions of much of the social sciences (e.g., the existence of coherent preferences or stable attitudes), and raise serious normative questions about democratic responsiveness. Many scholars and pundits assume that framing effects are highly robust in political contexts. Using a new theory and an experiment with more than 550 participants, I show that this is not the case framing effects do not occur in many political settings. Elite competition and citizens inter- personal conversations often vitiate and eliminate framing effects. However, I also find that when framing effects persist, they can be even more pernicious than often thought not only do they suggest incoherent preferences but they also stimulate increased confidence in those preferences. My results have broad implications for preference formation, rational choice theory, political psychology, and experimental design.

Covariate Balancing Propensity Score
Imai, Kosuke
Ratkovic, Marc

Uploaded 07-13-2012
Keywords causal inference
instrumental variables
inverse propensity score weighting
marginal structural models
observational studies
propensity score matching
randomized experiments
Abstract The propensity score plays a central role in a variety of settings for causal inference. In particular, matching and weighting methods based on the estimated propensity score have become increasingly common in observational studies. Despite their popularity and theoretical appeal, the main practical difficulty of these methods is that the propensity score must be estimated. Researchers have found that slight misspecification of the propensity score model can result in substantial bias of estimated treatment effects. In this paper, we introduce covariate balancing propensity score (CBPS) estimation, which simultaneously optimizes the covariate balance and the prediction of treatment assignment. We exploit the dual characteristics of the propensity score as a covariate balancing score and the conditional probability of treatment assignment and estimate the CBPS within the generalized method of moments or empirical likelihood framework. We find that the CBPS dramatically improves the poor empirical performance of propensity score matching and weighting methods reported in the literature. We also show that the CBPS can be extended to a number of other important settings, including the estimation of generalized propensity score for non-binary treatments, causal inference in longitudinal settings, and the generalization of experimental and instrumental variable estimates to a target population.

The importance of statistical methodology for analyzing data from field experimentation: Evaluating voter mobilization strategies
Imai, Kosuke

Uploaded 07-08-2002
Keywords field experiments
causal inference
instrumental variables
Abstract We introduce a set of new Markov chain Monte Carlo algorithms for Bayesian analysis of the multinomial probit model. Our Bayesian representation of the model places a new, and possibly improper, prior distribution directly on the identifiable parameters and thus is relatively easy to interpret and use. Our algorithms, which are based on the method of marginal data augmentation, involve only draws from standard distributions and dominate other available Bayesian methods in that they are as quick to converge as the fastest methods but with a more attractive prior specification.

Reasoning about Interference Between Units}
Bowers, Jake
Fredrickson, Mark
Panagopoulos, Costas

Uploaded 07-13-2012
Keywords interference
randomization inference
randomized experiments
Fisher's sharp null hypothesis
causal inference
Abstract If an experimental treatment is experienced by both treated and control group units, tests of hypotheses about causal effects may be difficult to conceptualize let alone execute. In this paper, we show how counterfactual causal models may be written and tested when theories suggest spillover or other network-based interference among experimental units. We show that the ``no interference'' assumption need not constrain scholars who have interesting questions about interference. We offer researchers the ability to model theories about how treatment given to some units may come to influence outcomes for other units. We further show how to test hypotheses about these causal effects, and we provide tools to enable researchers to assess the operating characteristics of their tests given their own models, designs, test statistics, and data. The conceptual and methodological framework we develop here is particularly applicable to social networks, but may be usefully deployed whenever a researcher wonders about interference between units. Interference between units need not be an untestable assumption; instead, interference is an opportunity to ask meaningful questions about theoretically interesting phenomena.

Binding the Frame: How Important are Frames for Survey Response?
Alvarez, R. Michael
Brehm, John

Uploaded 08-26-2000
Keywords framing
survey experiments
heteroskedastic choice
Abstract [not transcribed]

Estimating Average Causal Effects Under General Interference
Aronow, Peter
Samii, Cyrus

Uploaded 07-16-2012
Keywords interference
randomized experiments
causal inference
Abstract This paper presents randomization-based methods for estimating average causal effects under arbitrary interference of known form. We present conservative estimators of the randomization variance of the average treatment effects estimators and a justification for confidence intervals based on a normal approximation. Examples relevant to research in environmental protection, networks experiments, "viral marketing," two-stage disease prophylaxis trials, and stepped-wedge designs are presented.

Using the Predicted Responses from List Experiments as Explanatory Variables in Regression Models
Park, Bethany
Imai, Kosuke
Greene, Kenneth F.

Uploaded 07-16-2014
Keywords List experiments
Maximum likelihood
Item count technique
Survey methods
Comparative politics
EM algorithm
Abstract The list experiment, also known as the item count technique, is becoming increasingly popular as a survey methodology for eliciting truthful responses to sensitive questions. Recently, multivariate regression techniques have been developed to predict the unobserved response to sensitive questions using respondent characteristics. Nevertheless, no method exists for using this predicted response as an explanatory variable in another regression model. We address this gap by first improving the performance of a naive two-step estimator. Despite its simplicity, this improved two-step estimator can only be applied to linear models and is statistically inefficient. We therefore develop a maximum likelihood estimator that is fully efficient and applicable to a wide range of models. We use a simulation study to evaluate the empirical performance of the proposed methods. We also apply them to the Mexico 2012 Panel Study and examine whether vote-buying is associated with increased turnout and candidate approval. The proposed methods are implemented in open-source software.

Methods for Extremely Large Scale Media Experiments and Observational Studies
King, Gary
Schneer, Benjamin
White, Ariel

Uploaded 07-18-2014
Keywords experiments
public opinion
media effects
social media
text analysis
time series
causal inference
Abstract We develop statistical methods and large scale data engineering approaches for estimating the effects of a large number of mass and specialized media sites on opinions expressed in the daily flow of millions of social media posts. We first describe the instruments we adapt, develop, and validate for summarizing detailed opinions in social media posts, and then outline the procedures we devised for acquiring and summarizing news content from, and web traffic to, large numbers of media outlets. We then derive statistical methods for estimating the causal effect of changes in the news on social media opinions appropriate for observational, quasi-experimental, and experimental settings

Item Count Technique Estimators under Measurement Error
Ahlquist, John

Uploaded 07-22-2014
Keywords list experiments
measurement error
Abstract I compare the performance of the ICT-MLE (Imai 2011) and simple difference-in-means (DiM) estimators for survey list experiments under respondent error. I document that respondent error arises in practice and is likely to cause bias. I then report the results of Monte Carlo experiments examining the sensitivity of ICT-MLE and DiM to different kinds of respondent error. I find that, relative to simple t-tests and its generalizations, ICT-MLE is very sensitive to respondent error. This bias in population prevalence, covariate parameter, and standard error estimates becomes more extreme as the underlying prevalence of the sensitive item decreases. Error that is biased toward the extremes of the response distribution is particularly problematic. Some preliminary solutions are proposed.

Using Experiments to Improve Ideal Point Estimation in Text with an Application to Political Ads
Henderson, John

Uploaded 07-13-2015
Keywords text
ideal points
supervised learning
ridge regression
Abstract Researchers are rapidly developing new automated techniques to scale political speech on an ideological dimension. Yet, the task has proven difficult across many settings. Political advertisements, in particular, have eluded such efforts. Candidates air relatively few ads, containing limited policy information, and there is little agreement about how to model political speech in the campaign, much less in general. Rather than model the underlying ideological structure of words, I develop an experimental approach to directly measure the content of political ads. I randomly assign ads to subjects, recruited in a large-N survey, who are asked to guess the party (or ideological leaning) of the featured candidate. Ads are then scaled as their expected partisan guessing score. This score is well-measured given random assignment and subject recruitment, and can be used in a supervised learning approach to scale other ad text. Due to the inferential nature of the task, subjects are less likely to exhibit bias in their guessing. Further, I show that the average partisan signal in ads is synonymous with an ideological dimension in the minds of respondents. I implement a number of tests to assess party guessing as a way to scale ads, each of which indicate remarkable reliability and validity in the approach. Finally, I explore ways to scale up the guessing task to a much larger set of ads. Beyond scaling ads, the inferential approach outlined here can be generalized to measure a much wider array of dimensions contained in speech and text data.

Improving experiments with threshold blocking
Sävje, Fredrik

Uploaded 07-24-2015
Keywords blocking
experimental design
Abstract A common method to reduce the uncertainty of causal inferences from experiments is to assign treatments in fixed proportions within groups of similar units---blocking. Previous results indicate that one can expect substantial reductions in variance if these groups are formed so to contain exactly as many units as treatment conditions. This approach can be contrasted to threshold blocking which, instead of specifying a fixed size, requires that the groups contain a minimum number of units. In this paper I will investigate the advantages of respective method. In particular, I show that threshold blocking is superior to fixed-sized blocking in the sense that it, for any given objective and sample, always finds a weakly better grouping. For blocking problems where the objective function is unknown, this need, however, not hold and a fixed-sized design can perform better. I specifically examine the factors that govern how the methods perform in the common situation where the objective is unconditional variance but groups are constructed based on covariates. This reveals that the relative performance of threshold blocking increases when the covariates become more predictive of the outcome.

< prev 1 next>