logoPicV1 logoTextV1

Search Results

Below results based on the criteria 'randomization inference'
Total number of records returned: 6

Attributing Effects to A Cluster Randomized Get-Out-The-Vote Campaign: An Application of Randomization Inference Using Full Matching
Bowers, Jake
Hansen, Ben

Uploaded 07-18-2005
Keywords causal inference
randomization inference
attributable effects
full matching
instrumental variables
missing data
field experiments
Abstract Statistical analysis requires a probability model: commonly, a model for the dependence of outcomes $Y$ on confounders $X$ and a potentially causal variable $Z$. When the goal of the analysis is to infer $Z$'s effects on $Y$, this requirement introduces an element of circularity: in order to decide how $Z$ affects $Y$, the analyst first determines, speculatively, the manner of $Y$'s dependence on $Z$ and other variables. This paper takes a statistical perspective that avoids such circles, permitting analysis of $Z$'s effects on $Y$ even as the statistician remains entirely agnostic about the conditional distribution of $Y$ given $X$ and $Z$, or perhaps even denies that such a distribution exists. Our assumptions instead pertain to the conditional distribution $Z vert X$, and the role of speculation in settling them is reduced by the existence of random assignment of $Z$ in a field experiment as well as by poststratification, testing for overt bias before accepting a poststratification, and optimal full matching. Such beginnings pave the way for ``randomization inference'', an approach which, despite a long history in the analysis of designed experiments, is relatively new to political science and to other fields in which experimental data are rarely available. The approach applies to both experiments and observational studies. We illustrate this by applying it to analyze A. Gerber and D. Green's New Haven Vote 98 campaign. Conceived as both a get-out-the-vote campaign and a field experiment in political participation, the study assigned households to treatment and desired to estimate the effect of treatment on the individuals nested within the households. We estimate the number of voters who would not have voted had the campaign not prompted them to --- that is, the total number of votes attributable to the interventions of the campaigners --- while taking into account the non-independence of observations within households, non-random compliance, and missing responses. Both our statistical inferences about these attributable effects and the stratification and matching that precede them rely on quite recent developments from statistics; our matching, in particular, has novel features of potentially wide applicability. Our broad findings resemble those of the original analysis by citet{gerbergreen00}.

Making Inferences from 2x2 Tables: The Inadequacy of the Fisher Exact\r\nTest for Observational Data and a Principled Bayesian Alternative
Sekhon, Jasjeet

Uploaded 08-17-2005
Keywords Fisher exact test
randomization inference
permutation tests
Bayesian tests
difference of proportions
observational data
Abstract The Fisher exact test is the dominant method of making inferences from 2x2 tables where the number of observations is small. Although the Fisher test and approximations to it are used in a large number of studies, these tests rest on a data generating process which is inappropriate for most applications for which they are used. The canonical Fisher test assumes that both of the margins in a 2x2 table are fixed by construction---i.e., both the treatment and outcome margins are fixed a priori. If the data were generated by an alternative process, such as binomial, negative binomial or Poisson binomial sampling, the Fisher exact test and approximations to it do not have correct coverage. A Bayesian method is offered which has correct coverage, is powerful, is consistent with a binomial process and can be extended easily to other distributions. A prominent 2x2 table which has been used in the literature by Geddes (1990) and Sekhon (2004) to explore the relationship between foreign threat and social revolution (Skocpol, 1979) is reanalyzed. The Bayesian method finds a significant relationship even though the Fisher and related tests do not. A Monte Carlo sampling experiment is provided which shows that the Bayesian method dominates the usual alternatives in terms of both test coverage and power when the data are generated by a binomial process.

Variance Identification and Efficiency Analysis in Randomized Experiments under the Matched-Pair Design
Imai, Kosuke

Uploaded 07-17-2007
Keywords Average Treatment Effect
Causal Inference
Experimental Design
Matched Samples
Paired Comparison
Randomization Inference.
Abstract In his landmark article, Neyman (1923) introduced randomization-based inference in analyzing experiments under the completely randomized design. Under this framework, Neyman considered the statistical estimation of the sample average treatment effect and derived the variance of the standard estimator using the treatment assignment mechanism as the sole basis of inference. In this paper, I extend Neyman's analysis to randomized experiments under the matched-pair design where experimental units are paired based on their pre-treatment characteristics and the randomization of treatment is subsequently conducted within each matched pair. I study the variance identification for the standard estimator of average treatment effects and analyze the relative efficiency of the matched-pair design over the completely randomized design. I also show how to empirically evaluate the relative efficiency of the two designs using experimental data obtained under the matched-pair design. My randomization-based analysis clarifies some of the important questions raised in the literature and identifies a hiden and yet implausible assumption that is made for the efficiency analysis in a widely used textbook. Finally, the analytical results are illustrated with numerical and empirical examples.

A General Method for Detecting Interference Between Units in Randomized Experiments
Aronow, Peter

Uploaded 08-17-2010
Keywords Rubin Causal Model
Permutation test
Causal inference
Randomization inference
Abstract Interference between units may pose a threat to unbiased causal inference in randomized controlled experiments. Although the assumption of no interference is essential for causal inference, few options are available for testing this assumption. This paper presents the first reliable ex post method for detecting interference between units in randomized experiments. Naive estimators of interference that attempt to exploit the proximity of units may be biased because simple randomization of units into treatment does not imply simple randomization of proximity to treated units. However, through a randomization-based approach, the confounding associated with these naive estimators may be circumvented entirely. With a test statistic of the analyst's choice, a conditional randomization test allows for the calculation of the exact significance of the causal dependence of outcomes on the treatment status of other units. The efficacy and robustness of the method is demonstrated through simulation studies and, using this method, interference between units is detected in a field experiment designed to assess the effect of mailings on voter turnout.

Agnostic Notes on Regression Adjustments to Experimental Data: Reexamining Freedman's Critique
Lin, Winston

Uploaded 09-02-2011
Keywords Covariate adjustment
Randomization inference
Neyman's repeated sampling approach
Sandwich estimator
Social experiments
Abstract Freedman [Adv. in Appl. Math. 40 (2008a) 180–193; Ann. Appl. Stat. (2008b) 2 176–196] critiqued OLS regression adjustment of estimated treatment effects in randomized experiments, using Neyman’s model for randomization inference. This paper argues that in sufficiently large samples, the statistical problems he raised are either minor or easily fixed. OLS adjustment improves or does not hurt asymptotic precision when the regression includes a full set of treatment-covariate interactions. Asymptotically valid confidence intervals can be constructed with the Huber-White sandwich standard error estimator. Even the traditional OLS adjustment has benign large-sample properties when subjects are randomly assigned to two groups of equal size. The strongest reasons to support Freedman’s preference for unadjusted estimates are transparency and the dangers of specification search.

Reasoning about Interference Between Units}
Bowers, Jake
Fredrickson, Mark
Panagopoulos, Costas

Uploaded 07-13-2012
Keywords interference
randomization inference
randomized experiments
Fisher's sharp null hypothesis
causal inference
Abstract If an experimental treatment is experienced by both treated and control group units, tests of hypotheses about causal effects may be difficult to conceptualize let alone execute. In this paper, we show how counterfactual causal models may be written and tested when theories suggest spillover or other network-based interference among experimental units. We show that the ``no interference'' assumption need not constrain scholars who have interesting questions about interference. We offer researchers the ability to model theories about how treatment given to some units may come to influence outcomes for other units. We further show how to test hypotheses about these causal effects, and we provide tools to enable researchers to assess the operating characteristics of their tests given their own models, designs, test statistics, and data. The conceptual and methodological framework we develop here is particularly applicable to social networks, but may be usefully deployed whenever a researcher wonders about interference between units. Interference between units need not be an untestable assumption; instead, interference is an opportunity to ask meaningful questions about theoretically interesting phenomena.

< prev 1 next>