1

Paper

Shaken, Not Stirred: Evidence on Ballot Order Effects from the California Alphabet Lottery, 1978 - 2002 Ho, Daniel E. Imai, Kosuke Uploaded 02-02-2004 Keywords ballots elections causal inference natural experiment randomization fisher test partisan cue Abstract We analyze a natural experiment to answer the longstanding question of whether the name order of candidates on ballots affects election outcomes. Since 1975, California law has mandated randomizing the ballot order with a lottery, where alphabet letters would be shaken vigorously and selected from a container. Previous studies, relying overwhelmingly on non-randomized data, have yielded conflicting results about whether ballot order effects even exist. Using improved statistical methods, our analysis of statewide elections from 1978 to 2002 reveals that in general elections ballot order has a significant impact only on minor party candidates and candidates for nonpartisan offices. In primaries, however, being listed first benefits everyone. In fact, ballot order might have changed the winner in roughly nine percent of all primary races examined. These results are largely consistent with a theory of partisan cuing. We propose that all electoral jurisdictions randomize ballot order to minimize ballot effects.

2

Paper

Randomization Inference with Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election Imai, Kosuke Ho, Daniel Uploaded 07-21-2004 Keywords casual inference Fisher/'s exact test inversion political science voting behavior elections Abstract Since the 2000 U.S. Presidential election, social scientists have rediscovered a long tradition of research that investigates the effects of ballot format on voting. Using a new dataset collected by the New York Times, we investigate the causal effects of being listed on the first ballot page in the 2003 California gubernatorial recall election. California law mandates a complex randomization procedure of ballot order that approximates a classical randomized experiment in real world settings. The recall election also poses particular statistical challenges with an unprecedented 135 candidates running for the office. We apply (nonparametric) randomization inference based on Fisher's exact test, which incorporates the complex randomization procedure and yields accurate confidence intervals. Conventional asymptotic model-based inferences are found to be highly sensitive to assumptions and model specification. Randomization inference suggests that roughly half of the candidates gained more votes when listed on the first page of ballot.

3

Paper

The Insignificance of Null Hypothesis Significance Testing Gill, Jeff Uploaded 02-06-1999 Keywords hypothesis testing inverse probability Fisher Neyman-Pearson Bayesian approaches confidence sets meta-analysis Abstract The current method of hypothesis testing in the social sciences is under intense criticism yet most political scientists are unaware of the important issues being raised. Criticisms focus on the construction and interpretation of a procedure that has dominated the reporting of empirical results for over fifty years. There is evidence that null hypothesis significance testing as practiced in political science is deeply flawed and widely misunderstood. This is important since most empirical work in political science argues the value of findings through the use of the null hypothesis significance test. In this article I review the history of the null hypothesis significance testing paradigm in the social sciences and discuss major problems, some of which are logical inconsistencies while others are more interpretive in nature. I suggest alternative techniques to convey effectively the importance of data-analytic findings. These recommendations are illustrated with examples using empirical political science publications.

4

Paper

Making Inferences from 2x2 Tables: The Inadequacy of the Fisher Exact\r\nTest for Observational Data and a Principled Bayesian Alternative Sekhon, Jasjeet Uploaded 08-17-2005 Keywords Fisher exact test randomization inference permutation tests Bayesian tests difference of proportions observational data Abstract The Fisher exact test is the dominant method of making inferences from 2x2 tables where the number of observations is small. Although the Fisher test and approximations to it are used in a large number of studies, these tests rest on a data generating process which is inappropriate for most applications for which they are used. The canonical Fisher test assumes that both of the margins in a 2x2 table are fixed by construction---i.e., both the treatment and outcome margins are fixed a priori. If the data were generated by an alternative process, such as binomial, negative binomial or Poisson binomial sampling, the Fisher exact test and approximations to it do not have correct coverage. A Bayesian method is offered which has correct coverage, is powerful, is consistent with a binomial process and can be extended easily to other distributions. A prominent 2x2 table which has been used in the literature by Geddes (1990) and Sekhon (2004) to explore the relationship between foreign threat and social revolution (Skocpol, 1979) is reanalyzed. The Bayesian method finds a significant relationship even though the Fisher and related tests do not. A Monte Carlo sampling experiment is provided which shows that the Bayesian method dominates the usual alternatives in terms of both test coverage and power when the data are generated by a binomial process.