The Society for Political Methodology

Working Papers


2007

36 records found

New Search

Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program
Abadie, Alberto, Diamond, Alexis, Hainmueller, Jens
Submitted: 2007-01-20
Keywords: comparative case studies, causal inference, placebo tests, differences-in-differences, program evaluation
Abstract: (click to show/hide) Building on an idea in Abadie and Gardeazabal (2003), this article investigates the application of synthetic control methods to comparative case studies. We discuss the advantages of these methods and apply them to study the effects of Proposition 99, a large-scale tobacco control program that California implemented in 1988. We demonstrate that following Proposition 99 tobacco consumption fell markedly in California relative to a comparable synthetic control region. We estimate that by the year 2000 annual per-capita cigarette sales in California were about 26 packs lower than what they would have been in the absence of Proposition 99. Given that many policy interventions and events of interest in social sciences take place at an aggregate level (countries, regions, cities, etc.) and affect a small number of aggregate units, the potential applicability of synthetic control methods to comparative case studies is very large, especially in situations where traditional regression methods are not appropriate. The methods proposed in this article produce informative inference regardless of the number of available comparison units, the number of available time periods, and whether the data are individual (micro) or aggregate (macro). Software to compute the estimators proposed in this article is available at the authors web-pages.
Estimating Binary Dependent Variable Models Under Conditions of Specification Uncertainty
Berry, William, DeMeritt, Jacqueline, Esarey, Justin
Submitted: 2007-01-25
Keywords: logit, probit, binary dependent variable, specification uncertainty, interaction, Monte Carlo analysis
Abstract: (click to show/hide) Political scientists routinely use logit or probit models when their data involve binary dependent variables (BDVs). Yet the hypotheses we test with logit and probit are rarely specific enough to justify that one of these models is the correct functional form for the process (or true model) generating the data. In this situation of specification uncertainty, it is reasonable to assume that the model being estimated is misspecified. The only issue is the severity of the resulting distortion in results, i.e., whether logit or probit approximates the true model well enough to yield estimated effects that are acceptably close to the true ones. To study estimation in the presence of specification uncertainty, we conduct Monte Carlo analysis using a strategy of purposeful misspecification: we use various logit and probit models with different terms on data sets generated from a wide range of known true models involving a BDV, none of which takes the exact form of a logit or probit model. We find that a widely-employed approach for using logit or probit to test the hypothesis that an independent variable has a positive (or negative) effect on the probability that some event will occur-­by estimating the effect of the variable at central values of the independent variables­-is highly forgiving of specification uncertainty, yielding reasonably accurate inferences even when the true model is not logit or probit. Unfortunately, other applications of logit and probit­-including a common approach to testing a hypothesis that independent variables interact in influencing the probability of event occurrence­-are not nearly as forgiving of the uncertainty. In some situations of specification uncertainty, we can improve the quality of estimated effects by relying on the Akaike Information Criterion [AIC] to choose the terms to be included in a model, but even these improved estimates leave much to be desired.
Verifying Evidence of "Congressional Enactments of Race-Gender"
Grant, J. Tobin
Submitted: 2007-02-05
Keywords: replication, verification, interpretive methodology, qualitative methods, race, gender, Congress
Abstract: (click to show/hide) I report the results of a verification of Hawkesworth's 2003 "Congressional Enactments of Race-Gender" (CERG). This is a landmark analysis of race and gender in the U.S. Congress that is noteworthy for both its theory and its empirical evidence. A deeper look at the evidence and the context raises fundamental questions about the empirical validity of CERG's theory of race-gender in Congress. I conclude that racing-gendering in Congress is more nuanced than originally presented in CERG, and that further research is necessary to demonstrate empirically CERG's theory of Congress as a raced-gendered institution. This verification has important methodology implications, as it demonstrates why verification of empirical research -- including interpretive research -- should be a widely-practiced methodology within political science.
Authoritarian Reversals and Democratic Consolidation
Svolik, Milan
Submitted: 2007-02-21
Keywords: democratic consolidation, transitions to democracy, split-population models, cure rate models, mixture models
Abstract: (click to show/hide) I investigate the determinants and the process of authoritarian reversals and democratic consolidation. I employ a new empirical model that allows me to distinguish between two central dynamics: the likelihood that a democracy consolidates, and the timing of authoritarian reversals in democracies that are not consolidated. I demonstrate that existing democracies are a mixture of transitional and consolidated democracies rather than a single population. This approach leads to new insights into the causes of democratic consolidation that cannot be obtained with existing techniques. I find that the level of economic development, type of executive, and authoritarian past determine whether a democracy consolidates, but have no effect on the timing of reversals. That risk is only associated with economic recessions. I also find that the existing studies greatly underestimate the risk of early reversals while they simultaneously overestimate the risk of late reversals, and that a large number of existing democracies are in fact consolidated.
Opium for the Masses: How Foreign Media Can Stabilize Authoritarian Regimes
Kern, Holger, Hainmueller, Jens
Submitted: 2007-04-11
Keywords: instrumental variables, causal inference, local average response function, LATE, media effects, East Germany, democratization, regime legitimacy
Abstract: (click to show/hide) In this case study of the impact of West German television on public support for the East German communist regime, we evaluate the conventional wisdom in the democratization literature that foreign mass media undermine authoritarian rule. We exploit formerly classified survey data and a natural experiment to identify the effect of foreign media exposure using instrumental variable estimators. Contrary to conventional wisdom, East Germans exposed to West German television were more satisfied with life in East Germany and more supportive of the East German regime. To explain this surprising finding, we show that East Germans used West German television primarily as a source of entertainment. Behavioral data on regional patterns in exit visa applications and archival evidence on the reaction of the East German regime to the availability of West German television corroborate this result.
Understanding Wordscores
Lowe, Will
Submitted: 2007-04-25
Keywords: content analysis, wordscores, ideal point, item response theory
Abstract: (click to show/hide) Wordscores is a widely-used procedure for inferring policy positions, or scores, for new documents on the basis of scores for words derived from documents with known scores. It is computationally straightforward, requires no distributional assumptions, but has unresolved practical and theoretical problems: In applications, estimated document scores are on the wrong scale and Wordscores does not specify a statistical model so it is unclear what assumptions the method makes about political text or how to tell whether they fit particular applications. The first part of the paper demonstrates that badly scaled document score estimates reflect deeper problems with the method. The second part shows how to understand Wordscores as an approximation to correspondence analysis which itself approximates a statistical ideal point model for words. Problems with the method are identified with the conditions under which these layers of approximation fail to ensure consistent and unbiased estimation of the parameters of the ideal point model.
Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential?
Berry, William, Esarey, Justin, DeMeritt, Jacqueline
Submitted: 2007-05-06
Keywords: interaction, logit, probit
Abstract: (click to show/hide) Political scientists presenting binary dependent variable (BDV) models often offer hypotheses that independent variables interact in their influence on the probability that an event Y occurs, Pr(Y). A consensus appears to have evolved on how to test such hypotheses: (i) estimate a logit or probit model including product terms to specify the interaction, (ii) test the hypothesis by determining whether the coefficients for these terms are statistically significant, and (iii) if they are, describe the nature of the interaction by estimating how the marginal effect of one independent variable on Pr(Y) varies with the value of the other independent variables. We contend that in the BDV context, statistically significant product term coefficients are neither necessary nor sufficient for concluding that there is substantively meaningful interaction among variables in their influence on Pr(Y). Even when no product terms are included in a logit or probit model, if the marginal effect of one variable on Pr(Y) is related to another independent variable then substantively meaningful interaction is present, and describing such interaction is essential to an accurate portrayal of the data generating process at work. We propose a strategy for studying interaction in the BDV context that is consistent with the recent emphasis in the discipline on casting hypotheses in terms of effects on the probability of an event's occurrence and reporting estimated marginal effects on this probability.
Direct Democracy and Social Issues
Matsusaka, John
Submitted: 2007-05-29
Keywords: Direct democracy, initiative, social issues, representation
Abstract: (click to show/hide) This paper explores the connection between the initiative process -- the most potent form of direct democracy -- and social issues by examining laws on seven social issues in all 50 American states. Initiative states are 18 percent more likely than noninitiative states to choose a conservative than a liberal policy on the median issue after controlling for public opinion, demographic, and regional variables. The conservative shift is majoritarian: initiative states are 8 percent more likely than noninitiative states to choose laws that reflect the majority's preference. The initiative effect does not appear to depend on the institutional features that scholars and reformers often discuss.
Direct Democracy and Public Employees
Matsusaka, John
Submitted: 2007-05-29
Keywords: Direct democracy, public employees, initiative, patronage, interest groups
Abstract: (click to show/hide) In the public sector, employment may be inefficiently high because of patronage, and wages may be inefficiently high because of the strength of public employee interest groups. This paper explores whether the initiative process, a direct democracy institution of growing importance, can control these political economy problems, as proponents and some research suggests. Based on a sample of 500+ cities in 2000, I find that when public employees are allowed to bargain collectively, driving up wages, the initiative appears to cut wages by about 5 percent but has no measurable effect on employment. When public employees are not allowed to bargain collectively and patronage is a problem, initiatives appear to cut employment but not wages.
Power-law distributions in empirical data
Clauset, Aaron, Shalizi, Cosma, Newman, Mark
Submitted: 2007-06-11
Keywords: Power-law distributions, Pareto, Zipf, maximum likelihood, heavy-tailed distributions, likelihood ratio test, model selection
Abstract: (click to show/hide) Power-law distributions occur in many situations of scientific interest and have significant consequences for our understanding of natural and man-made phenomena. Unfortunately, the empirical detection and characterization of power laws is made difficult by the large fluctuations that occur in the tail of the distribution. In particular, standard methods such as least-squares fitting are known to produce systematically biased estimates of parameters for power-law distributions and should not be used in most circumstances. Here we describe statistical techniques for making accurate parameter estimates for power-law data, based on maximum likelihood methods and the Kolmogorov-Smirnov statistic. We also show how to tell whether the data follow a power-law distribution at all, defining quantitative measures that indicate when the power law is a reasonable fit to the data and when it is not. We demonstrate these methods by applying them to twenty-four real-world data sets from a range of different disciplines. Each of the data sets has been conjectured previously to follow a power-law distribution. In some cases we find these conjectures to be consistent with the data while in others the power law is ruled out.
Splitting a predictor at the upper quarter or third and the lower quarter or third
Gelman, Andrew, Park, David
Submitted: 2007-07-06
Keywords: discretization, linear regression, statistical communication, trichotomizing
Abstract: (click to show/hide) A linear regression of $y$ on $x$ can be approximated by a simple difference: the average values of $y$ corresponding to the highest quarter or third of $x$, minus the average values of $y$ corresponding to the lowest quarter or third of $x$. A simple theoretical analysis shows this comparison performs reasonably well, with 80\%--90\% efficiency compared to the linear regression if the predictor is uniformly or normally distributed. Discretizing $x$ into three categories claws back about half the efficiency lost by the commonly-used strategy of dichotomizing the predictor. We illustrate with the example that motivated this research: an analysis of income and voting which we had originally performed for a scholarly journal but then wanted to communicate to a general audience.
Partisans without constraint: Political polarization and trends in American public opinion
Gelman, Andrew, Baldassarri, Delia
Submitted: 2007-07-06
Keywords: issue alignment, partisanship, polarization
Abstract: (click to show/hide) Political polarization is commonly measured using the variation of responses on an individual issue. By this measure, research has shown that---despite many commentators' concerns about increased polarization---Americans' attitudes have become no more variable in recent decades. What has changed in the electorate is its level of partisanship. We define a new measure of political polarization as increased correlations in political attitudes and we distinguish between issue partisanship---the correlation of issue attitudes with party ID or ideology---and issue alignment---the correlation between pairs of issues. Using the National Election Studies (1972-2004), we find issue alignment to have increased by only 2 percentage points in correlation per decade. Issue partisanship has increased more than twice as fast, thus suggesting that changes in people's attitudes correspond more to a re-sorting of party labels among voters than to greater constraint on issue attitudes. Since parties are more polarized, they are now better at sorting individuals along ideological lines. Increased issue partisanship, in a context of persistently low issue constraint, might give greater voice to political extremists and single-issue advocates, and amplify dynamics of unequal representation.
Potential Ambiguities in a Directed Dyad Approach to State Policy Emulation
Boehmke, Frederick
Submitted: 2007-07-10
Keywords: state politics, state policy, diffusion, emulation, monte carlo, health policy, dyadic
Abstract: (click to show/hide) In this paper I discuss circumstances under which the dyadic model of policy diffusion can produce misleading estimates in favor of policy emulation. These circumstances arise in the context of state pain management policy, and correspond generally to policies that states are uniformly expanding. When this happens, dyadic models of policy diffusion conflate policy emulation and policy adoption: since early adopters are policy leaders, later adopters will appear to emulate them, even if they are merely stragglers acting on their own accord. I demonstrate the possibility of this ambiguity analytically and through Monte Carlo simulation. Both start with the assumption that the data are generated according to a standard, monadic model of policy adoption and then converted to a dyadic model, which can incorrectly produce evidence of emulation. I propose a simple modification of the dyadic emulation model --- conditioning on the opportunity to emulate --- and show that it is much less likely to produce inaccurate findings. I then return to the study of pain management policy and find substantial differences between the dyadic emulation model and the conditional emulation model.
Strategic Interaction and Interstate Crises: A Fixed-Effects Bayesian Quantal Response Estimator for Incomplete Information Games
Esarey, Justin, Mukherjee, Subhanan, Moore, Will
Submitted: 2007-07-12
Keywords: fixed effects, quantal response, crisis bargaining, EITM
Abstract: (click to show/hide) Two strategies have been laid out by a growing literature on how to properly test the hypotheses implied by a theory of strategic interaction. The first strategy focuses on conventional comparative statics and the proper specification of standard statistical models (OLS, logit or probit). The second strategy requires deriving a novel likelihood function directly from the model or theory and estimating the parameters with maximum likelihood or Bayesian methods. Both approaches have largely limited their attention to games of perfect information, though many important phenomena are studied using games of incomplete information. This study develops a statistical model for incomplete information games that we term the Fixed Effects Bayesian Quantal Response Model. Our FE-BQRE model, which lies in the domain of the second strategy, offers three advantages over existing efforts: it directly incorporates (i) Bayesian updating and (ii) signaling dynamics, and (iii) it mimics the temporal learning process that we believe takes place in international politics.
Extracting Systematic Social Science Meaning from Text
Hopkins, Daniel, King, Gary
Submitted: 2007-07-12
Keywords: automated content analysis, machine learning, simulated extrapolation, non-parametric estimation, internet, 2008 U.S. Presidential election
Abstract: (click to show/hide) We develop two methods of automated content analysis that give approximately unbiased estimates of quantities of theoretical interest to social scientists. With a small sample of documents hand coded into investigator-chosen categories, our methods can give accurate estimates of the proportion of text documents in each category in a larger population. Existing methods successful at maximizing the percent of documents correctly classified allow for the possibility of substantial estimation bias in the category proportions of interest. Our first approach corrects this bias for any existing classifier, with no additional assumptions. Our second method estimates the proportions without the intermediate step of individual document classification, and thereby greatly reduces the required assumptions. For both methods, we also correct statistically, apparently for the first time, for the far less-than-perfect levels of inter-coder reliability that typically characterize human attempts to classify documents, an approach that will normally outperform even population hand coding when that is feasible. We illustrate these methods by tracking the daily opinions of millions of people about candidates for the 2008 presidential nominations in online blogs, data we introduce and make available with this article, and through evaluations in available corpora from other areas, including movie reviews, university web sites, and Enron emails. We also offer easy-to-use software that implements all methods described.
Modeling Foreign Direct Investment as a Longitudinal Social Network
Jensen, Nathan, Martin, Andrew, Westveld, Anton
Submitted: 2007-07-13
Keywords: foreign direct investment, social network data, longitudinal data, hierarchical modeling, mixture modeling, Bayesian inference.
Abstract: (click to show/hide) An extensive literature in international and comparative political economy has focused on the how the mobility of capital affects the ability of governments to tax and regulate firms. The conventional wisdom holds that governments are in competition with each other to attract foreign direct investment (FDI). Nation-states observe the fiscal and regulatory decisions of competitor governments, and are forced to either respond with policy changes or risk losing foreign direct investment, along with the politically salient jobs that come with these investments. The political economy of FDI suggests a network of investments with complicated dependencies. We propose an empirical strategy for modeling investment patterns in 24 advanced industrialized countries from 1985-2000. Using bilateral FDI data we estimate how increases in flows of FDI affect the flows of FDI in other countries. Our statistical model is based on the methodology developed by Westveld & Hoff (2007). The model allows the temporal examination of each notion's activity level in investing, attractiveness to investors, and reciprocity between pairs of nations. We extend the model by treating the reported inflow and outflow data as independent replicates of the true value and allowing for a mixture model for the fixed effects portion of the network model. Using a fully Bayesian approach, we also impute missing data within the MCMC algorithm used to fit the model.
Non-parametric Mechanisms and Causal Modeling
Glynn, Adam, Quinn, Kevin
Submitted: 2007-07-15
Keywords: Neyman-Rubin model, non-parametric structural equations, causal inference, covariate selection, unmeasured confounding
Abstract: (click to show/hide) Political scientists tend to think about causality in terms of mechanisms. In this paper we argue that non-parametric structural equation models are consistent with how many empirical political scientists think about causality and are consistent with the powerful and well-respected Neyman-Rubin Causal Model. Furthermore, using examples we demonstrate that two important practical questions are more easily addressed within the mechanistic framework: What (if any) set or sets of conditioning variables will allow the identification of average causal effects in a regression or matching model? When unmeasured confounding is present, what (if any) adjustment will non-parametrically identify the average causal effect?
Bayesian Analysis of Structural Changes: Historical Changes in US Presidential Uses of Force Abroad
Park, Jong Hee
Submitted: 2007-07-16
Keywords: structural changes, changepoint models, discrete time series data, use of force data, state space models, time-varying parameter models, Bayesian inference
Abstract: (click to show/hide) While many theoretical models in political science are inspired by structural changes in politics, most empirical methods assume stable patterns of causal processes and fail to capture dynamic changes in theoretical relationships. In this paper, I introduce an efficient Bayesian approach to the multiple changepoint problem presented by Chib (1998) and discuss the utility of the Bayesian changepoint models in the context of generalized linear models. As an illustration, I revisit the debate over whether and how U.S. presidents have used forces abroad in response to domestic factors since 1890.
Data Experiments: Model Specifications as Treatments
Clarke, Kevin A.
Submitted: 2007-07-16
Keywords: Nonnested model testing, randomized complete block designs, nonparametrics
Abstract: (click to show/hide) This paper introduces the first model discrimination test for three or more competing nonnested models. The key to the approach is treating rival model specifications as experimental treatments applied to a set of observations. Viewed in this way, competing specifications can be tested using techniques designed for analyzing multiple related samples. To this end, two such procedures are adapted for the purpose of comparing three or more nonnested model specifications. The models to be compared may be nonnested either in their covariates or in their functional forms. The tests are straightforward and can be implemented using standard statistical software. The usefulness of the tests is demonstrated through real-world applications drawn from comparative politics and international relations.
The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation
Imai, Kosuke, King, Gary, Nall, Clayton
Submitted: 2007-07-17
Keywords: causal inference, community intervention trials, field experiments, group-randomized trials, health policy, matched-pair design, noncompliance
Abstract: (click to show/hide) A basic feature of many field experiments is that investigators are only able to randomize clusters of individuals -- such as households, communities, firms, medical practices, schools, or classrooms -- even when the individual is the unit of interest. To recoup some of the resulting efficiency loss, many studies pair similar clusters and randomize treatment within pairs. Other studies (including almost all published political science field experiments) avoid pairing, in part because some prominent methodological articles claim to have identified serious problems with this 'matched-pair cluster-randomized' design. We prove that all such claims about problems with this design are unfounded. We then show that the estimator for matched-pair designs favored in the literature is appropriate only in situations where matching is not needed. To address this problem without modeling assumptions, we generalize Neyman's (1923) approach and propose a simple new estimator with much improved statistical properties. We also introduce methods to cope with individual-level noncompliance, which most existing approaches incorrectly assume away. We show that from the perspective of, among other things, bias, efficiency, or power, pairing should be used in cluster-randomized experiments whenever feasible; failing to do so is equivalent to discarding a considerable fraction of one's data. We develop these techniques in the context of a randomized evaluation we are conducting of the Mexican Universal Health Insurance Program.
Inductive Event Data Scaling using Item Response Theory
Schrodt, Philip A.
Submitted: 2007-07-17
Keywords: event data, IRT, latent trait, scaling, Rasch model, Goldstein scale, WEIS, CAMEO
Abstract: (click to show/hide) Political event data are frequently converted to an interval-level measurement by assigning a numerical scaled value to each event. All of the existing scaling systems rely on non-replicable expert assessments to determine these numerical scores. This paper uses item response theory (IRT) to derive scales inductively, using event data on Israeli interactions with Lebanon and the Palestinians for 1991-2007. Monthly scores on a latent trait are calculated using three IRT models: the single-parameter Rasch model, and two-parameter models that add discrimination and guessing parameters. The three formulations produce generally comparable scores (correlations of 0.90 or higher). The Rasch scales are less successful than the expert-derived Goldstein scale in reconciling the somewhat divergent sets of events derived from the Agence France Presse and Reuters news services. This is in all likelihood due largely to a low weighting given uses of force by the IRT because such events are common in these two dyads. A factor analysis of the event counts shows that a single cooperation-conflict dimension generally accounts for about two-thirds of the variance in these dyads, but a second case-specific dimension explains another 20%. Finally, moving averages of the derived scores generally correlate well with the Goldstein values, suggesting that IRT may provide a route towards deriving purely inductive, and hence replicable, scales.
Statistics for Digits
Mebane, Walter
Submitted: 2007-07-17
Keywords: election forensics, 2BL test, Benford's Law, vote counts, outliers, anomalies, election fraud
Abstract: (click to show/hide) I show how election results may be used to calibrate a test that compares the second digits of a set of precinct-level vote counts to the frequencies expected according to Benford's law. For the votes cast for two competing candidates, the calibration is accomplished by tuning a simulation mechanism that mixes normal and negative binomial distributions so that the first two moments of the simulated distribution match the moments observed in a set of precincts. I illustrate the method using data from the counties that had the ten largest values of the digit test statistic for the major party candidates in the 2000 and 2004 U.S. presidential election. Calibration suggests that the peculiar features of the joint distribution of candidate support and precinct sizes explain several of the large test statistic values. I show that artificial manipulations can significantly increase the test statistic's value even relative to the increased distribution the tuned mechanism is producing. So the test can sometimes detect systematic distortions in vote counts even when the baseline mechanism does not produce counts that have digits that are distributed as specified by Benford's law.
Models of Path Dependence with an Empirical Application
Jackson, John, Kollman, Ken
Submitted: 2007-07-17
Keywords: Path dependence, partisanship, non-linear least squares
Abstract: (click to show/hide) It is now commonplace in the social sciences to describe an outcome or process as path dependent. By path dependence, researchers generally mean that the sequence of events prior to the observation of the outcome has explanatory power. The paper develops models that have both path dependent and non-path dependent properties, depending upon the value of a particular parameter. The paper then uses non-linear least squares and a Monte Carlo simulation to explore how well this parameter can be estimated, meaning how well scholars can discriminate betwen the two processes. The methodology is applied to the evolution of attitudes on aid to minorities and partisanship between 1956 and 2000. The results are consistent with the path dependent model.
Back to the Future: Modeling Time Dependence in Binary Data
Carter, David, Signorino, Curtis S.
Submitted: 2007-07-17
Keywords:
Abstract: (click to show/hide) Since Beck, Katz, and Tucker (1998), the use of time dummies or splines has become the standard method to model temporal dependence in binary data. There are potential problems with both of these approaches, especially in the case of time dummies. We propose a simpler alternative: using t, t^2, and t^3 to approximate the hazard. This cubic polynomial is trivial to implement and avoids problems with time dummies such as quasi-complete separation and issues with splines such as interpretation or knot selection. It also accommodates non-proportional hazards in a more straightforward way than either time dummies or splines. Monte Carlo analysis and reanalysis of numerous published empirical results are used to show that our method performs as well as splines and better than time dummies. Non-proportional hazards are also simple to model with a cubic polynomial. We present new results with data from Crowley and Skocpol (2001) to demonstrate how to model and interpret a non-proportional hazard.
Variance Identification and Efficiency Analysis in Randomized Experiments under the Matched-Pair Design
Imai, Kosuke
Submitted: 2007-07-17
Keywords: Average Treatment Effect, Causal Inference, Experimental Design, Matched Samples, Paired Comparison, Randomization Inference.
Abstract: (click to show/hide) In his landmark article, Neyman (1923) introduced randomization-based inference in analyzing experiments under the completely randomized design. Under this framework, Neyman considered the statistical estimation of the sample average treatment effect and derived the variance of the standard estimator using the treatment assignment mechanism as the sole basis of inference. In this paper, I extend Neyman's analysis to randomized experiments under the matched-pair design where experimental units are paired based on their pre-treatment characteristics and the randomization of treatment is subsequently conducted within each matched pair. I study the variance identification for the standard estimator of average treatment effects and analyze the relative efficiency of the matched-pair design over the completely randomized design. I also show how to empirically evaluate the relative efficiency of the two designs using experimental data obtained under the matched-pair design. My randomization-based analysis clarifies some of the important questions raised in the literature and identifies a hiden and yet implausible assumption that is made for the efficiency analysis in a widely used textbook. Finally, the analytical results are illustrated with numerical and empirical examples.
Path, Phat, and State Dependence in Observation-driven Markov
Walker, Robert
Submitted: 2007-07-17
Keywords: Markov models, qualitative time series, ergodic theorem
Abstract: (click to show/hide) Many social science theories posit dynamics that depend in important ways on the present state and focus on a reasonably small number of states. Despite the importance of theoretical notions of path dependence, empirical models, with a few exceptions (Alvarez, Cheibub, Limongi and Przewroski 2000; Epstein, Bates, Goldstein, O'Halloryn, and Kristensen 2006; Beck. Jackman, Epstein, and O'Halloryn 2001), have paid little attention to the implications of state dependence for empirical studies. This despite the fact that there are many possible ways in which history might matter -- we focus on the categorization given by Page (2006) -- and these different ways that history might matter manifest themselves in sets of models that can be tested and compared. This paper considers the basic properties of observation-driven Markov chains [stationarity/time homogeneity, communication, transience, periodicity, irreducibility, and ergodicity] and the issues that arise in their implementation as likelihood estimators to provide a window into methods for the study of path dependence. Application of these concepts to longitudinal data on human rights abuses and exchange rate regime transitions provides evidence that history may also not exert uniform effects. The empirical examples highlight the subtle substantive assumptions that manifest in different modeling choices. The human rights example calls for an important qualification in the widely studied relationship between democracy and human rights abuses. The exchange rate regime example highlights the usefulness of Markov models for multinomial processes.
Bargaining and Society: A Statistical Model of the Ultimatum Game
Signorino, Curtis
Submitted: 2007-07-18
Keywords: bargaining, random utility models, strategic, ultimatum, game, equilibrium, stochastic
Abstract: (click to show/hide) In this paper we derive a statistical estimator for the popular Ultimatum bargaining game. Using monte carlo data generated by a strategic bargaining process, we show that the estimator correctly recovers the relationship between dependent variables, such as the proposed division and bargaining failure, relative to substantive variables that comprise players' utilities. We then use the model to analyze bargaining data in a number of contexts. The current example examines the effects of demographics on bargaining behavior in experiments conducted on U.S. and Russian participants.
The Spatial Probit Model of Interdependent Binary Outcomes: Estimation, Interpretation, and Presentation
Franzese, Robert, Hays, Jude
Submitted: 2007-07-20
Keywords: Spatial Probit, Bayesian Gibbs-Sampler Estimator, Recursive Importance-Sampling Estimator, Interdependence, Diffusion, Contagion, Emulation
Abstract: (click to show/hide) We have argued and shown elsewhere the ubiquity and prominence of spatial interdependence in political science research and noted that much previous practice has neglected this interdependence or treated it solely as nuisance to the serious detriment of sound inference. Previously, we considered only linear-regression models of spatial and/or spatio-temporal interdependence. In this paper, we turn to binary-outcome models. We start by stressing the ubiquity and centrality of interdependence in binary outcomes of interest to political and social scientists and note that, again, this interdependence has been ignored in most contexts where it likely arises and that, in the few contexts where it has been acknowledged, the endogeneity of the spatial lag has not be recognized. Next, we explain some of the severe challenges for empirical analysis posed by spatial interdependence in binary-outcome models, and then we follow recent advances in the spatial-econometric literature to suggest Bayesian or recursive-importance-sampling (RIS) approaches for tackling estimation. In brief and in general, the estimation complications arise because among the RHS variables is an endogenous weighted spatial-lag of the unobserved latent outcome, y*, in the other units; Bayesian or RIS techniques facilitate the complicated nested optimization exercise that follows from that fact. We also advance that literature by showing how to calculate estimated spatial effects (as opposed to parameter estimates) in such models, how to construct confidence regions for those (adopting a simulation strategy for the purpose), and how to present such estimates effectively.
A default prior distribution for logistic and other regression models
Gelman, Andrew, Jakulin, Aleks, Pittau, Maria Grazia, Su, Yu-Sung
Submitted: 2007-08-03
Keywords: Bayesian inference, generalized linear model, least squares, hierarchical model, linear regression, logistic regression, multilevel model, noninformative prior distribution
Abstract: (click to show/hide) We propose a new prior distribution for classical (non-hierarchical) logistic regression models, constructed by first scaling all nonbinary variables to have mean 0 and standard deviation 0.5, and then placing independent Student-$t$ prior distributions on the coefficients. As a default choice, we recommend the Cauchy distribution with center 0 and scale 2.5, which in the simplest setting is a longer-tailed version of the distribution attained by assuming one-half additional success and one-half additional failure in a logistic regression. We implement a procedure to fit generalized linear models in R with this prior distribution by incorporating an approximate EM algorithm into the usual iteratively weighted least squares. We illustrate with several examples, including a series of logistic regressions predicting voting preferences, an imputation model for a public health data set, and a hierarchical logistic regression in epidemiology. We recommend this default prior distribution for routine applied use. It has the advantage of always giving answers, even when there is complete separation in logistic regression (a common problem, even when the sample size is large and the number of predictors is small) and also automatically applying more shrinkage to higher-order interactions. This can be useful in routine data analysis as well as in automated procedures such as chained equations for missing-data imputation.
Estimating Party Policy Positions with Uncertainty Based on Manifesto Codings
Benoit, Kenneth, Laver, Michael, Mikhaylov, Slava
Submitted: 2007-08-21
Keywords: Comparative Manifesto Project, Mapping party positions, party policy, error estimates, measurement error
Abstract: (click to show/hide) Spatial models of party competition are central to modern political science. Before we can elaborate such models empirically, we need reliable and valid measurements of agents' positions on salient policy dimensions. The primary empirical times series of estimated party positions in many countries derives from the content analysis of party manifestos by the Comparative Manifesto Project (CMP). Despite widespread use of the CMP data, and despite the fact that estimates in these data arise from documents coded once, and once only, by a single human researcher, the level of error in the CMP estimates has never been estimated or even fully characterized. This greatly undermines the value of the CMP dataset as a scientific resource. It is in many ways remarkable that so much has been published in the best professional journals using data that almost certainly has substantial, but completely uncharacterized, error. We remedy this situation. We outline the process of generating CMP document codings and positional estimates. Error in this process arises, not only from the obvious source of coder unreliability, but also from fundamental variability in the stochastic process by which latent party positions are translated into observable manifesto texts. Using the quasi-sentence codings from the CMP project, we reproduce the error-generating process by simulating coder unreliability and bootstrapping analyses of coded quasi-sentences to reproduce both forms of error. Using our estimates of these errors, we suggest and demonstrate ways to correct otherwise biased inferences derived from statistical analyses of the CMP data.
Sharp Bounds on the Causal Effects in Randomized Experiments with ``Truncation-by-Death''
Imai, Kosuke
Submitted: 2007-08-23
Keywords: Average treatment effect, Causal inference, Direct and indirect effect, Identification, Principal stratification, Quantile treatment effect.
Abstract: (click to show/hide) Many randomized experiments suffer from the ``truncation-by-death'' problem where potential outcomes are not defined for some subpopulations. For example, in medical trials, quality-of-life measures are only defined for surviving patients, and various skip-pattern questions are analyzed in social science survey experiments. In this paper, I derive the sharp bounds on causal effects under various assumptions. My identification analysis is based on the idea that the ``truncation-by-death'' problem can be formulated as the contaminated data problem. The proposed analytical techniques can be applied to other settings in causal inference including the estimation of direct and indirect effects and the analysis of three-arm randomized experiments with noncompliance.
Misunderstandings among Experimentalists and Observationalists about Causal Inference
Imai, Kosuke, King, Gary, Stuart, Elizabeth
Submitted: 2007-09-16
Keywords: matching, blocking, causal inference, experimental design, observational studies, average treatment effects, covariate balance, field experiments, survey experiments
Abstract: (click to show/hide) We attempt to clarify, and suggest how to avoid, several serious misunderstandings about and fallacies of causal inference in experimental and observational research. These issues concern some of the most basic advantages and disadvantages of each basic research design. Problems include improper use of hypothesis tests for covariate balance between the treated and control groups, and the consequences of using randomization, blocking before randomization, and matching after treatment assignment to achieve covariate balance. Applied researchers in a wide range of scientific disciplines seem to fall prey to one or more of these fallacies, and as a result make suboptimal design or analysis choices. To clarify these points, we derive a new four-part decomposition of the key estimation errors in making causal inferences. We then show how this decomposition can help scholars from different experimental and observational research traditions better understand each other's inferential problems and attempted solutions. (This paper is forthcoming in the Journal of the Royal Statistical Society, but we have some time for revisions and would value any comments anyone might have. This is a revised and much more general version of an earlier paper, "The Balance Test Fallacy in Causal Inference".)
An Introduction to the Dataverse Network as an Infrastructure for Data Sharing
King, Gary
Submitted: 2007-09-16
Keywords: data sharing, replication, citation, analysis, archiving, preservation, informatics
Abstract: (click to show/hide) We introduce a set of integrated developments in web application software, networking, data citation standards, and statistical methods designed to increase scholarly recognition for data contributions; put some of the universe of data and data sharing practices on firmer ground; and facilitate the public distribution of persistent, authorized, and verifiable data, with powerful and easy-to-use technology, even when the data are confidential or proprietary. Our goal is to solve some of the political and sociological problems of data sharing via technological means, with the result intended to benefit both the scientific community and the sometimes apparently contradictory goals of individual researchers. (More information on this project is available at http://TheData.org.)
The political consequences of transitions out of marriage in Great Britain
Kern, Holger
Submitted: 2007-11-20
Keywords: causal inference, matching, Great Britain, marriage, divorce, widowhood, turnout
Abstract: (click to show/hide) This paper uses British Household Panel Survey data to estimate the effects of divorce and widowhood on political attitudes and political behavior. In contrast to previous research, which mostly relied on cross-sectional data, a matched propensity score analysis does not find any effects of transitions out of marriage on policy preferences, party identification, and vote choice. The results also show that divorce (but not widowhood) substantially reduces electoral participation. Some preliminary evidence suggests that this effect of divorce on turnout is partially attributable to the increased residential mobility that accompanies divorce.
Two Genes Predict Voter Turnout
Fowler, James, Dawes, Christopher
Submitted: 2007-11-26
Keywords:
Abstract: (click to show/hide) Fowler, Baker, and Dawes (2007) recently showed in two independent studies of twins that voter turnout has very high heritability. Here we investigate two specific genes that may contribute to this heritability via their impact on neurochemical processes that influence social behavior. Using data from the National Longitudinal Study of Adolescent Health, we show that a polymorphism of the MAOA gene significantly increases the likelihood of voting. We also find evidence of a gene-environment interaction between religious attendance and a polymorphism of the 5HTT gene that significantly increases voter turnout. These are the first results to ever link specific genes to political behavior and they suggest that political scientists should take seriously the claim that at least some variation in political behavior is due to innate predispositions.
Polity by Design; an engineering approach
Kwatra, Saurabh
Submitted: 2007-12-12
Keywords: Genuine Political Engineering, Design Methodologies, Interdisciplinary, Governance
Abstract: (click to show/hide) Rules by which societies govern themselves are called institutions. Institutions can be political, economic, social, but generally they are a complex combination of these. Universities and Academies of higher education include a course or paper titled 'Political Engineering'; as reflected in the title, some kind of so-called engineering is applied to political science. The phrase 'so-called' has been used with intent and some disgrace is associated with it. This paper, while justifying these adjectives, is categorically the first bold attempt to apply genuine engineering practice to political science. Till date, so-called Political Engineering as taught and done, uses tools of economic theory, game theory, social-choice theory and formal logic to both understand (analyze) and create (synthesize) institutions. The choice of word engineering is a misnomer as the word is used very loosely, almost to the extent of disrespect for it. Still, institutions designed by using this loose engineering meet certain technical specifications and are therefore undoubtedly superior to their haphazardly evolved cousins. Just like genuine industrial design engineering when applied to medical technology translates an advanced momentum exchange theory to manufacture a regenerative flow-type blood pump, (pseudo) political-social-economic engineering translates its rational-choice-analysis (probably with some inbuilt equal opportunity axiom) into an Internet-enabled stepped-fashion Auctions (a modern bargaining and arbitration Procedure). This paper deals with the application of genuine engineering design methodologies to political institutions; it should be termed as honest political engineering. As an example of genuine political engineering exercise, consider an analogy between the physical world of technologists and the ruled society in democracy. Compare the problem of 'working out' the ideal area of a road-roller for resurfacing a highway of a given area most efficiently with the problem of 'computing' the size of House of Commons (number of elected representatives) required to democratically govern the given size of British population with optimum efficiency. The developed software, 'political machinery' solves these problems, taking input parameters like, population to be governed, per capita income, the kind of economy, environmental favor-ability, corruption level and statistical figures of previous successes or failures of earlier governments. The alternative analysis-synthesis approach, as is done in reverse engineering, would be performed by the program - number of Members of Parliament being the numeric output! Perhaps the coiner of the term, Political Machinery was a machine designer! I wait when parliamentarians and policy-makers are replaced by engineer-turned-designers.