Working Papers
2006
43 records found
Should Voters be Encyclopedias? Measuring the Political Sophistication of Survey Respondents
Lawrence, Christopher
Submitted: 2006-12-23
Keywords: political sophistication, public opinion, item-response theory models, political knowledge, measurement, NES, DPES
Abstract: (click to show/hide) In this paper, I apply item-response theory models to the problem of measuring the political sophistication of survey respondents in the United States and the Netherlands, discuss the advantages of IRT models over traditional measurement techniques (additive indices, interviewer evaluations) for second-stage analysis, and demonstrate the construct validity of the IRT-based measures. I also demonstrate the relative performance of knowledge items and items constructed from party/candidate relative placement questions on both the NES and DPES.
Lawrence, Christopher
Submitted: 2006-12-23
Keywords: political sophistication, public opinion, item-response theory models, political knowledge, measurement, NES, DPES
Abstract: (click to show/hide) In this paper, I apply item-response theory models to the problem of measuring the political sophistication of survey respondents in the United States and the Netherlands, discuss the advantages of IRT models over traditional measurement techniques (additive indices, interviewer evaluations) for second-stage analysis, and demonstrate the construct validity of the IRT-based measures. I also demonstrate the relative performance of knowledge items and items constructed from party/candidate relative placement questions on both the NES and DPES.
Bayesian and Likelihood Inference for 2 x 2 Ecological Tables: An Incomplete Data Approach
Imai, Kosuke, Lu, Ying, Strauss, Aaron
Submitted: 2006-12-16
Keywords: Coarse data, Contextual effects, Data augmentation, EM algorithm, Missing information principle, Nonparametric Bayesian Modeling.
Abstract: (click to show/hide) Ecological inference is a statistical problem where aggregate-level data are used to make inferences about individual-level behavior. Recent years have witnessed resurgent interest in ecological inference among political methodologists and statisticians. In this paper, we conduct a theoretical and empirical study of Bayesian and likelihood inference for 2 x 2 ecological tables by applying the general statistical framework of incomplete data. We first show that the ecological inference problem can be decomposed into three factors: distributional effects which address the possible misspecification of parametric modeling assumptions about the unknown distribution of missing data, contextual effects which represent the possible correlation between missing data and observed variables, and aggregation effects which are directly related to the loss of information caused by data aggregation. We then examine how these three factors affect inference and offer new statistical methods to address each of them. To deal with distributional effects, we propose a nonparametric Bayesian model based on a Dirichlet process prior which relaxes common parametric assumptions. We also specify the statistical adjustments necessary to account for contextual effects. Finally, while little can be done to cope with aggregation effects, we offer a method to quantify the magnitude of such effects in order to formally assess its severity. We use simulated and real data sets to empirically investigate the consequences of these three factors and to evaluate the performance of our proposed methods. C code, along with an easy-to-use R interface, is publicly available for implementing our proposed methods.
Imai, Kosuke, Lu, Ying, Strauss, Aaron
Submitted: 2006-12-16
Keywords: Coarse data, Contextual effects, Data augmentation, EM algorithm, Missing information principle, Nonparametric Bayesian Modeling.
Abstract: (click to show/hide) Ecological inference is a statistical problem where aggregate-level data are used to make inferences about individual-level behavior. Recent years have witnessed resurgent interest in ecological inference among political methodologists and statisticians. In this paper, we conduct a theoretical and empirical study of Bayesian and likelihood inference for 2 x 2 ecological tables by applying the general statistical framework of incomplete data. We first show that the ecological inference problem can be decomposed into three factors: distributional effects which address the possible misspecification of parametric modeling assumptions about the unknown distribution of missing data, contextual effects which represent the possible correlation between missing data and observed variables, and aggregation effects which are directly related to the loss of information caused by data aggregation. We then examine how these three factors affect inference and offer new statistical methods to address each of them. To deal with distributional effects, we propose a nonparametric Bayesian model based on a Dirichlet process prior which relaxes common parametric assumptions. We also specify the statistical adjustments necessary to account for contextual effects. Finally, while little can be done to cope with aggregation effects, we offer a method to quantify the magnitude of such effects in order to formally assess its severity. We use simulated and real data sets to empirically investigate the consequences of these three factors and to evaluate the performance of our proposed methods. C code, along with an easy-to-use R interface, is publicly available for implementing our proposed methods.
Using Graphs Instead of Tables to Improve the Presentation of Empirical Results in Political Science
Kastellec, Jonathan, Leoni, Eduardo
Submitted: 2006-11-15
Keywords: statistical graphics, tables, presentation, descriptive statistics, regression results
Abstract: (click to show/hide) When political scientists present empirical results, they are much more likely to use tables rather than graphs, despite the fact that the latter greatly increases the clarity of presentation and makes it easier for a reader or listener to draw clear and correct inferences. Using a sample of leading journals, we document this tendency and suggest reasons why researchers prefer tables. We argue the extra work required in producing graphs is rewarded by greatly enhanced presentation and communication of empirical results. We illustrate their benefits by turning several published tables into graphs, including tables that present descriptive data and regression results. We show that regression graphs properly emphasize point estimates and confidence intervals rather than null significance hypothesis testing, and that they can successfully present the results of multiple regression models. A move away from tables and towards graphs would increase the quality of the discipline's communicative output and make empirical findings more accessible to every type of audience.
Kastellec, Jonathan, Leoni, Eduardo
Submitted: 2006-11-15
Keywords: statistical graphics, tables, presentation, descriptive statistics, regression results
Abstract: (click to show/hide) When political scientists present empirical results, they are much more likely to use tables rather than graphs, despite the fact that the latter greatly increases the clarity of presentation and makes it easier for a reader or listener to draw clear and correct inferences. Using a sample of leading journals, we document this tendency and suggest reasons why researchers prefer tables. We argue the extra work required in producing graphs is rewarded by greatly enhanced presentation and communication of empirical results. We illustrate their benefits by turning several published tables into graphs, including tables that present descriptive data and regression results. We show that regression graphs properly emphasize point estimates and confidence intervals rather than null significance hypothesis testing, and that they can successfully present the results of multiple regression models. A move away from tables and towards graphs would increase the quality of the discipline's communicative output and make empirical findings more accessible to every type of audience.
Negative Results in Social Science
Lehrer, David, Leschke, Janine, Lhachimi, Stefan, Vasiliu, Ana, Weiffen, Brigitte
Submitted: 2006-11-11
Keywords: methodology, negative results, philosophy of science, publication bias
Abstract: (click to show/hide) Do academic publication standards reflect or determine research results? The article proposes minimal criteria for distinguishing useful ‘unpublishable’ results from low-quality research, and argues that the virtues of negative results have been overlooked. We consider the fate these results have suffered thus far, review arguments for and against their publication, and introduce a new initiative—a journal to disseminate negative results and advance debate on their recognition and use.
Lehrer, David, Leschke, Janine, Lhachimi, Stefan, Vasiliu, Ana, Weiffen, Brigitte
Submitted: 2006-11-11
Keywords: methodology, negative results, philosophy of science, publication bias
Abstract: (click to show/hide) Do academic publication standards reflect or determine research results? The article proposes minimal criteria for distinguishing useful ‘unpublishable’ results from low-quality research, and argues that the virtues of negative results have been overlooked. We consider the fate these results have suffered thus far, review arguments for and against their publication, and introduce a new initiative—a journal to disseminate negative results and advance debate on their recognition and use.
Forecasting House Seats from Generic Congressional Polls
Bafumi, Joseph, Erikson, Robert S., Wlezien, Christopher
Submitted: 2006-10-25
Keywords: generic ballot polls, forecast, 2006 midterm election, congressional seats, simulations
Abstract: (click to show/hide) We provide some guidance for translating the results of generic congressional polls into the election outcome for 2006. Via computer simulation based on statistical analysis of historical data, we show how generic vote polls can be used to forecast the election outcome. We convert the results of generic vote polls into a projection of the actual national vote for Congress and ultimately into the partisan division of seats in the House of Representatives. Our model allows both a point forecast—our expectation of the seat division between Republicans and Democrats—and an estimate of the probability of partisan control. Based on current generic ballot polls, we forecast an expected Democratic gain of 32 seats with Democratic control (a gain of 15 seats or more) a near certainty.
Bafumi, Joseph, Erikson, Robert S., Wlezien, Christopher
Submitted: 2006-10-25
Keywords: generic ballot polls, forecast, 2006 midterm election, congressional seats, simulations
Abstract: (click to show/hide) We provide some guidance for translating the results of generic congressional polls into the election outcome for 2006. Via computer simulation based on statistical analysis of historical data, we show how generic vote polls can be used to forecast the election outcome. We convert the results of generic vote polls into a projection of the actual national vote for Congress and ultimately into the partisan division of seats in the House of Representatives. Our model allows both a point forecast—our expectation of the seat division between Republicans and Democrats—and an estimate of the probability of partisan control. Based on current generic ballot polls, we forecast an expected Democratic gain of 32 seats with Democratic control (a gain of 15 seats or more) a near certainty.
Social Preferences and Political Participation
Dawes, Christopher, Fowler, James
Submitted: 2006-10-23
Keywords:
Abstract: (click to show/hide) This paper examines the link between social preferences and political activity using experimental methods. We conduct a laboratory experiment in which subjects are asked a series of questions about their past political participation and then are instructed to play five rounds of a modified dictator game (Andreoni and Miller 2002). The results of the dictator game are used to classify each subject’s preferences. We find that subjects who are most interested in increasing total welfare are more likely to participate in politics than subjects with selfish preferences, whereas subjects most interested in reducing the difference between their own well-being and the well-being of others are no more likely to participate in politics than subjects with selfish preferences.
Dawes, Christopher, Fowler, James
Submitted: 2006-10-23
Keywords:
Abstract: (click to show/hide) This paper examines the link between social preferences and political activity using experimental methods. We conduct a laboratory experiment in which subjects are asked a series of questions about their past political participation and then are instructed to play five rounds of a modified dictator game (Andreoni and Miller 2002). The results of the dictator game are used to classify each subject’s preferences. We find that subjects who are most interested in increasing total welfare are more likely to participate in politics than subjects with selfish preferences, whereas subjects most interested in reducing the difference between their own well-being and the well-being of others are no more likely to participate in politics than subjects with selfish preferences.
A Tournament of Party Decision Rules
Fowler, James, Laver, Michael
Submitted: 2006-10-20
Keywords:
Abstract: (click to show/hide) In the spirit of Axelrod’s famous series of tournaments for strategies in the repeat-play prisoner’s dilemma, we conducted a “tournament of party decision rules” in a dynamic agent-based spatial model of party competition. A call was issued for researchers to submit rules for selecting party positions in a two-dimensional policy space. Each submitted rule was pitted against all others in a suite of very long-running simulations in which all parties falling below a declared support threshold for two consecutive elections “died” and one new party was “born” each election at a random spatial location, using a rule randomly drawn from the set submitted. The policy-selection rule most successful at winning votes over the very long run was declared the “winner”. The most successful rule was identified unambiguously and combined a number of striking features. It satisficed rather than maximized in the short run; it was “parasitic” on choices made by other successful rules; and it was hard-wired not to attack other agents using the same rule, which it identified using a “secret handshake”. We followed up the tournament with a second suite of simulations in a more evolutionary setting in which the selection probability of a rule was a function of its “fitness”, measured in terms of the previous success of agents using the same rule. In this setting, the rule that won the original tournament pulled even further ahead of the competition. Treated as a discovery tool, tournament results raise a series of intriguing issues for those involved in the modeling of party competition.
Fowler, James, Laver, Michael
Submitted: 2006-10-20
Keywords:
Abstract: (click to show/hide) In the spirit of Axelrod’s famous series of tournaments for strategies in the repeat-play prisoner’s dilemma, we conducted a “tournament of party decision rules” in a dynamic agent-based spatial model of party competition. A call was issued for researchers to submit rules for selecting party positions in a two-dimensional policy space. Each submitted rule was pitted against all others in a suite of very long-running simulations in which all parties falling below a declared support threshold for two consecutive elections “died” and one new party was “born” each election at a random spatial location, using a rule randomly drawn from the set submitted. The policy-selection rule most successful at winning votes over the very long run was declared the “winner”. The most successful rule was identified unambiguously and combined a number of striking features. It satisficed rather than maximized in the short run; it was “parasitic” on choices made by other successful rules; and it was hard-wired not to attack other agents using the same rule, which it identified using a “secret handshake”. We followed up the tournament with a second suite of simulations in a more evolutionary setting in which the selection probability of a rule was a function of its “fitness”, measured in terms of the previous success of agents using the same rule. In this setting, the rule that won the original tournament pulled even further ahead of the competition. Treated as a discovery tool, tournament results raise a series of intriguing issues for those involved in the modeling of party competition.
Seeking 50% of Seats, Needing More than 50% of Votes: Predicting the Seats-Votes Curve in the 2006 Elections
Kastellec, Jonathan, Gelman, Andrew, Chandler, Jamie
Submitted: 2006-10-13
Keywords:
Abstract: (click to show/hide) After their stunning loss of both houses of Congress in 1994, the Democrats have averaged over 50% of the vote in Congressional races in every year except 2002, yet they have not regained control of the House. The same is true with the Senate: in the last three elections (during which the 100 Senators were elected), Democratic candidates have earned three million more votes than Republican candiates, yet they are outnumbered by Republicans in the Senate as well. 2006 is looking better for the Democrats, but our calculations show that they need to average at least 52% of the vote (which is more than either party has received since 1992) to have an even chance of taking control of the House of Representatives. Why are things so tough? Looking at the 2004 election, the Democrats won their victories with an average of 69% of the vote, while the Republicans averaged 65% in their contests, thus 'wasting' fewer votes. More formally, we estimated the seats-votes curve for 2006 by constructing a model to predict the 2006 election from 2004, and then validating the method by applying it to previous elections (predicting 2004 from 2002, and so forth). We predict that the Democrats will need 49% of the average vote to have a 10% chance, 52% of the vote to have an even chance, and 55% of the vote to have a 90% chance of winning the House. The Democrats might be able to do it, but it won't be easy.
Kastellec, Jonathan, Gelman, Andrew, Chandler, Jamie
Submitted: 2006-10-13
Keywords:
Abstract: (click to show/hide) After their stunning loss of both houses of Congress in 1994, the Democrats have averaged over 50% of the vote in Congressional races in every year except 2002, yet they have not regained control of the House. The same is true with the Senate: in the last three elections (during which the 100 Senators were elected), Democratic candidates have earned three million more votes than Republican candiates, yet they are outnumbered by Republicans in the Senate as well. 2006 is looking better for the Democrats, but our calculations show that they need to average at least 52% of the vote (which is more than either party has received since 1992) to have an even chance of taking control of the House of Representatives. Why are things so tough? Looking at the 2004 election, the Democrats won their victories with an average of 69% of the vote, while the Republicans averaged 65% in their contests, thus 'wasting' fewer votes. More formally, we estimated the seats-votes curve for 2006 by constructing a model to predict the 2006 election from 2004, and then validating the method by applying it to previous elections (predicting 2004 from 2002, and so forth). We predict that the Democrats will need 49% of the average vote to have a 10% chance, 52% of the vote to have an even chance, and 55% of the vote to have a 90% chance of winning the House. The Democrats might be able to do it, but it won't be easy.
Genetic Variation in Political Participation
Fowler, James, Baker, Laura, Dawes, Christopher
Submitted: 2006-10-12
Keywords:
Abstract: (click to show/hide) The decision to vote has puzzled scholars for decades. Theoretical models predict little or no variation in participation in large population elections and empirical models have typically explained only a relatively small portion of individual-level variance in turnout behavior. However, these models have not considered the hypothesis that part of the variation in voting behavior can be attributed to genetic effects. Matching public voter turnout records in Los Angeles to a twin registry, we study the heritability of political behavior in monozygotic and dizygotic twins. The results show that genes account for a significant proportion of the variation in voter turnout. We also replicate these results with data from the National Longitudinal Study of Adolescent Health and show that they extend to a broad class of acts of political participation. These are the first findings to suggest that humans exhibit genetic variation in their tendency to participate in political activities.
Fowler, James, Baker, Laura, Dawes, Christopher
Submitted: 2006-10-12
Keywords:
Abstract: (click to show/hide) The decision to vote has puzzled scholars for decades. Theoretical models predict little or no variation in participation in large population elections and empirical models have typically explained only a relatively small portion of individual-level variance in turnout behavior. However, these models have not considered the hypothesis that part of the variation in voting behavior can be attributed to genetic effects. Matching public voter turnout records in Los Angeles to a twin registry, we study the heritability of political behavior in monozygotic and dizygotic twins. The results show that genes account for a significant proportion of the variation in voter turnout. We also replicate these results with data from the National Longitudinal Study of Adolescent Health and show that they extend to a broad class of acts of political participation. These are the first findings to suggest that humans exhibit genetic variation in their tendency to participate in political activities.
Does Private Money Buy Public Policy? Campaign Contributions and Regulatory Outcomes in Telecommunications
de Figueiredo, Rui
Submitted: 2006-10-06
Keywords: campaign contributions, regulation, selection bias, omitted variable bias
Abstract: (click to show/hide) To what extent can market participants affect the outcomes of regulatory policy? In this paper, we study the effects of one potential source of influence – campaign contributions – from competing interests in the local telecommunications industry, on regulatory policy decisions of state public utility commissions. Using a unique new data set, we find, in contrast to much of the literature on campaign contributions, that there is a significant effect of private money on regulatory outcomes. This result is robust to numerous alternative model specifications. We also assess the extent of omitted variable bias that would have to exist to obviate the estimated result. We find that for our result to be spurious, omitted variables would have to explain more than five times the variation in the mix of private money as is explained by the variables included in our analysis. We consider this to be very unlikely.
de Figueiredo, Rui
Submitted: 2006-10-06
Keywords: campaign contributions, regulation, selection bias, omitted variable bias
Abstract: (click to show/hide) To what extent can market participants affect the outcomes of regulatory policy? In this paper, we study the effects of one potential source of influence – campaign contributions – from competing interests in the local telecommunications industry, on regulatory policy decisions of state public utility commissions. Using a unique new data set, we find, in contrast to much of the literature on campaign contributions, that there is a significant effect of private money on regulatory outcomes. This result is robust to numerous alternative model specifications. We also assess the extent of omitted variable bias that would have to exist to obviate the estimated result. We find that for our result to be spurious, omitted variables would have to explain more than five times the variation in the mix of private money as is explained by the variables included in our analysis. We consider this to be very unlikely.
Statistical Backwards Induction: A Simple Method for Estimating Statistical Strategic Models
Bas, Muhammet, Signorino, Curtis, Walker, Robert
Submitted: 2006-09-22
Keywords: discrete choice, strategic, QRE, logit, probit, statistical backwards induction, limited information estimation
Abstract: (click to show/hide) We present a simple method for estimating regressions based on extensive-form games. Our procedure, which can be implemented in most standard statistical packages, involves sequentially estimating standard logits (or probits) in a manner analogous to backwards induction. We demonstrate that the technique produces consistent parameter estimates and show how to calculate consistent standard errors using model-dependent analytical and general simulation techniques. To illustrate the method, we replicate Leblang’s (2003) study of speculative attacks by financial markets and government responses to these attacks.
Bas, Muhammet, Signorino, Curtis, Walker, Robert
Submitted: 2006-09-22
Keywords: discrete choice, strategic, QRE, logit, probit, statistical backwards induction, limited information estimation
Abstract: (click to show/hide) We present a simple method for estimating regressions based on extensive-form games. Our procedure, which can be implemented in most standard statistical packages, involves sequentially estimating standard logits (or probits) in a manner analogous to backwards induction. We demonstrate that the technique produces consistent parameter estimates and show how to calculate consistent standard errors using model-dependent analytical and general simulation techniques. To illustrate the method, we replicate Leblang’s (2003) study of speculative attacks by financial markets and government responses to these attacks.
Should the Democrats move to the left on economic policy?
Gelman, Andrew
Submitted: 2006-09-20
Keywords: median voter, Presidential election, public opinion, spatial model of voting
Abstract: (click to show/hide) Could John Kerry have gained votes in the recent Presidential election by more clearly distinguishing himself from George Bush on economic policy? At first thought, the logic of political preferences would suggest not: the Republicans are to the right of most Americans on economic policy, and so in a one-dimensional space with party positions measured with no error, the optimal strategy for the Democrats would be to stand infinitesimally to the left of the Republicans. The median voter theorem suggests that each party should keep its policy positions just barely distinguishable from the opposition. In a multidimensional setting, however, or when voters vary in their perceptions of the parties' positions, a party can benefit from putting some daylight between itself and the other party on an issue where it has a public-opinion advantage (such as economic policy for the Democrats). We set up a plausible theoretical model in which the Democrats could achieve a net gain in votes by moving to the left on economic policy, given the parties' positions on a range of issue dimensions. We then evaluate this model based on survey data on voters' perceptions of their own positions and those of the candidates in 2004. Under our model, it turns out to be optimal for the Democrats to move slightly to the {em right} but staying clearly to the left of the Republicans' current position on economic issues.
Gelman, Andrew
Submitted: 2006-09-20
Keywords: median voter, Presidential election, public opinion, spatial model of voting
Abstract: (click to show/hide) Could John Kerry have gained votes in the recent Presidential election by more clearly distinguishing himself from George Bush on economic policy? At first thought, the logic of political preferences would suggest not: the Republicans are to the right of most Americans on economic policy, and so in a one-dimensional space with party positions measured with no error, the optimal strategy for the Democrats would be to stand infinitesimally to the left of the Republicans. The median voter theorem suggests that each party should keep its policy positions just barely distinguishable from the opposition. In a multidimensional setting, however, or when voters vary in their perceptions of the parties' positions, a party can benefit from putting some daylight between itself and the other party on an issue where it has a public-opinion advantage (such as economic policy for the Democrats). We set up a plausible theoretical model in which the Democrats could achieve a net gain in votes by moving to the left on economic policy, given the parties' positions on a range of issue dimensions. We then evaluate this model based on survey data on voters' perceptions of their own positions and those of the candidates in 2004. Under our model, it turns out to be optimal for the Democrats to move slightly to the {em right} but staying clearly to the left of the Republicans' current position on economic issues.
Can political science literatures be believed? A study of publication bias in the APSR and the AJPS
Gerber, Alan, Malhotra, Neil
Submitted: 2006-09-07
Keywords: publication bias
Abstract: (click to show/hide) Despite great attention to the quality of research methods in individual studies, if the publication decisions of journals are a function of the statistical significance of research findings, the published literature as a whole may not produce an accurate measure of true effects. This paper examines the two most prominent political science journals (the APSR and the AJPS) and two major literatures in the discipline (the effect of negative advertisements and economic voting) to see if there is evidence of publication bias. We examine the effect of the .05 significance level on the pattern of published findings using what we term a “caliper” test and can reject the hypothesis of no publication bias at the 1 in 100,000,000 level. Our findings therefore strongly suggest that the results reported in the leading political science journals and in two important literatures are misleading and inaccurate due to publication bias. We also discuss some of the reasons for publication bias and propose reforms to reduce its impact on research.
Gerber, Alan, Malhotra, Neil
Submitted: 2006-09-07
Keywords: publication bias
Abstract: (click to show/hide) Despite great attention to the quality of research methods in individual studies, if the publication decisions of journals are a function of the statistical significance of research findings, the published literature as a whole may not produce an accurate measure of true effects. This paper examines the two most prominent political science journals (the APSR and the AJPS) and two major literatures in the discipline (the effect of negative advertisements and economic voting) to see if there is evidence of publication bias. We examine the effect of the .05 significance level on the pattern of published findings using what we term a “caliper” test and can reject the hypothesis of no publication bias at the 1 in 100,000,000 level. Our findings therefore strongly suggest that the results reported in the leading political science journals and in two important literatures are misleading and inaccurate due to publication bias. We also discuss some of the reasons for publication bias and propose reforms to reduce its impact on research.
A 'Politically Robust' Experimental Design for Public Policy Evaluation, with Application to the Mexican Universal Health Insurance Program
King, Gary, Gakidou, Emmanuela, Ravishankar, Nirmala, Moore, Ryan, Lakin, Jason, Vargas, Manett, Téllez-Rojo, Martha María, Avila, Juan Eugenio Hernández, Avila, Mauricio Hernández, Llamas, Héctor Hernández
Submitted: 2006-09-05
Keywords:
Abstract: (click to show/hide) We develop an approach to conducting large scale randomized public policy experiments intended to be more robust to the political interventions that have ruined some or all parts of many similar previous efforts. Under our proposed design, the benefits of random assignment would remain even if we lose observations; our inferences can still be unbiased even if politics disrupts two of the three steps in our analytical procedures; and other empirical checks are available to validate the overall design. We illustrate with a design and empirical validation of a planned evaluation of the Mexican Seguro Popular de Salud (Universal Health Insurance) program. Seguro Popular, which is intended to grow to provide medical care, drugs, preventative services, and financial health protection to the 50 million Mexicans without health insurance, is one of the largest health reforms of any country in the last two decades. The evaluation is also large scale, constituting one of the largest policy experiments to date and what may be the largest randomized health policy experiment ever.
King, Gary, Gakidou, Emmanuela, Ravishankar, Nirmala, Moore, Ryan, Lakin, Jason, Vargas, Manett, Téllez-Rojo, Martha María, Avila, Juan Eugenio Hernández, Avila, Mauricio Hernández, Llamas, Héctor Hernández
Submitted: 2006-09-05
Keywords:
Abstract: (click to show/hide) We develop an approach to conducting large scale randomized public policy experiments intended to be more robust to the political interventions that have ruined some or all parts of many similar previous efforts. Under our proposed design, the benefits of random assignment would remain even if we lose observations; our inferences can still be unbiased even if politics disrupts two of the three steps in our analytical procedures; and other empirical checks are available to validate the overall design. We illustrate with a design and empirical validation of a planned evaluation of the Mexican Seguro Popular de Salud (Universal Health Insurance) program. Seguro Popular, which is intended to grow to provide medical care, drugs, preventative services, and financial health protection to the 50 million Mexicans without health insurance, is one of the largest health reforms of any country in the last two decades. The evaluation is also large scale, constituting one of the largest policy experiments to date and what may be the largest randomized health policy experiment ever.
The Future of Partisan Symmetry as a Judicial Test for Partisan Gerrymandering after LULAC v. Perry
Grofman, Bernard, King, Gary
Submitted: 2006-08-04
Keywords: Gerrymandering, redistricting, random effects, partisan symmetry
Abstract: (click to show/hide) While the Supreme Court in Bandemer v. Davis found partisan gerrymandering to be justiciable, no challenged redistricting plan in the subsequent 20 years has been held unconstitutional on partisan grounds. Then, in Vieth v. Jubilerer, five justices concluded that some standard might be adopted in a future case, if a manageable rule could be found. When gerrymandering next came before the Court, in LULAC v. Perry, we along with our colleagues filed an Amicus Brief (King et al., 2005), proposing that a test be based in part on the partisan symmetry standard. Although the issue was not resolved, our proposal was discussed and positively evaluated in three of the opinions, including the plurality judgment, and for the first time for any proposal the Court gave a clear indication that a future legal test for partisan gerrymandering will likely include partisan symmetry. A majority of Justices now appear to endorse the view that the measurement of partisan symmetry may be used in partisan gerrymandering claims as “a helpful (though certainly not talismanic) tool” (Justice Stevens, joined by Justice Breyer), provided one recognizes that “asymmetry alone is not a reliable measure of unconstitutional partisanship” and possibly that the standard would be applied only after at least one election has been held under the redistricting plan at issue (Justice Kennedy, joined by Justices Souter and Ginsburg). We use this essay to respond to the request of Justices Souter and Ginsburg that “further attention … be devoted to the administrability of such a criterion at all levels of redistricting and its review.” Building on our previous scholarly work, our Amicus Brief, the observations of these five Justices, and a supporting consensus in the academic literature, we offer here a social science perspective on the conceptualization and measurement of partisan gerrymandering and the development of relevant legal rules based on what is effectively the Supreme Court’s open invitation to lower courts to revisit these issues in the light of LULAC v. Perry. (Forthcoming, January 2007 Election Law Journal. Comments welcome.)
Grofman, Bernard, King, Gary
Submitted: 2006-08-04
Keywords: Gerrymandering, redistricting, random effects, partisan symmetry
Abstract: (click to show/hide) While the Supreme Court in Bandemer v. Davis found partisan gerrymandering to be justiciable, no challenged redistricting plan in the subsequent 20 years has been held unconstitutional on partisan grounds. Then, in Vieth v. Jubilerer, five justices concluded that some standard might be adopted in a future case, if a manageable rule could be found. When gerrymandering next came before the Court, in LULAC v. Perry, we along with our colleagues filed an Amicus Brief (King et al., 2005), proposing that a test be based in part on the partisan symmetry standard. Although the issue was not resolved, our proposal was discussed and positively evaluated in three of the opinions, including the plurality judgment, and for the first time for any proposal the Court gave a clear indication that a future legal test for partisan gerrymandering will likely include partisan symmetry. A majority of Justices now appear to endorse the view that the measurement of partisan symmetry may be used in partisan gerrymandering claims as “a helpful (though certainly not talismanic) tool” (Justice Stevens, joined by Justice Breyer), provided one recognizes that “asymmetry alone is not a reliable measure of unconstitutional partisanship” and possibly that the standard would be applied only after at least one election has been held under the redistricting plan at issue (Justice Kennedy, joined by Justices Souter and Ginsburg). We use this essay to respond to the request of Justices Souter and Ginsburg that “further attention … be devoted to the administrability of such a criterion at all levels of redistricting and its review.” Building on our previous scholarly work, our Amicus Brief, the observations of these five Justices, and a supporting consensus in the academic literature, we offer here a social science perspective on the conceptualization and measurement of partisan gerrymandering and the development of relevant legal rules based on what is effectively the Supreme Court’s open invitation to lower courts to revisit these issues in the light of LULAC v. Perry. (Forthcoming, January 2007 Election Law Journal. Comments welcome.)
Sensitive Questions, Truthful Answers? Modeling the List Experiment Multivariately With LISTIT
Corstange, Daniel
Submitted: 2006-08-03
Keywords: list experiment, survey, sensitive questions, listit
Abstract: (click to show/hide) Standard estimation procedures assume that empirical observations are accurate reflections of the true values of the dependent variable, but this assumption is dubious when model self-reported data on sensitive topics. List experiments can nullify incentives for respondents to lie to interviewers, but current data analysis techniques are limited to difference-in-means tests. I present a revised procedure and statistical estimator called listit to model the process multivariately. Monte Carlo simulations and a field test in Lebanon explore the behavior of this estimator.
Corstange, Daniel
Submitted: 2006-08-03
Keywords: list experiment, survey, sensitive questions, listit
Abstract: (click to show/hide) Standard estimation procedures assume that empirical observations are accurate reflections of the true values of the dependent variable, but this assumption is dubious when model self-reported data on sensitive topics. List experiments can nullify incentives for respondents to lie to interviewers, but current data analysis techniques are limited to difference-in-means tests. I present a revised procedure and statistical estimator called listit to model the process multivariately. Monte Carlo simulations and a field test in Lebanon explore the behavior of this estimator.
Statistical Analysis of Randomized Experiments with Nonignorable Missing Binary Outcomes
Imai, Kosuke
Submitted: 2006-07-24
Keywords: Causal Inference, Instrumental Variables, Intention-to-Treat Effect, Latent Ignorability, Noncompliance, Treatment Effect, Sensitivity Analysis
Abstract: (click to show/hide) Missing data are frequently encountered in the statistical analysis of randomized experiments. In this article, I propose statistical methods that can be used to analyze randomized experiments with a nonignorable missing binary outcome where the missing-data mechanism may depend on the unobserved values of the outcome variable itself. I first introduce an identification strategy for the average treatment effect and compare it with the existing alternative approaches in the literature. I then derive the maximum likelihood estimator and its asymptotic properties, and discuss possible estimation methods. Furthermore, since the proposed identification assumption is not directly verifiable from the data, I show how to conduct a sensitivity analysis based on the parameterization that links the key identification assumption with the causal quantities of interest. Then, the proposed methodology is extended to the analysis of randomized experiments with noncompliance. Although the method introduced in this article may not directly apply to randomized experiments with non-binary outcomes, I briefly discuss possible identification strategies in more general situations. Finally, I apply the proposed methodology to analyze data from the German election experiment and the influenza vaccination study, which originally motivated the methodological problems addressed in this article.
Imai, Kosuke
Submitted: 2006-07-24
Keywords: Causal Inference, Instrumental Variables, Intention-to-Treat Effect, Latent Ignorability, Noncompliance, Treatment Effect, Sensitivity Analysis
Abstract: (click to show/hide) Missing data are frequently encountered in the statistical analysis of randomized experiments. In this article, I propose statistical methods that can be used to analyze randomized experiments with a nonignorable missing binary outcome where the missing-data mechanism may depend on the unobserved values of the outcome variable itself. I first introduce an identification strategy for the average treatment effect and compare it with the existing alternative approaches in the literature. I then derive the maximum likelihood estimator and its asymptotic properties, and discuss possible estimation methods. Furthermore, since the proposed identification assumption is not directly verifiable from the data, I show how to conduct a sensitivity analysis based on the parameterization that links the key identification assumption with the causal quantities of interest. Then, the proposed methodology is extended to the analysis of randomized experiments with noncompliance. Although the method introduced in this article may not directly apply to randomized experiments with non-binary outcomes, I briefly discuss possible identification strategies in more general situations. Finally, I apply the proposed methodology to analyze data from the German election experiment and the influenza vaccination study, which originally motivated the methodological problems addressed in this article.
Bargaining and Society: A Statistical Model of the Ultimatum Game
Signorino, Curtis, Ramsay, Kristopher
Submitted: 2006-07-20
Keywords: bargaining, ultimatum, game theory, statistics, strategic, rationality
Abstract: (click to show/hide) In this paper we derive a statistical estimator for the popular Ultimatum bargaining game. Using monte carlo data generated by a strategic bargaining process, we show that the estimator correctly recovers the relationship between dependent variables, such as the proposed division and bargaining failure, relative to substantive variables that comprise players' utilities. We then use the model to analyze bargaining data in a number of contexts. The current example examines the effects of demographics on bargaining behavior in experiments conducted on U.S. and Russian participants.
Signorino, Curtis, Ramsay, Kristopher
Submitted: 2006-07-20
Keywords: bargaining, ultimatum, game theory, statistics, strategic, rationality
Abstract: (click to show/hide) In this paper we derive a statistical estimator for the popular Ultimatum bargaining game. Using monte carlo data generated by a strategic bargaining process, we show that the estimator correctly recovers the relationship between dependent variables, such as the proposed division and bargaining failure, relative to substantive variables that comprise players' utilities. We then use the model to analyze bargaining data in a number of contexts. The current example examines the effects of demographics on bargaining behavior in experiments conducted on U.S. and Russian participants.
Is It Worth Going the Extra Mile to Improve Causal Inference? Understanding Voting in Los Angeles County
Brady, Henry E., Hui, Iris
Submitted: 2006-07-19
Keywords: Counterfactual, matching, geography, GIS, voting
Abstract: (click to show/hide) Two seemingly unrelated approaches to quantitative analysis have recently become more popular in social science applications. The first approach is the explicit consideration of counterfactuals in causal inference and the development of various matching techniques to choose control cases comparable to treated cases in terms of some predefined characteristics. To be useful, these methods require the identification of important characteristics that are likely to ensure that a statistical condition called “conditional independence” is met. The second trend is the increased attention given to geography and the use of spatial statistics. Although these two approaches have found their ways into the social science research separately, we think that they can be fruitfully combined. Geography and Geographic Information Systems (GIS) can improve matching and causal inference. Geography can be conceptualized in terms of “distance” and “place” which can provide guidance about potentially important characteristics that can be used to improve matching. After developing a conceptual framework that shows how this can be done, we present two empirical examples which combine counterfactual thinking with geographical ideas. The first example looks at the cost of voting and demonstrates the utility of matching using zip codes and distance to polling place. The second example looks at the performance of the InkaVote voting system in Los Angeles by matching precincts in LA with geographically adjacent precincts in surrounding counties. This example demonstrates the strengths and weaknesses of geographic proximity as a matching variable. In pursuing these examples, we also show how recent progress in GIS techniques provides tools that can deepen researchers’ understanding of their idea.
Brady, Henry E., Hui, Iris
Submitted: 2006-07-19
Keywords: Counterfactual, matching, geography, GIS, voting
Abstract: (click to show/hide) Two seemingly unrelated approaches to quantitative analysis have recently become more popular in social science applications. The first approach is the explicit consideration of counterfactuals in causal inference and the development of various matching techniques to choose control cases comparable to treated cases in terms of some predefined characteristics. To be useful, these methods require the identification of important characteristics that are likely to ensure that a statistical condition called “conditional independence” is met. The second trend is the increased attention given to geography and the use of spatial statistics. Although these two approaches have found their ways into the social science research separately, we think that they can be fruitfully combined. Geography and Geographic Information Systems (GIS) can improve matching and causal inference. Geography can be conceptualized in terms of “distance” and “place” which can provide guidance about potentially important characteristics that can be used to improve matching. After developing a conceptual framework that shows how this can be done, we present two empirical examples which combine counterfactual thinking with geographical ideas. The first example looks at the cost of voting and demonstrates the utility of matching using zip codes and distance to polling place. The second example looks at the performance of the InkaVote voting system in Los Angeles by matching precincts in LA with geographically adjacent precincts in surrounding counties. This example demonstrates the strengths and weaknesses of geographic proximity as a matching variable. In pursuing these examples, we also show how recent progress in GIS techniques provides tools that can deepen researchers’ understanding of their idea.
Presidential Approval: the case of George W. Bush
Beck, Nathaniel, Jackman, Simon, Rosenthal, Howard
Submitted: 2006-07-19
Keywords: presidential approval, public opinion, polls, house effects, dynamic linear model, Bayesian statistics, Markov chain Monte Carlo, state space, pages of killer graphs
Abstract: (click to show/hide) We use a Bayesian dynamic linear model to track approval for George W. Bush over time. Our analysis deals with several issues that have been usually addressed separately in the extant literature. First, our analysis uses polling data collected at a higher frequency than is typical, using over 1,100 published national polls, and data on macro-economic conditions collected at the weekly level. By combining this much poll information, we are much better poised to examine the public's reactions to events over shorter time scales than can the typical analysis of approval that utilizes monthly or quarterly approval. Second, our statistical modeling explicitly deals with the sampling error of these polls, as well as the possibility of bias in the polls due to house effects. Indeed, quite aside from the question of ``what drives approval?'', there is considerable interest in the extent to which polling organizations systematically diverge from one another in assessing approval for the president. These bias parameters are not only necessary parts of any realistic model of approval that utilizes data from multiple polling organizations, but easily estimated via the Bayesian dynamics linear model.
Beck, Nathaniel, Jackman, Simon, Rosenthal, Howard
Submitted: 2006-07-19
Keywords: presidential approval, public opinion, polls, house effects, dynamic linear model, Bayesian statistics, Markov chain Monte Carlo, state space, pages of killer graphs
Abstract: (click to show/hide) We use a Bayesian dynamic linear model to track approval for George W. Bush over time. Our analysis deals with several issues that have been usually addressed separately in the extant literature. First, our analysis uses polling data collected at a higher frequency than is typical, using over 1,100 published national polls, and data on macro-economic conditions collected at the weekly level. By combining this much poll information, we are much better poised to examine the public's reactions to events over shorter time scales than can the typical analysis of approval that utilizes monthly or quarterly approval. Second, our statistical modeling explicitly deals with the sampling error of these polls, as well as the possibility of bias in the polls due to house effects. Indeed, quite aside from the question of ``what drives approval?'', there is considerable interest in the extent to which polling organizations systematically diverge from one another in assessing approval for the president. These bias parameters are not only necessary parts of any realistic model of approval that utilizes data from multiple polling organizations, but easily estimated via the Bayesian dynamics linear model.
Spatio-Temporal Models for Political-Science Panel and Time-Series-Cross-Section Data
Franzese, Robert, Hays, Jude
Submitted: 2006-07-18
Keywords: Spatial Econometrics, Spatial-Lag Model, Spatio-Temporal Model, Panel Data, Time-Series-Cross-Section Data, Spatio-Temporal Multiplier, Spatio-Temporal Dynamics, Spatio-Temporal Steady-State Effects
Abstract: (click to show/hide) Building from our broader project exploring spatial-econometric models for political science, this paper discusses estimation, interpretation, and presentation of spatio-temporal models. We first present a generic spatio-temporal-lag model and two methods, OLS and ML, for estimating parameters in such models. We briefly consider those estimators’ properties analytically before showing next how to calculate and to present the spatio-temporal dynamic and long-run steady-state equilibrium effects—i.e., the spatio-temporal substance of the model—implied by the coefficient estimates. Then, we conduct Monte Carlo experiments to explore the properties of the OLS and ML estimators, and, finally, we conclude with a reanalysis of Beck, Gleditsch, and Beardsley’s (2006) state-of-the-art study of directed export flows among major powers.
Franzese, Robert, Hays, Jude
Submitted: 2006-07-18
Keywords: Spatial Econometrics, Spatial-Lag Model, Spatio-Temporal Model, Panel Data, Time-Series-Cross-Section Data, Spatio-Temporal Multiplier, Spatio-Temporal Dynamics, Spatio-Temporal Steady-State Effects
Abstract: (click to show/hide) Building from our broader project exploring spatial-econometric models for political science, this paper discusses estimation, interpretation, and presentation of spatio-temporal models. We first present a generic spatio-temporal-lag model and two methods, OLS and ML, for estimating parameters in such models. We briefly consider those estimators’ properties analytically before showing next how to calculate and to present the spatio-temporal dynamic and long-run steady-state equilibrium effects—i.e., the spatio-temporal substance of the model—implied by the coefficient estimates. Then, we conduct Monte Carlo experiments to explore the properties of the OLS and ML estimators, and, finally, we conclude with a reanalysis of Beck, Gleditsch, and Beardsley’s (2006) state-of-the-art study of directed export flows among major powers.
Alternative Balance Metrics for Bias Reduction in Matching Methods for Causal Inference
Sekhon, Jasjeet
Submitted: 2006-07-18
Keywords: Matching, Causal Inference, Genetic Matching, Balance Metrics
Abstract: (click to show/hide) Sekhon (2006; 2004a) and Diamond and Sekhon (2005) propose a matching method, called Genetic Matching, which algorithmically maximizes the balance of covariates between treat- ment and control observations via a genetic search algorithm (Sekhon and Mebane 1998). The method is neutral as to what measures of balance one wishes to optimize. By default, cumulative probability distribution functions of a variety of standardized statistics are used as balance metrics and are optimized without limit. The statistics are not used to conduct formal hypothesis tests, because no measure of balance is a monotonic function of bias in the estimand of interest and because we wish to maximize balance. Descriptive measures of discrepancy generally ignore key information related to bias which is captured by probability distribution functions of standardized test statistics. For example, using several descriptive metrics, one is unable reliably to recover the experimental benchmark in a testbed dataset for matching estimators (Dehejia and Wahba 1999). And these metrics, unlike those based on optimized distribution functions, perform poorly in a series of Monte Carlo sampling experiments just as one would expect given their properties.
Sekhon, Jasjeet
Submitted: 2006-07-18
Keywords: Matching, Causal Inference, Genetic Matching, Balance Metrics
Abstract: (click to show/hide) Sekhon (2006; 2004a) and Diamond and Sekhon (2005) propose a matching method, called Genetic Matching, which algorithmically maximizes the balance of covariates between treat- ment and control observations via a genetic search algorithm (Sekhon and Mebane 1998). The method is neutral as to what measures of balance one wishes to optimize. By default, cumulative probability distribution functions of a variety of standardized statistics are used as balance metrics and are optimized without limit. The statistics are not used to conduct formal hypothesis tests, because no measure of balance is a monotonic function of bias in the estimand of interest and because we wish to maximize balance. Descriptive measures of discrepancy generally ignore key information related to bias which is captured by probability distribution functions of standardized test statistics. For example, using several descriptive metrics, one is unable reliably to recover the experimental benchmark in a testbed dataset for matching estimators (Dehejia and Wahba 1999). And these metrics, unlike those based on optimized distribution functions, perform poorly in a series of Monte Carlo sampling experiments just as one would expect given their properties.
Election Forensics: Vote Counts and Benford's Law
Mebane, Walter R.
Submitted: 2006-07-18
Keywords: election fraud, vote fraud, Benford's Law, election forensics
Abstract: (click to show/hide) How can we be sure that the declared election winner actually got the most votes? Was the election stolen? This paper considers a statistical method based on the pattern of digits in vote counts (the second-digit Benford's Law, or 2BL) that may be useful for detecting fraud or other anomalies. The method seems to be useful for vote counts at the precinct level but not for counts at the level of individual voting machines, at least not when the way voters are assigned to machines induces a pattern I call roughly equal division with leftovers (REDWL). I demonstrate two mechanisms that can cause precinct vote counts in general to satisfy 2BL. I use simulations to illustrate that the 2BL test can be very sensitive when vote counts are subjected to various kinds of manipulation. I use data from the 2004 election in Florida and the 2006 election in Mexico to illustrate use of the 2BL tests.
Mebane, Walter R.
Submitted: 2006-07-18
Keywords: election fraud, vote fraud, Benford's Law, election forensics
Abstract: (click to show/hide) How can we be sure that the declared election winner actually got the most votes? Was the election stolen? This paper considers a statistical method based on the pattern of digits in vote counts (the second-digit Benford's Law, or 2BL) that may be useful for detecting fraud or other anomalies. The method seems to be useful for vote counts at the precinct level but not for counts at the level of individual voting machines, at least not when the way voters are assigned to machines induces a pattern I call roughly equal division with leftovers (REDWL). I demonstrate two mechanisms that can cause precinct vote counts in general to satisfy 2BL. I use simulations to illustrate that the 2BL test can be very sensitive when vote counts are subjected to various kinds of manipulation. I use data from the 2004 election in Florida and the 2006 election in Mexico to illustrate use of the 2BL tests.
An Automated Method of Topic-Coding Legislative Speech Over Time with Application to the 105th-108th U.S. Senate
Quinn, Kevin, Monroe, Burt, Colaresi, Michael, Crespin, Michael, Radev, Dragomir
Submitted: 2006-07-18
Keywords: legislatures, agendas, content analysis, Bayesian, time series, cluster analysis, unsupervised learning
Abstract: (click to show/hide) We describe a method for statistical learning from speech documents that we apply to the Congressional Record in order to gain new insight into the dynamics of the political agenda. Prior efforts to evaluate the attention of elected representatives across topic areas have largely been expensive manual coding exercises and are generally circumscribed along one or more features of detail: limited time periods, high levels of temporal aggregation, and coarse topical categories. Conversely, the Congressional Record has scarcely been used for such analyses, largely because it contains too much information to absorb manually. We describe here a method for inferring, through the patterns of word choice in each speech and the dynamics of word choice patterns across time, (a) what the topics of speeches are, and (b) the probability that attention will be paid to any given topic or set of topics over time. We use the model to examine the agenda in the United States Senate from 1997-2004, based on a new database of over 70 thousand speech documents containing over 70 million words. We estimate the model for 42 topics and provide evidence that we can reveal speech topics that are both distinctive and inter-related in substantively meaningful ways. We demonstrate further that the dynamics our model gives us leverage into important questions about the dynamics of the political agenda.
Quinn, Kevin, Monroe, Burt, Colaresi, Michael, Crespin, Michael, Radev, Dragomir
Submitted: 2006-07-18
Keywords: legislatures, agendas, content analysis, Bayesian, time series, cluster analysis, unsupervised learning
Abstract: (click to show/hide) We describe a method for statistical learning from speech documents that we apply to the Congressional Record in order to gain new insight into the dynamics of the political agenda. Prior efforts to evaluate the attention of elected representatives across topic areas have largely been expensive manual coding exercises and are generally circumscribed along one or more features of detail: limited time periods, high levels of temporal aggregation, and coarse topical categories. Conversely, the Congressional Record has scarcely been used for such analyses, largely because it contains too much information to absorb manually. We describe here a method for inferring, through the patterns of word choice in each speech and the dynamics of word choice patterns across time, (a) what the topics of speeches are, and (b) the probability that attention will be paid to any given topic or set of topics over time. We use the model to examine the agenda in the United States Senate from 1997-2004, based on a new database of over 70 thousand speech documents containing over 70 million words. We estimate the model for 42 topics and provide evidence that we can reveal speech topics that are both distinctive and inter-related in substantively meaningful ways. We demonstrate further that the dynamics our model gives us leverage into important questions about the dynamics of the political agenda.
Election Forensics: Vote Counts and Benford's Law
Mebane, Walter R.
Submitted: 2006-07-17
Keywords: election forensics, Benford's law, vote fraud, election fraud, Florida 2004, Mexico 2006
Abstract: (click to show/hide) How can we be sure that the declared election winner actually got the most votes? Was the election stolen? This paper considers a statistical method based on the pattern of digits in vote counts (the second-digit Benford's Law, or 2BL) that may be useful for detecting fraud or other anomalies. The method seems to be useful for vote counts at the precinct level but not for counts at the level of individual voting machines, at least not when the way voters are assigned to machines induces a pattern I call roughly equal division with leftovers (REDWL). I demonstrate two mechanisms that can cause precinct vote counts in general to satisfy 2BL. I use simulations to illustrate that the 2BL test can be very sensitive when vote counts are subjected to various kinds of manipulation. I use data from the 2004 election in Florida and the 2006 election in Mexico to illustrate use of the 2BL tests.
Mebane, Walter R.
Submitted: 2006-07-17
Keywords: election forensics, Benford's law, vote fraud, election fraud, Florida 2004, Mexico 2006
Abstract: (click to show/hide) How can we be sure that the declared election winner actually got the most votes? Was the election stolen? This paper considers a statistical method based on the pattern of digits in vote counts (the second-digit Benford's Law, or 2BL) that may be useful for detecting fraud or other anomalies. The method seems to be useful for vote counts at the precinct level but not for counts at the level of individual voting machines, at least not when the way voters are assigned to machines induces a pattern I call roughly equal division with leftovers (REDWL). I demonstrate two mechanisms that can cause precinct vote counts in general to satisfy 2BL. I use simulations to illustrate that the 2BL test can be very sensitive when vote counts are subjected to various kinds of manipulation. I use data from the 2004 election in Florida and the 2006 election in Mexico to illustrate use of the 2BL tests.
Modeling Structural Changes: Bayesian Estimation of Multiple Changepoint Models and State Space Models
Park, Jong Hee
Submitted: 2006-07-17
Keywords: Multiple changepoint model, State space model, Markov chain Monte Carlo methods, Bayes factors, Data augmentation.
Abstract: (click to show/hide) While theoretical models in political science are inspired by structural changes in politics, most empirical methods assume stable patterns of causal relationships. Static models with constant parameters do not properly capture dynamic changes in the data and lead to incorrect parameter estimates. In this paper, I introduce two Bayesian time series models, which can detect and estimate possible structural changes in temporal data: multiple changepoint models and state space models. To emphasize the utility of the models to political scientists, I show some examples in the context of discrete dependent variables. Then, I apply these models to an important debate in international politics over U.S. use of force abroad. The findings of the multiple changepoint and state space models demonstrate that the predictors of presidential use of force have shifted dramatically.
Park, Jong Hee
Submitted: 2006-07-17
Keywords: Multiple changepoint model, State space model, Markov chain Monte Carlo methods, Bayes factors, Data augmentation.
Abstract: (click to show/hide) While theoretical models in political science are inspired by structural changes in politics, most empirical methods assume stable patterns of causal relationships. Static models with constant parameters do not properly capture dynamic changes in the data and lead to incorrect parameter estimates. In this paper, I introduce two Bayesian time series models, which can detect and estimate possible structural changes in temporal data: multiple changepoint models and state space models. To emphasize the utility of the models to political scientists, I show some examples in the context of discrete dependent variables. Then, I apply these models to an important debate in international politics over U.S. use of force abroad. The findings of the multiple changepoint and state space models demonstrate that the predictors of presidential use of force have shifted dramatically.
Expressive Bayesian Voters, their Turnout Decisions, and Double Probit
Achen, Christopher
Submitted: 2006-07-17
Keywords: turnout, expressive, Bayesian, probit, scobit, EITM
Abstract: (click to show/hide) Voting is an expressive act. Since people are not born wanting to express themselves politically, the desire to vote must be acquired, either by learning about the candidates, by using party identification as a cognitive shortcut, or by contact from a trusted source. Modeled as Bayesian updating, this simple explanatory framework has dramatic implications for the understanding of voter turnout. It mathematically implies the main empirical generalizations familiar from the literature, it predicts hitherto unnoticed patterns that appear in turnout data, it provides a better fitting statistical model (double probit) for sample surveys of turnout, and it allows researchers to forecast turnout patterns in new elections when circumstances change. Thus the case is strengthened for the Bayesian voter model as a central organizing principle for public opinion and voting behavior.
Achen, Christopher
Submitted: 2006-07-17
Keywords: turnout, expressive, Bayesian, probit, scobit, EITM
Abstract: (click to show/hide) Voting is an expressive act. Since people are not born wanting to express themselves politically, the desire to vote must be acquired, either by learning about the candidates, by using party identification as a cognitive shortcut, or by contact from a trusted source. Modeled as Bayesian updating, this simple explanatory framework has dramatic implications for the understanding of voter turnout. It mathematically implies the main empirical generalizations familiar from the literature, it predicts hitherto unnoticed patterns that appear in turnout data, it provides a better fitting statistical model (double probit) for sample surveys of turnout, and it allows researchers to forecast turnout patterns in new elections when circumstances change. Thus the case is strengthened for the Bayesian voter model as a central organizing principle for public opinion and voting behavior.
Estimating Incumbency Advantage and Campaign Spending Effect without the Simultaneity Bias
Fukumoto, Kentaro
Submitted: 2006-07-16
Keywords: Incumbency Advantage, Campaign Spending, Simultaneity Bias, Bayesian Nash equilibria, normal vote
Abstract: (click to show/hide) In estimating incumbency advantage and campaign spending effect, simultaneity problem is composed of stochastic dependence and parametric dependence. Scholars have tried to solve the former, while the present paper intends to tackle the latter. Its core idea is to estimate parameters by maximizing likelihood of all endogenous variables (vote, both parties' candidate qualities and campaign spending) simultaneously. In order to do it, I take advantage of theories of electoral politics rigorously, model each endogenous variables by the others (or their expectation), derive Bayesian Nash equilibria, and plug them into my estimator. I show superiority of my model compared to the conventional estimators by Monte Carlo simulation. Empirical application of this model to the recent U.S. House election data demonstrates that incumbency advantage is smaller than previously shown and that entry of incumbent and strong challenger is motivated by electoral prospect.
Fukumoto, Kentaro
Submitted: 2006-07-16
Keywords: Incumbency Advantage, Campaign Spending, Simultaneity Bias, Bayesian Nash equilibria, normal vote
Abstract: (click to show/hide) In estimating incumbency advantage and campaign spending effect, simultaneity problem is composed of stochastic dependence and parametric dependence. Scholars have tried to solve the former, while the present paper intends to tackle the latter. Its core idea is to estimate parameters by maximizing likelihood of all endogenous variables (vote, both parties' candidate qualities and campaign spending) simultaneously. In order to do it, I take advantage of theories of electoral politics rigorously, model each endogenous variables by the others (or their expectation), derive Bayesian Nash equilibria, and plug them into my estimator. I show superiority of my model compared to the conventional estimators by Monte Carlo simulation. Empirical application of this model to the recent U.S. House election data demonstrates that incumbency advantage is smaller than previously shown and that entry of incumbent and strong challenger is motivated by electoral prospect.
A Hierarchical Bayesian Framework for Item Response Theory Models with Applications in Ideal Point Estimation
Lu, Ying, Wang, Xiaohui
Submitted: 2006-07-15
Keywords: item response theory, testlet response theory, random and fixed effect models, vote cast data, roll call analysis
Abstract: (click to show/hide) Ideal point estimation, a variation of item response theory models, has been widely used by political scientists to analyze legislative behaviors. However, many existing ideal point estimation research is based on unrealistic assumptions of independence of different individuals' decisions towards all cases/bills and the independence of one's decisions towards different cases/bills. The violation of such assumptions leads to bias and inefficiency in parameter estimation. More importantly, failing to address these assumptions has hampered the ideal point estimation research from offering intuitive and concise explanations on complex legislative behaviors such as multidimensionality, strategic voting, temporary coalitions. In this paper, we extend one testlet response theory model by Bradlow, Wainer and Wang(1999) to a comprehensive hierarchical Bayesian statistical framework that allows researchers to model inter-individual and intra-individual correlations through random effects and/or fixed effects. Through simulations and an analysis of the US Supreme Court vote cast data, we show that the proposed framework holds good promise for tackling many unsettled issues in ideal point estimations. As a companion to this paper, we also offer an easy-to-use R package with C code that implements the methods discussed herein.
Lu, Ying, Wang, Xiaohui
Submitted: 2006-07-15
Keywords: item response theory, testlet response theory, random and fixed effect models, vote cast data, roll call analysis
Abstract: (click to show/hide) Ideal point estimation, a variation of item response theory models, has been widely used by political scientists to analyze legislative behaviors. However, many existing ideal point estimation research is based on unrealistic assumptions of independence of different individuals' decisions towards all cases/bills and the independence of one's decisions towards different cases/bills. The violation of such assumptions leads to bias and inefficiency in parameter estimation. More importantly, failing to address these assumptions has hampered the ideal point estimation research from offering intuitive and concise explanations on complex legislative behaviors such as multidimensionality, strategic voting, temporary coalitions. In this paper, we extend one testlet response theory model by Bradlow, Wainer and Wang(1999) to a comprehensive hierarchical Bayesian statistical framework that allows researchers to model inter-individual and intra-individual correlations through random effects and/or fixed effects. Through simulations and an analysis of the US Supreme Court vote cast data, we show that the proposed framework holds good promise for tackling many unsettled issues in ideal point estimations. As a companion to this paper, we also offer an easy-to-use R package with C code that implements the methods discussed herein.
Conditional Partisanship: Looking for Partisan Effects on Roll Call Votes in the U.S. House
Patty, John
Submitted: 2006-07-15
Keywords: Roll call voting, House Journal, Partisanship
Abstract: (click to show/hide) In this paper, I examine a simple procedure in the United States House of Representatives, approving the Journal, and its implications for legislative business. In particular, following a suggestion made by Sinclair (1995), I examine the hypothesis that such votes are more than simply pro forma motions or dilatory tactics by the minority party. Rather, the taking of such a vote represents a signal (perhaps to members of the House, but at least to the analyst) that the day’s ensuing business is important to at least one party’s leadership and that it is expected to be a close vote. Considering the 102nd-107th Congresses, I show that a recorded vote on the Speaker’s approval of the Journal indicates that the legislative day’s business will be both more contentious (i.e., recorded votes have a smaller margin of passage) and more partisan (i.e., recorded votes are more likely to be “party unity” votes). In addition, the fit of Poole’s Optimal Classification estimates for legislators’ preferences is higher for recorded votes taken on such days. In addition, I discuss the marginal effect of the type and timing of legislative business on these findings, as well as the identity of who calls for the vote on the Journal. Of particular interest are the differential effects for appropriations and “procedural” matters.
Patty, John
Submitted: 2006-07-15
Keywords: Roll call voting, House Journal, Partisanship
Abstract: (click to show/hide) In this paper, I examine a simple procedure in the United States House of Representatives, approving the Journal, and its implications for legislative business. In particular, following a suggestion made by Sinclair (1995), I examine the hypothesis that such votes are more than simply pro forma motions or dilatory tactics by the minority party. Rather, the taking of such a vote represents a signal (perhaps to members of the House, but at least to the analyst) that the day’s ensuing business is important to at least one party’s leadership and that it is expected to be a close vote. Considering the 102nd-107th Congresses, I show that a recorded vote on the Speaker’s approval of the Journal indicates that the legislative day’s business will be both more contentious (i.e., recorded votes have a smaller margin of passage) and more partisan (i.e., recorded votes are more likely to be “party unity” votes). In addition, the fit of Poole’s Optimal Classification estimates for legislators’ preferences is higher for recorded votes taken on such days. In addition, I discuss the marginal effect of the type and timing of legislative business on these findings, as well as the identity of who calls for the vote on the Journal. Of particular interest are the differential effects for appropriations and “procedural” matters.
Two's Company, Three's an Equilibrium: Strategic Voting and Multicandidate Elections
Patty, John
Submitted: 2006-07-15
Keywords: Multicandidate elections, undominated equilibrium, spatial competition
Abstract: (click to show/hide) In this paper, I characterize equilibria in multicandidate elections. Recognizing that electoral equilibrium involves both candidates’ and voters’ strategies, I first prove existence of pure strategy electoral equilibria when candidates seek to maximize their vote share. Accordingly, the main difficulty with electoral equilibria is multiplicity. I prove that, even after restricting attention to subgame perfect Nash equilibria in weakly undominated strategies, the set of electoral equilibria is very large. I provide characterizations of candidates’ equilibrium platforms, type distributions under which there exist convergent equilibria in which all candidates announce identical platforms, and platforms that can not win in equilibrium. I also examine welfare implications of the results, connections between the noncooperative equilibria, the core, and the uncovered set. Finally, I consider the implications of probability of victory maximization by the candidates.
Patty, John
Submitted: 2006-07-15
Keywords: Multicandidate elections, undominated equilibrium, spatial competition
Abstract: (click to show/hide) In this paper, I characterize equilibria in multicandidate elections. Recognizing that electoral equilibrium involves both candidates’ and voters’ strategies, I first prove existence of pure strategy electoral equilibria when candidates seek to maximize their vote share. Accordingly, the main difficulty with electoral equilibria is multiplicity. I prove that, even after restricting attention to subgame perfect Nash equilibria in weakly undominated strategies, the set of electoral equilibria is very large. I provide characterizations of candidates’ equilibrium platforms, type distributions under which there exist convergent equilibria in which all candidates announce identical platforms, and platforms that can not win in equilibrium. I also examine welfare implications of the results, connections between the noncooperative equilibria, the core, and the uncovered set. Finally, I consider the implications of probability of victory maximization by the candidates.
What to do About Missing Values in Time Series Cross-Section Data
King, Gary, Honaker, James
Submitted: 2006-07-14
Keywords: Missing data, multiple imputation, EM, IP, EMis, time series, cross-section
Abstract: (click to show/hide) Applications of modern methods for analyzing data with missing values, based primarily on multiple imputation, have in the last half-decade become common in American politics and political behavior. Scholars in these fields have thus increasingly avoided the biases and inefficiencies caused by ad hoc methods like listwise deletion and best guess imputation. However, researchers in much of comparative politics and international relations, and others with similar data, have been unable to do the same because the best available imputation methods work poorly with the time-series cross-section data structures common in these fields. We attempt to rectify this situation. First, we build a multiple imputation model that allows smooth time trends, shifts across cross-sectional units, and correlations over time and space, resulting in far more accurate imputations. Second, we build nonignorable missingness models by enabling analysts to incorporate knowledge from area studies experts via priors on individual missing cell values, rather than on difficult-to-interpret model parameters. Third, since these tasks could not be accomplished within existing imputation algorithms, in that they cannot handle as many variables as needed even in the simpler cross-sectional data for which they were designed, we also develop a new algorithm that substantially expands the range of computationally feasible data types and sizes for which multiple imputation can be used. These developments made it possible for us to implement our methods in new open source software which, unlike all existing multiple imputation packages, virtually never crashes.
King, Gary, Honaker, James
Submitted: 2006-07-14
Keywords: Missing data, multiple imputation, EM, IP, EMis, time series, cross-section
Abstract: (click to show/hide) Applications of modern methods for analyzing data with missing values, based primarily on multiple imputation, have in the last half-decade become common in American politics and political behavior. Scholars in these fields have thus increasingly avoided the biases and inefficiencies caused by ad hoc methods like listwise deletion and best guess imputation. However, researchers in much of comparative politics and international relations, and others with similar data, have been unable to do the same because the best available imputation methods work poorly with the time-series cross-section data structures common in these fields. We attempt to rectify this situation. First, we build a multiple imputation model that allows smooth time trends, shifts across cross-sectional units, and correlations over time and space, resulting in far more accurate imputations. Second, we build nonignorable missingness models by enabling analysts to incorporate knowledge from area studies experts via priors on individual missing cell values, rather than on difficult-to-interpret model parameters. Third, since these tasks could not be accomplished within existing imputation algorithms, in that they cannot handle as many variables as needed even in the simpler cross-sectional data for which they were designed, we also develop a new algorithm that substantially expands the range of computationally feasible data types and sizes for which multiple imputation can be used. These developments made it possible for us to implement our methods in new open source software which, unlike all existing multiple imputation packages, virtually never crashes.
A State-Space Approach to Economic Popularity Functions
Pickup, Mark
Submitted: 2006-07-11
Keywords:
Abstract: (click to show/hide) Economic popularity functions are central to the debate over whether voters use evaluations of the economy in their decision to support their government or not. This is of particular importance to the key democratic principle of electoral accountability that parties in power should and are held accountable for the outcomes of their actions and policies through the electoral process. Given the evidence from many nations that the economy is an issue of importance to the electorate, which they believe the government has control over, the inconsistent findings with regards to the impact of the economy on party popularity has made conclusive evaluations of the principle of electoral accountability elusive. This study demonstrates that the difficulty lies in a series of methodological flaws found in current approaches to developing popularity functions. Most analysts using public opinion time-series data have not applied the necessary methods to take into account the problems which such data can pose – problems such as complex error structures, shifting and compound non-stationary dynamics and noisy data. Accordingly, this study explicates a state-space Bayesian approach that addresses these methodological issues. In doing so, it outlines a technique that may be applied to a wide range of public opinion dynamic modelling issues.
Pickup, Mark
Submitted: 2006-07-11
Keywords:
Abstract: (click to show/hide) Economic popularity functions are central to the debate over whether voters use evaluations of the economy in their decision to support their government or not. This is of particular importance to the key democratic principle of electoral accountability that parties in power should and are held accountable for the outcomes of their actions and policies through the electoral process. Given the evidence from many nations that the economy is an issue of importance to the electorate, which they believe the government has control over, the inconsistent findings with regards to the impact of the economy on party popularity has made conclusive evaluations of the principle of electoral accountability elusive. This study demonstrates that the difficulty lies in a series of methodological flaws found in current approaches to developing popularity functions. Most analysts using public opinion time-series data have not applied the necessary methods to take into account the problems which such data can pose – problems such as complex error structures, shifting and compound non-stationary dynamics and noisy data. Accordingly, this study explicates a state-space Bayesian approach that addresses these methodological issues. In doing so, it outlines a technique that may be applied to a wide range of public opinion dynamic modelling issues.
The Balance Test Fallacy in Matching Methods for Causal Inference
Imai, Kosuke, King, Gary, Stuart, Elizabeth
Submitted: 2006-06-29
Keywords: causal inference, covariate balance, matching, treatment effect
Abstract: (click to show/hide) Matching methods are widely used to adjust for possibly confounded treatment assignment when making causal inferences. The success of the matching adjustment depends on generating as much equivalence as possible between the distribution of pre-treatment covariates in the treated and control groups. In numerous articles across a diverse variety of academic fields that use matching, researchers evaluate the degree of equivalence by conducting hypothesis tests, most commonly the $t$-test for the mean difference of each of the covariates in the two matched groups. We demonstrate that these hypothesis tests are fallacious and discuss better alternatives.
Imai, Kosuke, King, Gary, Stuart, Elizabeth
Submitted: 2006-06-29
Keywords: causal inference, covariate balance, matching, treatment effect
Abstract: (click to show/hide) Matching methods are widely used to adjust for possibly confounded treatment assignment when making causal inferences. The success of the matching adjustment depends on generating as much equivalence as possible between the distribution of pre-treatment covariates in the treated and control groups. In numerous articles across a diverse variety of academic fields that use matching, researchers evaluate the degree of equivalence by conducting hypothesis tests, most commonly the $t$-test for the mean difference of each of the covariates in the two matched groups. We demonstrate that these hypothesis tests are fallacious and discuss better alternatives.
Incentives, Complexity, and Motivations in Experiments
Bassi, Anna, Morton, Rebecca, Williams, Kenneth
Submitted: 2006-06-24
Keywords:
Abstract: (click to show/hide) We compare three motivation procedures in a voting experiment: 1) subjects paid a flat fee for participating, 2) subjects paid according to choices as is typical in a political economy experiment, and 3) subjects paid double the typical amount. We also vary complexity of the voting game. Financial incentives significantly increase the probability that subjects choose Bayesian-Nash predicted strategies. In the simpler game the typical financial incentive is sufficient; higher payments have no effect. But in the complex game, increasing financial incentives beyond the typical level is consequential. Further, repetition interacts with typical financial incentives in the complex game to increase the likelihood of Bayesian-Nash strategies. The evidence suggests that financial incentives increase subjects' cognitive attention to experimental tasks as individuals would be in comparable observational settings, which enhances theory evaluation in experiments and the external validity of the results.
Bassi, Anna, Morton, Rebecca, Williams, Kenneth
Submitted: 2006-06-24
Keywords:
Abstract: (click to show/hide) We compare three motivation procedures in a voting experiment: 1) subjects paid a flat fee for participating, 2) subjects paid according to choices as is typical in a political economy experiment, and 3) subjects paid double the typical amount. We also vary complexity of the voting game. Financial incentives significantly increase the probability that subjects choose Bayesian-Nash predicted strategies. In the simpler game the typical financial incentive is sufficient; higher payments have no effect. But in the complex game, increasing financial incentives beyond the typical level is consequential. Further, repetition interacts with typical financial incentives in the complex game to increase the likelihood of Bayesian-Nash strategies. The evidence suggests that financial incentives increase subjects' cognitive attention to experimental tasks as individuals would be in comparable observational settings, which enhances theory evaluation in experiments and the external validity of the results.
Scaling regression inputs by dividing by two standard deviations
Gelman, Andrew
Submitted: 2006-06-10
Keywords: regression, standardization, $z$-score
Abstract: (click to show/hide) Interpretation of regression coefficients is sensitive to the scale of the inputs. One method often used to place input variables on a common scale is to divide each variable by its standard deviation. Here we propose dividing each variable by {em two} standard deviations, so that the generic comparison is with inputs equal to the mean $pm 1$ standard deviation. The resulting coefficients are then directly comparable for untransformed binary predictors. We have implemented the procedure as a function in R. We illustrate the method with a simple public-opinion analysis that is typical of regressions in social science.
Gelman, Andrew
Submitted: 2006-06-10
Keywords: regression, standardization, $z$-score
Abstract: (click to show/hide) Interpretation of regression coefficients is sensitive to the scale of the inputs. One method often used to place input variables on a common scale is to divide each variable by its standard deviation. Here we propose dividing each variable by {em two} standard deviations, so that the generic comparison is with inputs equal to the mean $pm 1$ standard deviation. The resulting coefficients are then directly comparable for untransformed binary predictors. We have implemented the procedure as a function in R. We illustrate the method with a simple public-opinion analysis that is typical of regressions in social science.
Network Analysis and the Law: Measuring the Legal Importance of Supreme Court Precedents
Fowler, James, Johnson, Timothy, Spriggs II, James F., Jeon, Sangick, Wahlbeck, Paul
Submitted: 2006-06-02
Keywords:
Abstract: (click to show/hide) We construct the complete network of 28,951 majority opinions written by the U.S. Supreme Court and the cases they cite from 1792 to 2005. We illustrate some basic properties of this network and then describe a method for creating importance scores using the data to identify the most important Court precedents at any point in time. This method yields dynamic rankings that can be used to predict the future citation behavior of state courts, the U.S. Courts of Appeals, and the U.S. Supreme Court, and these rankings outperform several commonly used alternative measures of case importance.
Fowler, James, Johnson, Timothy, Spriggs II, James F., Jeon, Sangick, Wahlbeck, Paul
Submitted: 2006-06-02
Keywords:
Abstract: (click to show/hide) We construct the complete network of 28,951 majority opinions written by the U.S. Supreme Court and the cases they cite from 1792 to 2005. We illustrate some basic properties of this network and then describe a method for creating importance scores using the data to identify the most important Court precedents at any point in time. This method yields dynamic rankings that can be used to predict the future citation behavior of state courts, the U.S. Courts of Appeals, and the U.S. Supreme Court, and these rankings outperform several commonly used alternative measures of case importance.
Fitting Multilevel Models When Predictors and Group Effects Correlate
Bafumi, Joseph
Submitted: 2006-04-27
Keywords: Multilevel models, random effects, fixed effects, unit effects, group effects, Gauss-Markov
Abstract: (click to show/hide) Random effects models (that is, regressions with varying intercepts that are modeled with error) are avoided by some social scientists because of potential issues with bias and uncertainty estimates. Particularly, when one or more predictors correlate with the group or unit effects, a key Gauss-Markov assumption is violated and estimates are compromised. However, this problem can easily be solved by including the average of each individual-level predictors in the group-level regression. We explain the solution, demonstrate its effectiveness using simulations, show how it can be applied in some commonly-used statistical software, and discuss its potential for substantive modeling.
Bafumi, Joseph
Submitted: 2006-04-27
Keywords: Multilevel models, random effects, fixed effects, unit effects, group effects, Gauss-Markov
Abstract: (click to show/hide) Random effects models (that is, regressions with varying intercepts that are modeled with error) are avoided by some social scientists because of potential issues with bias and uncertainty estimates. Particularly, when one or more predictors correlate with the group or unit effects, a key Gauss-Markov assumption is violated and estimates are compromised. However, this problem can easily be solved by including the average of each individual-level predictors in the group-level regression. We explain the solution, demonstrate its effectiveness using simulations, show how it can be applied in some commonly-used statistical software, and discuss its potential for substantive modeling.
A Robust Transformation Procedure for Interpreting Political Texts
Martin, Lanny, Vanberg, Georg
Submitted: 2006-04-25
Keywords: content analysis, wordscores
Abstract: (click to show/hide) In a recent article in the American Political Science Review, Laver, Benoit, and Garry propose a new method for conducting content analysis. Their Wordscores approach, by automating text coding procedures, represents a fundamental advance in content analysis and will potentially have a large long-term impact on research across the discipline. In this research note, we contend that the usefulness of this procedure is unfortunately limited by the fact that the transformation procedure used by the authors (which is meant to allow for the substantive interpretation of results) has two significant shortcomings. Specifically, it distorts the metric on which content scores are placed—hindering the ability of scholars to make meaningful comparisons across texts—and it is very sensitive to the texts that are scored—opening up the possibility that researchers may generate, inadvertently or not, results that depend on the texts they choose to include in their analyses. We propose (and have written program code to implement) a transformation procedure that solves these problems.
Martin, Lanny, Vanberg, Georg
Submitted: 2006-04-25
Keywords: content analysis, wordscores
Abstract: (click to show/hide) In a recent article in the American Political Science Review, Laver, Benoit, and Garry propose a new method for conducting content analysis. Their Wordscores approach, by automating text coding procedures, represents a fundamental advance in content analysis and will potentially have a large long-term impact on research across the discipline. In this research note, we contend that the usefulness of this procedure is unfortunately limited by the fact that the transformation procedure used by the authors (which is meant to allow for the substantive interpretation of results) has two significant shortcomings. Specifically, it distorts the metric on which content scores are placed—hindering the ability of scholars to make meaningful comparisons across texts—and it is very sensitive to the texts that are scored—opening up the possibility that researchers may generate, inadvertently or not, results that depend on the texts they choose to include in their analyses. We propose (and have written program code to implement) a transformation procedure that solves these problems.
Incumbency as a Source of Contamination in Mixed Electoral Systems
Hainmueller, Jens, Kern, Holger Lutz
Submitted: 2006-03-10
Keywords: contamination, mixed electoral systems, causal inference, regression-discontinuity design, treatment effects, incumbency
Abstract: (click to show/hide) In this paper we demonstrate empirically that incumbency is a source of contamination in Germany's mixed electoral system. Using a quasi-experimental research design that allows for causal inference under a weaker set of assumptions than the regression models commonly used in the electoral systems literature, we find that incumbency causes a gain of $1.4$ to $1.7$ percentage points in PR vote shares. We also present simulations of Bundestag seat distributions to demonstrate that contamination effects caused by incumbency are sufficiently large to trigger significant shifts in parliamentary majorities
Hainmueller, Jens, Kern, Holger Lutz
Submitted: 2006-03-10
Keywords: contamination, mixed electoral systems, causal inference, regression-discontinuity design, treatment effects, incumbency
Abstract: (click to show/hide) In this paper we demonstrate empirically that incumbency is a source of contamination in Germany's mixed electoral system. Using a quasi-experimental research design that allows for causal inference under a weaker set of assumptions than the regression models commonly used in the electoral systems literature, we find that incumbency causes a gain of $1.4$ to $1.7$ percentage points in PR vote shares. We also present simulations of Bundestag seat distributions to demonstrate that contamination effects caused by incumbency are sufficiently large to trigger significant shifts in parliamentary majorities
JPDA: Just Plain Data Analysis
Klass, Gary
Submitted: 2006-01-22
Keywords: research methods, teaching, graphical display
Abstract: (click to show/hide) Just plain data analysis (JPDA) is the most common form of quantitative analysis in the discipline of political science. It involves technical, analytic and even artistic skills and knowledge that are not presented (or not well presented) in most quantitative or qualitative research methods courses or textbooks. Nevertheless these are skills that our students can apply in much their other political science coursework and in their future careers. Moreover, political science research would have much to gain were its practitioners to develop some of these skills.
Klass, Gary
Submitted: 2006-01-22
Keywords: research methods, teaching, graphical display
Abstract: (click to show/hide) Just plain data analysis (JPDA) is the most common form of quantitative analysis in the discipline of political science. It involves technical, analytic and even artistic skills and knowledge that are not presented (or not well presented) in most quantitative or qualitative research methods courses or textbooks. Nevertheless these are skills that our students can apply in much their other political science coursework and in their future careers. Moreover, political science research would have much to gain were its practitioners to develop some of these skills.
The Swing Voter's Curse in the Laboratory
Battaglini, Marco, Morton, Rebecca, Palfrey, Thomas
Submitted: 2006-01-12
Keywords:
Abstract: (click to show/hide) This paper reports the first laboratory study of the swing voter's curse and provides insights on the larger theoretical and empirical literature on 'pivotal voter' models. Our experiment controls for different information levels of voters, as well as the size of the electorate, the distribution of preferences, and other theoretically relevant parameters. The design varies the share of partisan voters and the prior belief about a payoff relevant state of the world. Our results support the equilibrium predictions of the Feddersen-Pesendorfer model, and clearly reject the notion that voters in the laboratory use naive decision-theoretic strategies. The voters act as if they are aware of the swing voter's curse and adjust their behavior to compensate. While the compensation is not complete and there is some heterogeneity in individual behavior, we find that aggregate outcomes, such as efficiency, turnout, and margin of victory, closely track the theoretical predictions.
Battaglini, Marco, Morton, Rebecca, Palfrey, Thomas
Submitted: 2006-01-12
Keywords:
Abstract: (click to show/hide) This paper reports the first laboratory study of the swing voter's curse and provides insights on the larger theoretical and empirical literature on 'pivotal voter' models. Our experiment controls for different information levels of voters, as well as the size of the electorate, the distribution of preferences, and other theoretically relevant parameters. The design varies the share of partisan voters and the prior belief about a payoff relevant state of the world. Our results support the equilibrium predictions of the Feddersen-Pesendorfer model, and clearly reject the notion that voters in the laboratory use naive decision-theoretic strategies. The voters act as if they are aware of the swing voter's curse and adjust their behavior to compensate. While the compensation is not complete and there is some heterogeneity in individual behavior, we find that aggregate outcomes, such as efficiency, turnout, and margin of victory, closely track the theoretical predictions.
Spatial Econometrics and Political Science
Darmofal, David
Submitted: 2006-01-10
Keywords: Spatial econometrics, Galton's problem, spatial autocorrelation
Abstract: (click to show/hide) Many theories in political science predict the spatial clustering of similar behaviors among neighboring units of observation. This spatial autocorrelation poses implications for both inference and modeling that are distinct from the more familiar serial dependence in time series analysis. In this paper, I examine how political scientists can diagnose and model the spatial dependence that our theories predict. This diagnosis and modeling entails three simple sequential steps. First, univariate spatial autocorrelation is diagnosed via global and local measures of spatial autocorrelation. Next, diagnostics are applied to a model with covariates to determine whether any spatial dependence diagnosed in the first step persists after the behavior has been modeled. If it does, the researcher simply chooses the spatial econometric specification indicated by the diagnostics. I present Monte Carlo results that demonstrate the importance of diagnosing and modeling spatial dependence in our data. I conclude by discussing how researchers can easily apply spatial econometric models in their research.
Darmofal, David
Submitted: 2006-01-10
Keywords: Spatial econometrics, Galton's problem, spatial autocorrelation
Abstract: (click to show/hide) Many theories in political science predict the spatial clustering of similar behaviors among neighboring units of observation. This spatial autocorrelation poses implications for both inference and modeling that are distinct from the more familiar serial dependence in time series analysis. In this paper, I examine how political scientists can diagnose and model the spatial dependence that our theories predict. This diagnosis and modeling entails three simple sequential steps. First, univariate spatial autocorrelation is diagnosed via global and local measures of spatial autocorrelation. Next, diagnostics are applied to a model with covariates to determine whether any spatial dependence diagnosed in the first step persists after the behavior has been modeled. If it does, the researcher simply chooses the spatial econometric specification indicated by the diagnostics. I present Monte Carlo results that demonstrate the importance of diagnosing and modeling spatial dependence in our data. I conclude by discussing how researchers can easily apply spatial econometric models in their research.
