Why "Significance Bias" Matters for Political Science
It is common practice among political scientists that quantitative results must be statistically significant as a necessary condition of their receiving further scientific consideration. Unfortunately, statistically significant estimates of a relationship are typically larger than the true magnitude. The consequence is that scientifically notable (e.g., published) results will typically overstate the substantive significance of their findings, particularly early in a research program when null or weak findings are unpublishable. Worse yet, the actual false positive rate is often much larger than the nominal value of the significance test. We show that these forms of “significance bias” have a substantive impact on political science research. We also propose new methodological strategies to correct for and prevent significance bias.
Keywords not supplied
Document ID Number