Do Subtle Linguistic Interventions Priming a Social Identity as a Voter Have Outsized Effects on Voter Turnout? Evidence from a New Replication Experiment. (with Alan S. Gerber and Gregory A. Huber). Forthcoming at Political Psychology.
An ongoing debate in political psychology is about whether small wording differences have outsized behavioral effects. A leading example is whether subtle linguistic cues embedded in voter mobilization messages dramatically increase turnout. An initial study analyzing two small-scale field experiments argued that describing someone as a voter (noun) instead of one who votes (verb) increases turnout rates 11 to 14 points because the noun activates a person’s social identity as a voter. A subsequent study analyzing a large-scale field experiment challenged this claim and found no effect. But questions about the initial claim’s domain of applicability persist. The subsequent study may not have reproduced the conditions necessary for the psychological phenomenon to occur, specifically the electoral contexts were not competitive or important enough for the social identity to matter. To address the first of these critiques, as well as other potential explanations for different results between the first two studies, we conduct a large-scale replication field experiment. We find no evidence that this minor wording change increases turnout levels. This research provides new evidence that the strategy of invoking the self does not appear to consistently increase turnout and calls into question whether subtle linguistic cues have outsized behavioral effects.
The Effect on Turnout of Campaign Mobilization Messages Addressing Ballot Secrecy Concerns: A Replication Experiment. (with Alan S. Gerber, Gregory A. Huber, and Catlan E. Reardon). 2017. PLOS ONE. 12(8): e0182199.
Given the persistence of public doubts about the integrity of ballot secrecy, which depress turnout, two prior experiments have shown precise evidence that both official governmental and unofficial mobilization campaigns providing assurances about ballot secrecy increase turnout among recently registered nonvoters. To assess whether these findings replicate in other political settings, we describe a replication experiment where a non-governmental, non-partisan mobilization campaign sent similar treatment mailings containing assurances about ballot secrecy protections to recently registered nonvoters during the 2014 general election in Mississippi. We find that sending this mailer has no effect on turnout rates in this setting, which is characterized by an unusually low baseline turnout rate. These results are consistent with past research concluding that nonpartisan Get Out The Vote (GOTV) mail has very weak effects among very low turnout propensity registrants, and suggest that there are heterogeneous effects of ballot secrecy treatments associated with subjects’ characteristics and the electoral context.
Nongovernmental Campaign Communication Providing Ballot Secrecy Assurances Increases Turnout: Results from Two Large-Scale Experiments. (with Alan S. Gerber, Gregory A. Huber, and Andrew Gooch). 2017. Political Science Research and Methods. (First View June 6, 2017)
Doubts about the integrity of ballot secrecy persist and depress political participation among the American public. Prior experiments have shown that official communications directly addressing these doubts increase turnout among registered voters who had not previously voted, but evaluations of similar messages sent by non-governmental campaigns have yielded only suggestive effects. We build on past research and analyze two large-scale field experiments where a private non-partisan non-profit group sought to increase turnout by communicating ballot secrecy assurances in a direct mail voter mobilization campaign during the 2014 midterm election. Our main finding is that a private group’s mailing increases turnout by about 1 percentage point among registered nonvoters. This finding is precisely estimated and robust across state political contexts.
The Generalizability of Social Pressure Effects on Turnout Across High-Salience Electoral Contexts: Field Experimental Evidence from 1.96 Million Citizens in 17 States. (with Alan S. Gerber, Gregory A. Huber, and Andrew Gooch). 2017. American Politics Research. 45(4): 533-559.
Prior experiments show that campaign communications revealing subjects’ past turnout and applying social pressure to vote (the “Self” treatment) increase turnout. However, nearly all existing studies are conducted in low salience elections, raising concerns that published findings are not generalizable and are an artifact of sample selection and publication bias. Addressing the need for further replication in high salience elections, we analyze a field experiment involving 1.96 million subjects where a nonpartisan campaign randomly sent Self treatment mailers, containing a subject’s vote history and a comparison of each subject’s history to their state median registrant’s turnout behavior, in high salience elections across 17 states in 2014. Sending the Self mailer increases turnout by 0.7 points, or 2.2%. This effect is largely consistent across states, with somewhat larger effects observed in states with lower ex ante election salience. Our study provides precise evidence that social pressure effects on turnout are generalizable.
Working Papers and Manuscripts in Preparation
How Citizens Evaluate Trade-offs between Descriptive and Partisan Representation
An active debate in racial politics asks whether racial liberals and minorities would trade off descriptive representation for partisan (or other substantive) representation when they come into conflict. A dominant line of research argues that descriptive representation is paramount given its symbolic importance. Another burgeoning line of research instead argues that partisanship matters more in contemporary American politics when party and race are correlated. I evaluate these competing arguments in the context of racial redistricting, specifically when creating Democratic majority-minority districts has the ``perverse effect’’ of creating Republican districts and gerrymanders, which presents a trade-off between descriptive and partisan representation. Analyzing data from a survey experiment, I find that among those facing a trade-off, contra expectations in the literature, preferences for descriptive representation do not strictly dominate partisan preferences. I find no differences in trade-off evaluations between minorities and whites after controlling for subjects’ baseline preferences. Most importantly, I find strong evidence that among those facing this trade-off, preferences for Democratic legislative control dominate preferences for marginal increases in descriptive representation. These findings corroborate arguments that partisan preferences are central to racial representation in modern-day American politics, and have policy implications for political and legal theories of how to advance racial representation in a racially polarized party system.
Can the Government Deter Discrimination? Evidence from a Randomized Intervention in New York City (with Andrew M. Guess and Macartan Humphreys)
Racial discrimination persists despite established anti-discrimination laws. A common government strategy to deter discrimination is to publicize the law and communicate potential penalties for violations. We study this strategy by coupling an audit experiment with a randomized intervention involving nearly 700 landlords in New York City and report the first causal estimates of the effect on rental discrimination against Blacks and Hispanics of a targeted government messaging campaign. We uncover discrimination levels higher than prior estimates indicate, especially against Hispanics, who are approximately six percentage points less likely to receive callbacks and offers than whites. We find suggestive evidence that government messaging can reduce discrimination against Hispanics, but not against Blacks. The findings confirm discrimination’s persistence and suggest that government messaging can address it in some settings, but more work is needed to understand the contexts under which such appeals are most effective.
Do Minority Democrats Provide Better Minority Policy Representation than Non-Minority Democrats? A Design-Based Approach and Evidence from Latino Representation in the U.S. States
As racial and ethnic minorities have become closely aligned with the Democratic Party and as Democrats have become more racially liberal, what is the added value of descriptive minority representation for minority policy representation above and beyond electing a Democrat? I argue that theory can yield different predictions of the direction of this effect even when holding district and electoral conditions and the party of the elected legislator constant. To test these competing predictions, I examine the case of Latino representation in U.S. state legislative districts. Applying a regression discontinuity design to an original database of election returns in 23 U.S. states combined with data on over 900,000 introduced bills, I assess the effect of electing a Latino Democrat instead of a non-Latino Democrat in safe Democratic districts on the roll call voting and bill sponsorship behavior of the elected legislator on Latino-interest symbolic and policy bills. Legislators who just win in these district-elections exhibit similar roll call voting and bill sponsorship behavior on symbolic Latino-interest bills and anti-Latino policy bills regardless of their ethnicity. I find suggestive evidence that the marginal Latino Democrat elected to office from these districts sponsors more pro-Latino policy bills than the marginal non-Latino Democrat. These findings suggest that in a context where parties are ideologically and racially polarized such that Democratic representation is considered a proxy for racial representation, descriptive representation continues to matter to ensure that minority policy priorities are included on legislative agendas across the U.S. states.
Who Deserves Disability Insurance? Understanding Perceptions of Deservingness and Social Insurance Attitudes in the United States. (with Gregory A. Huber)
Perceptions of the deservingness of policy beneficiaries appear to shape attitudes toward redistributive programs. Less well understood is whether this heuristic shapes attitudes toward social insurance programs, which unlike redistributive welfare programs require beneficiaries to finance a risk pool in advance and only allow benefit receipt upon experiencing a pre-defined risk and claiming insurance. This may occur if individuals perceive these programs to grant benefits to a concentrated few, even if it is not designed to do so. Using the case of Social Security Disability Insurance (SSDI), a program whose size and beneficiaries’ deservingness have been politicized in recent decades, we evaluate this argument’s validity and examine the contours of public support for SSDI. Analyzing survey data and data from two experiments conducted on the 2016 Cooperative Congressional Election Study, we show that attitudes toward SSDI and its beneficiaries are shaped by mass perceptions of the deservingness of policy beneficiaries and by argumentative appeals about the risk-pooling features of the program. We discuss substantive implications for the study of policy feedback for politicized social insurance programs.
The Comparative Effectiveness on Turnout of Positively versus Negatively Framed Voter Mobilization Appeals (with Alan S. Gerber, Gregory A. Huber, and Catlan E. Reardon)
An ongoing debate in political science is whether mobilization appeals encouraging voting are more effective when framed positively, by highlighting the desirable behavior of a referent and encouraging consistent behavior, or when framed negatively, by highlighting the referent’s lack of desirable behavior as problematic. Few published studies have experimentally assessed this question and results are mixed across existing studies. We address the need for additional replication by designing and analyzing data from two field experiments conducted across four states in the 2014 primary and general elections. We find no difference between the effects on turnout of voter mobilization campaigns that frame such appeals positively rather than negatively. The results are not sensitive to election context, the mode of treatment delivery, or whether the appeals involve a social or self referent. Additional research is needed to understand the conditions under which framing appeals positively versus negatively have distinguishable effects on turnout.
When Does Increasing Mobilization Effort Increase Turnout? New Theory and Evidence from a Field Experiment on Reminder Calls (with Alan S. Gerber, Gregory A. Huber, and Catlan E. Reardon)
When does increasing mobilization effort increase turnout? Recent experiments find second calls containing a reminder to vote increase turnout beyond an initial contact. We argue existing studies cannot explain why reminder calls are effective because they test bundled treatments including a late mobilization attempt, a late mobilization attempt given earlier contact, and potentially activating reciprocity established in earlier contact. Moreover, existing work undertheorizes the causal role of reciprocity. We develop a reciprocity-based theory and design and analyze a two-round voter mobilization field experiment to test reciprocity as a mechanism explaining reminder call effects. Reminder calls increase turnout by 1.2 percentage points among subjects contacted in an earlier attempt. Enhancing reciprocity, operationalized as providing a reminder call offer during an early call, does not increase turnout beyond a second call. Lastly we fail to find heterogeneous effects of reminder calls by stated preference for a reminder or by stated vote intention.
Publication Bias: Before and After ICMJE Registration Requirements (with Grant M. Gordon and Macartan Humphreys)
There is increasing support for the use of research registries in social sciences. One possible advantage of the use of a registry is that it would limit the scope for publication or analysis biases that result from selecting statistically significant results. However, to date, there is surprisingly little evidence for the claim that registration will reduce these biases. We look to historical data from medical publishing for evidence, comparing the distribution of p-values before and after the introduction of registration in prominent journals. We couple this analysis with a pre-analysis survey of medical experts and social scientists to assess their prior expectations of the impact of registration on medical publishing and to assess their perceptions on the specificity and sensitivity of our test of effects. Although there is evidence of publication bias in medical studies, our registered analyses uncovered no evidence that registration affected that bias, leading us to moderately downgrade our confidence in the curative effects of registration.
Testing Mechanisms in Experiments: A Principal Stratification Approach
Social scientists often seek to evaluate claims about whether a treatment effect is “explained by” an intermediate or secondary post-treatment variable. Common identification strategies used in applied social experiments such as mechanism experiments and causal mediation analyses require identifying assumptions that are empirically untestable and often substantively implausible. In this research note, I present an alternative inferential approach for assessing mechanisms under minimal assumptions. Specifically, I provide an application of principal stratification, a well-established framework in the causal inference literature, to make inferences about the plausibility of hypothesized “mechanisms” by conceptualizing subjects as belonging to a principal stratum defined in terms of their potential response on a secondary post-treatment variable to all possible treatments; redefining inferential targets as principal causal effects, or average intent to treat effects by stratum; and assessing associations between principal causal effects and stratum membership. Implications for experimental design and analysis are discussed.
10 Things You Need to Know About Heterogeneous Treatment Effects. 2017. EGAP Methods Guide.