Survey Professionalism: New Evidence from Browsing Data
R&R at Political Analysis
Abstract: Online panels have become an important resource for research in political science, but the financial compensation offered to panelists incentivizes them to become ``survey professionals,” which raises concerns about data quality. We provide evidence on survey professionalism using behavioral web browsing data from three U.S. samples, recruited via Lucid, YouGov, and Facebook (total $n = 3,886$). Survey professionalism is common but varies across samples: By our most conservative measure, we identify 1.7% of respondents on Facebook, 7.9% of respondents on YouGov, and 34.3% of respondents on Lucid as survey professionals. However, evidence that professionals lower data quality is limited: they do not systematically differ demographically or politically from non-professionals and do not exhibit more response instability, although they are somewhat more likely to speed, straightline, and take questionnaires repeatedly. While concerns are warranted, we conclude that survey professionals do not, by and large, distort inferences of research based on online panels
Joint work with Bernhard Clemm von Hohenberg, Tiago Ventura, Magdalena Wojcieszak, Jonathan Nagler and Ericka Menchen-Trevino.
Testing the Causal Impact of Social Media Reduction Around the Globe
Under Review at Nature
Abstract: More than half of the world’s population now uses social media. There is widespread debate among the general public, politicians, and academics about social media’s impact on important variables, such as intergroup conflict and well-being. However, many studies on the impact of social media are either correlational or focus primarily on samples from the United States and Western Europe. We plan to conduct a global field experiment to measure the causal impact of incentivizing people to substantially reduce their social media usage across 23 countries (projected n > 8,000) for two weeks. We will then test hypotheses on how social media reduction influences four main outcomes: news knowledge, exposure to online hostility, intergroup attitudes, and well-being. We will also explore how the effects of social media reduction vary across different world regions, focusing on three theoretically-informed country-level moderators: income level, inequality, and democracy. This large-scale, high-powered global field experiment, and the global dataset resulting from it, will provide much-needed evidence to inform ongoing debates about the impact of social media usage across diverse cultural and political contexts.
*Led by Steven Rathje, Nejla Asimovic and Tiago Ventura. Coauthors: Sarah Mughal, Claire E. Robertson, Christopher Barrie, …., Joshua A. Tucker, Jay J. Van Bavel. *
Understanding Beliefs in Misinformation: Repetition, Partisan Signals and Bayesian Processing
Under Review
Abstract: Partisan motivations and repeated exposure are two dominant explanations for how individuals form beliefs about political misinformation.
Yet, there is little research that integrates these processes, despite each pointing to different interventions to combat the spread of false information, especially in online information environments.
In this paper, we situate both frameworks within a unified Bayesian model of belief formation and design survey experiments to explore several implications of this theoretical framework.
We find that both partisan motivated reasoning and prior exposure (`illusory truth effects’) manifest in our data, and that they exacerbate each other, painting a bleak picture of how the steady drumbeat of partisan-flavored misinformation online influences public beliefs.
However, we also find that the duration of these biases attenuates sharply over time and that attaching warning labels to false information mitigates the manifestation of both cognitive biases.
The findings suggest that partisan motivations dominate belief formation in political settings, with prior exposure to misinformation playing a secondary role.
These results contribute to a deeper understanding of cognitive biases in political information processing and provide a structured way of thinking about how best to understand the phenomenon of online misinformation, shifting the focus from the role of mass-level beliefs for falsehoods to the role of political elites and partisan media spreading rumors.
*Joint work with Jim Bisbbe, Sarah Graham and Joshua A. Tucker**
In Progress (Papers available upon request)
The Effects of Partisan Geographical Segregation on Online Behavior on Twitter
Abstract: Heightened levels of affective polarization have become a striking feature of contemporary politics.
The role of social media in facilitating ties between citizens who express similar political positions is often raised as the primary mechanism behind this.
However, more recent evidence shows the discourse around online echo chambers has vastly overstated their prevalence in online environments.
To understand these mixed findings, some have advanced an alternative causal mechanism whereby rather than isolating users in homogenous online communities, social media take users outside of their ``local bubbles” and actually increase their exposure to content that they would hardly encounter in their offline networks.
Our article empirically assesses these conflicting views by linking a novel dataset of voters’ offline partisan segregation extracted from publicly available voter files for 180 million US voters with their online network segregation on Twitter.
We investigate the relationship between offline and online partisan segregation using measures of geographical and network exposure to outgroup voters.
Using a dataset connecting over 900 thousand voters with their Twitter profiles, we provide evidence for high levels of both online and offline ingroup and, most importantly, that levels of such offline isolation, if anything, are higher offline than online.
Our results inform the emerging literature on political communication and the homophily of online networks and provide novel evidence about partisan sorting online and offline.
Joint work with Megan Brown, Tiago Ventura, Joshua A. Tucker, Jonathan Nagler.
- Presented at MPSA and APSA 2023
Reducing Social Media Usage During Elections: Evidence from a Multi-Country WhatsApp Deactivation Experiment
Abstract: Recent research has investigated how social media platforms may spread misinformation and encourage harmful political discourse, which fuels political polarization, prejudice, and offline violence. We deploy online field experiments in Brazil, India, and South Africa to examine how restricting the use of WhatsApp, the world’s most widely used messaging app, affects information exposure, political attitudes, and individual well-being. We incentivize participants to either (1) stop consuming multimedia content on WhatsApp or (2) limit overall WhatsApp usage to 10 minutes per day for four weeks ahead of each country’s elections. We find that our interventions significantly reduced participants’ exposure to misinformation, online toxicity, and uncivil discussions about politics—but at the expense of keeping up with true political news. Using a wide range of measures, we detected no changes to political attitudes, but uncovered substantial gains to individual well-being as treated participants substituted WhatsApp usage for other activities. Results highlight the complex trade-offs associated with the effects of social media use on information consumption and its downstream effects.
Joint Work with Rajeshwari Majumdar, Shelley Liu, Carolina Torreblanco, and Joshua Tucker
- Presented at APSA 2024