Case Study 35.1: The Bad News Game — Designing and Evaluating a Misinformation Inoculation Game

Overview

Bad News is a browser-based "fake news game" developed through a collaboration between DROG (a Dutch communications agency specializing in behavioral influence campaigns) and Sander van der Linden's Social Decision-Making Lab at the University of Cambridge. First released publicly in 2018 at getbadnews.com, the game asks players to take on the role of a misinformation producer, using six manipulation techniques to grow a social media following and spread false narratives through a fictional online environment.

The game represents a landmark application of inoculation theory to real-world misinformation. Rather than lecturing users about media literacy, it creates an experiential learning environment in which players produce misinformation—gaining firsthand knowledge of how the techniques work and, it is theorized, developing psychological resistance to those same techniques when encountered from external sources.

This case study examines the design principles underlying Bad News, reviews the empirical evidence for its effectiveness, assesses its limitations, and summarizes replication and extension studies that have followed in its wake.

Design Principles

The Active Learning Premise

The central design decision of Bad News is to make players producers rather than consumers of misinformation. This choice reflects a specific theory of learning: that active engagement with material leads to deeper processing, better retention, and more robust transfer than passive observation.

In learning science terms, the game exploits several well-established mechanisms:

Generation effects. When learners generate material (rather than simply reading it), they process it more deeply and retain it better. In Bad News, players generate manipulative narratives by choosing from menus of options, selecting which techniques to deploy in each situation. Even this constrained form of generation appears to promote deeper processing than simply reading about the techniques.

Perspective-taking. By placing players in the role of a misinformation producer, the game invites them to understand the goals, strategies, and decision logic of bad actors. This perspective-taking may build a form of "theory of mind" for misinformation: an understanding of what misinformation producers are trying to do and why their techniques are effective.

Feedback loops. The game provides immediate feedback on the consequences of each choice, in the form of follower counts and credibility scores. This feedback connects players' choices to outcomes in a way that supports causal learning: doing X leads to Y.

Escalating challenge. Bad News introduces its six manipulation techniques sequentially, building from simpler to more complex techniques. This "scaffolded" structure is consistent with learning science research on the importance of sequencing instruction from foundational to advanced skills.

Narrative engagement. The game embeds the manipulation techniques in a developing narrative, which may promote deeper processing through "narrative transportation" — the state of being absorbed in a story, which has been shown to reduce counter-arguing and increase the impact of narrative-embedded information.

The Six Techniques

Bad News targets six manipulation techniques that its designers identified as central to contemporary online misinformation:

1. Impersonation. Creating fake social media accounts that impersonate real individuals or organizations (journalists, scientists, government agencies) to lend false credibility to misinformation. The game shows players how to create a convincing fake account and explains why audiences are susceptible to impersonation.

2. Emotion. Using emotionally charged language, imagery, and framing to provoke strong emotional responses (fear, outrage, disgust) that bypass analytical thinking. The game demonstrates how emotional triggers can override critical evaluation.

3. Polarization. Amplifying divisions between social groups (political, ethnic, religious) by framing issues in terms of us versus them. The game shows how polarizing content generates engagement and spreads virally.

4. Conspiracy. Framing events as the products of secret, malevolent coordination by powerful actors who suppress the truth. The game demonstrates the logical structure of conspiracy theories and their appeal.

5. Discrediting. Attacking the credibility of legitimate sources (scientists, journalists, fact-checkers, government officials) as a way of undermining confidence in accurate information. The game shows how discrediting can be more effective than providing counter-evidence.

6. Trolling. Using deliberately provocative, inflammatory content to generate outrage, confusion, and engagement, often without a specific informational goal. The game demonstrates how trolling can be deployed strategically to disrupt public discourse.

Satirical Tone and Moral Design

An important design choice is the game's deliberately satirical tone. Players are cast as obviously villainous characters pursuing transparently bad ends. The game uses humor and cartoonish villainy to signal that this is an educational exercise, not a genuine endorsement of the techniques being taught.

This framing serves two functions. It makes the game more engaging and reduces the ethical discomfort of "playing the bad guy." And it signals that the manipulation techniques are meant to be recognized as such — the game is about learning to see through these techniques, not to deploy them effectively.

The ethical stakes of the design are real, however. If players use the game to become more sophisticated misinformation producers rather than more resistant consumers, the intervention would backfire. The designers have been attentive to this risk, and the research literature has not found evidence of such iatrogenic effects — but the concern motivates ongoing monitoring.

Empirical Evaluation

Roozenbeek and van der Linden (2019): The Foundational Study

The first major empirical evaluation of Bad News was conducted by Jon Roozenbeek and Sander van der Linden (2019) and published in Palgrave Communications. The study recruited a large online convenience sample (n = 15,000+) through the game's website, making it one of the largest behavioral experiments on misinformation at the time.

Design: Participants completed a set of pre-game assessments (rating the reliability of fake and real news headlines), played Bad News, and then completed identical post-game assessments. The study used a within-subjects design without a separate control condition — a methodological limitation acknowledged by the authors.

Key findings: - Participants showed a significant decrease in the perceived reliability of fake news headlines after playing Bad News (the change was significant for fake news but not for real news, suggesting that the inoculation did not produce general skepticism). - The improvement was consistent across the political spectrum, with no significant moderation by political ideology. - The improvement was found across age groups, though effect sizes were smaller for older adults. - The improvement was found across countries, though effect sizes varied.

Limitations: The absence of a control condition means that the study cannot rule out alternative explanations (e.g., test-retest practice effects). The convenience sample recruited through the game's website is highly self-selected. The pre-post design without control makes effect size estimation imprecise.

Maertens et al. (2021): Longitudinal Randomized Controlled Trial

Roozenbeek Maertens, and colleagues (2021) conducted the most methodologically rigorous evaluation of Bad News, published in the Journal of Experimental Psychology: Applied. This study added a randomized control condition and multiple follow-up time points.

Design: Participants were randomly assigned to play Bad News or to play an active control game (an unrelated puzzle game). Outcome measures were administered before the intervention, immediately after, at two weeks, and at four weeks.

Key findings: - Bad News produced a significant improvement in fake news detection accuracy relative to the control condition, with an effect size of approximately d = 0.35 at immediate post-test. - The inoculation effect was maintained at two weeks follow-up, with a small but significant reduction in effect size. - By four weeks, the effect size had declined substantially, approaching but not reaching the level of the control condition. - The pattern suggests meaningful inoculation decay beginning around two weeks, with near-full decay by four to six weeks.

Implications: The results support the effectiveness of Bad News for immediate protection but raise concerns about durability, motivating further research on booster interventions.

Cross-Cultural Replication (Roozenbeek et al., 2022)

A subsequent study examined whether the Bad News inoculation effect holds across diverse cultural and political contexts. Participants from 19 countries played Bad News, and outcomes were compared across countries stratified by media freedom index and media literacy levels.

Key findings: - Significant inoculation effects were found in all 19 countries. - Effect sizes were inversely related to baseline media literacy: smaller effects in countries with higher existing media literacy (consistent with a ceiling effect interpretation). - Effect sizes were not consistently related to political freedom or media freedom, suggesting that the game's mechanism is not dependent on specific political contexts. - There was no evidence of differential effects based on the political orientation of participants' home countries.

Implications: The cross-cultural generalizability of Bad News is strong, making it a viable candidate for global deployment. The smaller effects in high-literacy contexts suggest that the game may add more value in contexts where baseline literacy is low — exactly the contexts where misinformation is likely to have the most impact.

Limitations and Criticisms

Self-Selection and Motivation

A consistent limitation across Bad News evaluations is the self-selection problem. Participants who voluntarily seek out and play a "fake news game" are likely already more interested in media literacy than the general population. They may have higher baseline detection accuracy and stronger motivation to improve. This means that effects observed in research studies may overestimate the effects that would be observed in populations who encounter the game without prior interest.

This problem is inherent to the game-based approach: games can be deployed broadly, but they cannot force engagement. A person who clicks through a game without genuine engagement is unlikely to benefit from the experience. This motivates the interest in shorter, more passive prebunking formats (like the YouTube ads studied by Google/Cambridge), which do not require voluntary engagement.

Demand Effects and Social Desirability

In studies where participants know they are being studied, demand effects may inflate performance: participants may perform better on post-tests because they understand that the study is about media literacy and feel motivated to appear discerning. This concern is partially addressed by the use of active control conditions (which also create demand effects), but residual concerns remain.

Transfer to Real-World Behavior

Most evaluations of Bad News measure ability to identify manipulative techniques in controlled laboratory-style tasks. There is limited direct evidence that improved detection accuracy in these tasks translates to changed behavior in real social media environments — sharing fewer false stories, clicking on fewer misinformation links, updating beliefs in response to corrections. The gap between laboratory performance and real-world behavior is a persistent challenge in media literacy research generally.

The "Villain" Problem and Unintended Effects

A theoretical concern that has received less empirical attention than it deserves is the possibility that playing Bad News teaches some users to be more sophisticated misinformation producers rather than more resistant consumers. The game's explicit training in impersonation, emotional manipulation, and conspiracy framing could in principle serve as a tutorial for aspiring bad actors. The available research has not found evidence of such effects in representative samples, but the question merits ongoing monitoring.

Replication Studies and Extensions

International Versions

Bad News has been translated into multiple languages and adapted for different cultural contexts. Studies of localized versions have generally replicated the core findings, with some variation in effect sizes across languages and cultural contexts. Adaptations have been required to ensure that the cultural references, platform types, and political framing are appropriate for each context.

Bad News Junior

A version of Bad News adapted for children (approximately ages 8-13), called Bad News Junior, was developed and released in 2021. The game simplifies the manipulation techniques and uses age-appropriate scenarios (school events, playground rumors) rather than the adult political context of the original. Early evaluations suggest that the junior version produces significant effects on children's ability to identify manipulative content, though effect sizes are somewhat smaller than for the adult version.

Domain-Specific Variants

Several domain-specific variants of the Bad News format have been developed and evaluated:

Go Viral! (COVID-19 misinformation, three techniques) was evaluated in a pre-registered RCT finding significant improvement in COVID-19 misinformation detection (Basol et al., 2020).

Harmony Square (election integrity misinformation) was developed in collaboration with CISA and has been used in educational contexts, though peer-reviewed evaluation data is more limited.

Cat Park (climate misinformation) was developed in collaboration with the Cambridge lab and has been deployed in educational contexts in the UK.

Mechanistic Studies

Beyond effectiveness evaluations, several studies have examined the mechanisms by which Bad News produces its effects. Studies examining the mediating role of "manipulation awareness" (the conscious recognition of specific manipulation techniques) have found that technique awareness partially mediates the effect on fake news detection accuracy. Studies examining the moderating role of prior exposure to misinformation have found that the inoculation effect is larger for participants with less prior exposure, consistent with the theoretical prediction that prebunking works better as prevention than as treatment.

Policy Implications and Deployment

Platform Integration

Bad News has been embedded in the curricula of several European schools and universities, and the game's free availability at getbadnews.com has facilitated wide organic adoption. But more structured deployment — as part of a deliberate media literacy strategy rather than voluntary individual use — requires institutional support, teacher training, and integration with assessment frameworks.

The Question of Booster Shots

The longitudinal data showing significant inoculation decay raises the practical question of how booster shots should be designed and delivered. One approach would be to redesign Bad News to include periodic "mini-games" or brief re-exposure activities that could be delivered at intervals of two to four weeks. Another approach would be to embed brief inoculation content in the social media platforms themselves, using the platform's notification infrastructure to deliver booster content.

Scaling with AI

One promising direction is the use of AI-generated content to dynamically adapt inoculation games to current misinformation trends. If the game's scenarios could be automatically updated based on real-time analysis of misinformation narratives circulating on social media, the game could maintain relevance as the misinformation landscape evolves. This would address one of the fundamental limitations of static game content: its inability to keep pace with the speed of misinformation adaptation.

Conclusion

Bad News represents a creative and empirically grounded application of inoculation theory to the contemporary misinformation challenge. Its design principles — active learning, technique-based instruction, escalating challenge, narrative engagement — are well-supported by learning science, and its empirical evaluations have found significant, replicable effects across diverse populations and cultural contexts.

The game's limitations — self-selection, transfer questions, inoculation decay — are significant but not disqualifying. They point toward productive directions for further research and development: larger-scale deployments, booster interventions, and passive delivery formats that do not require voluntary engagement.

Perhaps most importantly, Bad News has demonstrated that inoculation can be delivered in an engaging, accessible format that does not require participants to sit through lectures or absorb dense factual information. This proof of concept has been influential in inspiring subsequent prebunking games, the Google YouTube campaigns, and the broader prebunking research agenda. Its legacy is as much in what it demonstrated was possible as in its direct effects on any particular population of players.

Discussion Questions for Case Study 35.1

1. The Bad News game asks players to produce misinformation in order to build resistance to it. What assumptions about learning does this design rest on? Are those assumptions well-supported by the learning science literature?

2. The foundational Roozenbeek and van der Linden (2019) study used a within-subjects design without a control condition. What specific alternative explanations for the observed effects does this design fail to rule out? How consequential are these limitations?

3. Inoculation effects in the longitudinal studies decay substantially by four to six weeks. From a public health perspective, how significant is this finding? What real-world deployment strategies would be necessary to maintain effectiveness given this decay rate?

4. The self-selection problem — that users who voluntarily play Bad News are likely already more media-literate than the general population — is a persistent challenge for effectiveness research. What research designs could address this limitation?

5. Should Bad News or similar games be made compulsory in school settings? What are the ethical arguments for and against mandatory participation in prebunking games?