Case Study 2: The News Literacy Project and Checkology — Evaluating a Scalable Program

Overview

The News Literacy Project (NLP) is one of the most prominent and widely used news literacy education programs in the United States. Founded in 2008 by veteran journalist Alan Miller and relaunched with its flagship online platform Checkology in 2016, NLP has become a significant player in the effort to provide K–12 students with skills to evaluate news and information. This case study examines NLP's program design, curriculum philosophy, evidence base, and the larger questions its experience raises about scalable media literacy education.

Organizational Background

The News Literacy Project was founded with a specific mission: "to teach students how to know what to believe in the digital age." Miller, a Pulitzer Prize-winning journalist, began NLP as a journalist-in-the-classroom program, deploying working journalists to teach news literacy directly to students. This model leveraged the expertise and credibility of professional journalists while also making the intangible world of journalism production concrete and tangible for students.

As the organization grew, the journalist-in-classroom model became difficult to scale. Checkology, launched in 2016, was designed to bring NLP's curriculum to any classroom in the country (and eventually internationally) through an online learning management system. Teachers register their classes, students create accounts, and instruction occurs through a combination of video lessons, interactive exercises, and discussion activities that teachers facilitate.

By 2023, NLP reported that Checkology had been used by more than 600,000 students in all 50 U.S. states and more than 100 countries. The platform is available to teachers at no cost, funded primarily by philanthropic support.

Curriculum Design Philosophy

NLP's curriculum is built around a distinct philosophy that differentiates it from broader media literacy approaches:

Journalism-Centered: NLP focuses specifically on news and journalism rather than media literacy broadly construed. The organization believes that understanding how good journalism works — and what distinguishes it from other types of content — is the essential foundation for evaluating information in the digital age.

Evidence-Based Epistemology: A central NLP concept is the "news literacy standard" — evaluating claims based on independently verifiable evidence. NLP teaches students to ask: What is the evidence? Who gathered it? How was it verified? Can it be independently confirmed? This epistemological stance grounds news literacy in empirical verification rather than source trust or political ideology.

Journalist Practices as Models: Rather than simply teaching abstract critical thinking skills, NLP uses the practices of professional journalists as concrete models for information evaluation. If you would evaluate a news claim the way a careful journalist evaluates a tip — seeking corroboration, considering source motivations, checking primary documents — you have the essential tools for news literacy.

Non-Partisan Framing: NLP is explicit about its non-partisan stance. The organization does not evaluate specific outlets' political bias or tell students which outlets to trust. Instead, it teaches processes and skills applicable across the partisan spectrum.

Checkology Curriculum: Key Modules

Checkology's curriculum is organized into modules, each approximately 20–45 minutes in length. Key modules include:

"The Free Press and You"

This foundational module explains the First Amendment's press freedom protections, the history of press freedom in the United States, and the role of a free and independent press in democratic governance. Students learn why journalist independence from government control matters and how press freedom connects to other democratic rights. The module uses historical examples (publication of the Pentagon Papers, coverage of civil rights events) to illustrate the stakes of press freedom.

"Is It Legit?"

This module teaches systematic source evaluation using NLP's framework: - Type of content: Is this a news report, analysis, opinion, advertising, or entertainment? - Source evaluation: Who produced this? What is their expertise and independence? - Evidence evaluation: What evidence supports the claims? Is it verified? - Context evaluation: Is important context missing? Is this part of a larger story?

Students apply the framework to authentic examples of news content, sponsored content, and misinformation.

"The Verification Collection"

This module teaches specific fact-checking and verification techniques used by professional journalists and fact-checkers: - Reverse image search to verify photo authenticity - Using the Wayback Machine to check website history - Identifying astroturf organizations through tax records - Checking official government databases for factual claims - Reading corrections and retraction notices

"Propaganda Techniques"

This module teaches students to recognize seven classic propaganda techniques: name-calling, glittering generalities, transfer, testimonial, plain folks, card stacking, and bandwagon. Students analyze historical examples of wartime propaganda alongside contemporary political communication, developing pattern recognition for manipulative persuasion techniques.

"News vs. Opinion vs. Something Else"

Students learn to distinguish between news reporting (intended to inform based on verified facts), commentary and opinion (intended to persuade based on argument and evidence), advertising (intended to sell), and entertainment (intended to engage or amuse). The module addresses the blurring of these categories in contemporary media — native advertising, opinion masquerading as news, news with strong editorial framing — as a particular challenge for news literacy.

"Can You Spot the Difference? Satire and Fake News"

This module addresses the specific challenge of news satire (The Onion, The Babylon Bee) and deliberately false news content that mimics journalism's format. Students learn indicators that help distinguish satire (publication name, disclaimer language, absurdity signals) from fake news (missing bylines, unfamiliar URLs, implausible but un-satirical claims).

Teacher Experience and Implementation Challenges

NLP has conducted extensive surveys of teacher experiences with Checkology. Teachers generally report high satisfaction with the content quality, relevance, and student engagement. Key themes from teacher feedback:

Strengths: - High student engagement, particularly with video content featuring journalists - Practical, applicable skills that students can immediately apply - Non-partisan framing that allows use across politically diverse school communities - Flexible integration options (as a standalone unit or embedded in English Language Arts, social studies, or journalism courses) - Strong teacher support resources including discussion guides and extension activities

Challenges: - Technology access and infrastructure disparities: schools with limited device access or unreliable internet struggle to use a web-based platform - Time pressure: fitting Checkology modules into overcrowded curricula is challenging; teachers often use only 2–3 modules rather than the full curriculum - Shallow implementation: the most common implementation pattern — a few modules used once, without sustained practice — is likely insufficient for durable learning - Assessment limitations: teachers report that assessing news literacy skills authentically is challenging; the multiple-choice assessments built into Checkology modules measure recall rather than transfer

Evidence Base: What the Research Shows

NLP's Internal Assessment

NLP has conducted internal assessments of Checkology's effectiveness using pre/post surveys measuring: - Knowledge of journalism concepts and practices - Confidence in evaluating news sources - Self-reported critical thinking behaviors

Across multiple cohorts, NLP reports pre/post gains on these measures, with students showing improved knowledge and increased confidence after completing Checkology. However, several methodological limitations constrain interpretation:

  • No comparison/control group: changes could reflect maturation, other learning experiences, or test-retest effects rather than Checkology specifically
  • Self-reported confidence may not reflect actual competency
  • Assessments measure recall and confidence rather than authentic news evaluation behavior
  • No follow-up measurement: whether gains persist over time is unknown

External Academic Research

Several academic researchers have examined news literacy interventions broadly, with some relevance to NLP's approach:

Kahne & Bowyer (2017): Research on youth civic media literacy found that news literacy education could improve students' ability to evaluate news quality, but that effects were substantially mediated by political interest and prior knowledge. Students with higher political interest and greater prior exposure to news showed larger gains.

Pennycook & Rand (2019): Research on "accuracy nudges" — brief interventions prompting people to consider accuracy — found modest effects on reducing intended sharing of misinformation. This research is tangentially supportive of news literacy approaches emphasizing slow, deliberate evaluation.

Craft, Ashley & Maksl (2016): A study of college students found that those with greater news literacy competencies (measured by an established news literacy scale) were better able to identify false news, suggesting that news literacy knowledge does translate to performance on identification tasks.

Limitations of the Evidence Base: No peer-reviewed RCT study has specifically evaluated Checkology's effectiveness compared to a control condition. The evidence supporting news literacy education generally — and NLP's program specifically — is suggestive but not definitive by the standards of rigorous educational research.

The Scalability Question

NLP's Checkology platform represents a serious attempt to solve one of media literacy education's fundamental challenges: how to deliver quality media literacy instruction at scale in a country where media literacy education is not mandated in most states and where time in the curriculum is already severely constrained.

The Checkology model makes tradeoffs characteristic of any scalability effort:

Gains from scalability: - Low cost to teachers and schools (free platform) - Consistent curriculum quality across diverse settings - Professional video content beyond what any individual teacher could create - Easy entry for teachers with no prior media literacy training

Losses from scalability: - Reduced local relevance and contextualization - Less integration with other subjects (Checkology as standalone unit vs. embedded practice) - Less teacher agency and professional judgment - Reduced emphasis on sustained, deep learning (program completion incentivizes module completion rather than mastery)

Educational researchers studying scalable interventions have consistently found that fidelity to implementation is as important as program design — and that scale tends to reduce implementation fidelity. A Checkology unit completed thoughtfully by an engaged teacher who connects it to ongoing class themes is a very different educational experience from the same unit completed perfunctorily by students clicking through videos to earn completion credit.

Critical Assessment: What Checkology Does Well and What It Misses

What Checkology Does Well

  • Provides accessible, engaging content that introduces news literacy concepts effectively
  • Grounds news literacy in journalism practices, giving students a concrete process model
  • Maintains genuine non-partisanship while still addressing politically sensitive topics
  • Teaches specific, practical verification skills (reverse image search, fact-checking databases)
  • Provides a free, scalable resource to teachers who lack time to develop their own curriculum

What Checkology Misses or Underemphasizes

  • Structural and political economy critique: Checkology teaches students to evaluate individual stories and sources but does not develop strong frameworks for understanding how ownership, advertising, and commercial pressures shape news production systemically.
  • Algorithmic literacy: The curriculum addresses social media and viral content but is less strong on algorithmic recommendation systems, filter bubbles, and platform incentive structures.
  • Production education: Students learn to evaluate media but have limited opportunity to create it; the production experience that builds deepest understanding of media construction is absent.
  • Affective dimensions: Checkology is primarily a cognitive and skills-based curriculum; the motivational, emotional, and social dimensions of news consumption — why people share misinformation, how tribal identity affects evaluation — receive less attention.
  • Transfer and habit formation: Like most brief curriculum interventions, Checkology is strongest in building knowledge and initial skills; its ability to create lasting habits of news literacy behavior is less clear.

Policy Implications

The News Literacy Project's experience has several implications for media literacy education policy:

  1. Scalability requires sustained commitment, not one-time programs: NLP's most effective implementations involve teachers who use Checkology as part of a sustained, integrated curriculum, not as a one-time unit. Policy should incentivize sustained integration over one-time exposure.

  2. Free, high-quality resources are necessary but not sufficient: Making Checkology free reduces the access barrier but does not ensure meaningful implementation. Teacher training and professional development must accompany curriculum resources.

  3. Assessment matters: Lack of rigorous external evaluation of news literacy programs — including NLP — leaves policymakers without the evidence base needed to make informed decisions about curriculum investment. Funding for rigorous evaluation should accompany program funding.

  4. State curriculum standards are crucial: Media literacy education is most consistently implemented where state curriculum standards require it. NLP's advocacy for state media literacy legislation is a strategically important complement to its curriculum work.

Discussion Questions

  1. NLP's non-partisan framing is both a practical asset (allowing use across politically diverse communities) and a possible limitation (preventing engagement with systemic critiques of media). How would you evaluate this tradeoff?

  2. What additional research would be needed to definitively establish whether Checkology effectively improves students' news literacy in ways that persist beyond the classroom?

  3. NLP's curriculum is journalism-centered: it teaches news literacy through the lens of journalism practices. What would be gained or lost by adopting a broader media literacy framework instead?

  4. The scalability of Checkology depends on its being teacher-independent (teachers don't need special expertise to use it). What are the educational costs of this design choice?

Sources and Further Reading

  • News Literacy Project. (2023). Checkology Teacher Survey Results. Internal report.
  • Craft, S., Ashley, S., & Maksl, A. (2016). News media literacy and conspiracy theory endorsement. Communication and the Public, 1(4), 388–401.
  • Kahne, J., & Bowyer, B. (2017). Educating for democracy in a partisan age. American Educational Research Journal, 54(1), 3–34.
  • Miller, A. C. (2019). The News Literacy Project: A solution to fake news in the digital age. Nieman Reports.
  • Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50.
  • Ashley, S., Poepsel, M., & Willis, E. (2010). Media literacy and news credibility. Journal of Media Literacy Education, 2(1), 37–46.