Case Study 01: The Pariser Test — Two Accounts, Two Realities

What Happens When You Create Politically Differentiated Google Profiles and Search for the Same Terms

Background

In the original filter bubble argument, Eli Pariser offered a thought experiment that has since been replicated by journalists, researchers, and curious individuals: create two internet accounts with different political profiles, use them to search for the same topics, and compare what each account sees. The experiment makes visible the personalization dynamics that are ordinarily invisible — the different search results, news feeds, and recommendations that different behavioral profiles generate for the same queries.

This case study examines the methodology, findings, and implications of the Pariser Test as it has been conducted across multiple platforms and by multiple investigators. It draws on journalistic investigations, academic research using similar methodologies, and the technical understanding of how personalization systems work to analyze what the test reveals — and what its limitations are — as a window into filter bubble dynamics.

The case study uses Google's search personalization as its primary focus because search is the most studied context for personalization testing, because Google's personalization is well-documented and widely experienced, and because search personalization raises particularly sharp epistemic concerns: when two people search for the same factual question and receive different results, the implications for shared reality are concrete and immediate.

Timeline

2010: Pariser's Original Observation Eli Pariser observes the differential personalization of his Facebook News Feed while working at MoveOn.org, laying the conceptual groundwork for the filter bubble argument.

2011: The Filter Bubble Publication and the Test's Popularization Pariser's book and TED Talk popularize the filter bubble concept and implicitly invite replication of the personalization difference experiment. Journalists and researchers begin conducting informal versions of the "two accounts" test.

2012-2015: Journalistic Investigations Multiple journalism outlets conduct documented experiments comparing search and social media results across politically differentiated accounts. The experiments consistently find differences, though their magnitude and interpretation are debated.

2016-2020: The Election Periods Filter bubble research intensifies around the 2016 and 2020 US elections. Academic researchers, journalism organizations, and advocacy groups conduct systematic versions of the two-account test for political queries. The Ad Fontes Media, AllSides, and similar organizations document content asymmetries across partisan information environments.

2018-2023: Academic Formalization Researchers including Robertson and colleagues develop methodologically rigorous approaches to auditing search personalization, using systematic data collection across multiple accounts and controlling for geographic and other confounds. Their research provides more precise estimates of personalization effects than journalistic experiments.

The Methodology

The core methodology is simple: create two accounts, cultivate different behavioral profiles, then use them to perform identical searches or browse the same platforms and compare outputs.

Account Creation

A careful version of the test creates two email addresses and associated accounts on Google (or the platform being tested). The accounts must be used consistently for several weeks to build meaningful behavioral profiles — a single day of use does not generate a sufficiently differentiated profile to produce large personalization effects.

Profile A (Liberal): Uses the account to search for progressive political topics, read liberal news sources, watch progressive YouTube content, follow liberal political accounts on linked platforms, and engage with content from organizations associated with progressive politics.

Profile B (Conservative): Uses a comparable account to search for conservative political topics, read conservative news sources, watch conservative YouTube content, follow conservative political accounts, and engage with content from organizations associated with conservative politics.

After several weeks of consistent differentiated use, both accounts search for identical queries and the results are compared.

What Is Compared

The comparison can examine multiple dimensions of the personalized information environment:

Search result composition: Which sources appear for the same queries? What is their perceived political orientation? In what order are they ranked?

News integration: When political search queries return news results, which news sources appear for each account? Do the accounts see the same stories or different stories about the same events?

Autocomplete suggestions: What does each account's search autocomplete suggest for partial political queries? Autocomplete reflects both general search popularity and individual behavioral profile.

Related searches: What related searches does each account's engine suggest after a completed query? Related search suggestions are personalized and may reflect each account's behavioral profile.

Key Findings

Findings from Journalistic Investigations

The most widely replicated journalistic finding is that politically differentiated accounts do receive somewhat different search results for contested political queries — but the differences are generally smaller than dramatic presentations of the filter bubble concept might suggest. A 2016 investigation by the Wall Street Journal's "Blue Feed, Red Feed" project (focused on Facebook rather than Google) provided the most vivid visualization: side-by-side columns showing the dramatically different news content served to politically differentiated Facebook profiles. For contested political events, the two feeds were often covering different aspects of the same story with different framing, different source authority, and sometimes different factual claims.

For Google search specifically, personalization differences tend to be more subtle than feed differences. Google's search results for many political queries remain relatively consistent across differentiated accounts at the level of which sources are returned, though rankings and relative prominence shift. The starkest personalization effects appear in related searches, autocomplete, and the knowledge panel content that appears alongside search results.

Findings from Academic Research

Robertson and colleagues' (2018) systematic audit of search personalization found that Google's personalization did affect which news sources appeared for political queries, but that the effect was smaller and more variable than the journalistic "filter bubble" narrative suggested. More significantly, they found that geographic variation (where you are searching from) produced larger differences in search results than political behavioral profile differences. Two users in the same city with different political profiles received more similar results than two users with identical profiles in different cities.

This finding challenges a simple account of political filter bubbles in search: the personalization effect is real, but geographic targeting (which is technically easier to implement) may produce larger information differences than behavioral political targeting.

Hannak and colleagues' (2013) earlier audit of Google personalization found that logged-in personalization affected 11.7 percent of search results compared to logged-out searches — a measurable effect, but less dramatic than popular accounts of filter bubbles might suggest.

Analysis: What the Test Reveals

The Pariser Test is a powerful pedagogical tool regardless of how large the measured effect is, because it makes visible something that is normally invisible: the fact that two people searching for the same thing may not see the same information. The test demonstrates personalization exists — and existence matters independently of magnitude, particularly for individual searches that are consequential.

The Magnitude Problem

The debate about how large filter bubble effects are in search is real and unresolved. Measured effects in academic research tend to be smaller than popular narrative suggests. But several considerations complicate the "filter bubbles are small" conclusion:

Compounding: Small personalization differences per search compound across thousands of searches over years. A person who consistently receives 10 percent different search results for political queries over five years has experienced a substantial cumulative informational divergence from someone with a different profile.

Critical moments: Personalization's impact may not be measured well by average effects. At specific high-stakes moments — searching for information during a breaking news event, researching a candidate before an election, verifying a claim you've encountered on social media — personalization effects may be more consequential than at typical moments.

Social cascades: If individuals with different personalized information environments share what they find on social media, the personalization effects cascade into social networks. Even modest individual personalization can produce large collective divergences when filtered through social sharing dynamics.

Interaction with motivation: Personalized results that confirm existing beliefs are likely to be read more carefully and remembered better than cross-cutting results — a motivated reasoning multiplier that amplifies modest personalization effects.

What the Test Cannot Show

The Pariser Test methodology has important limitations. It tests what a fully differentiated account sees, not what any real user sees. Real users do not exist at the extreme ends of political differentiation — most people's search histories are much more complex, mixing political queries with shopping, entertainment, local information, and everything else. The effect on a real user's search experience is likely more moderate than the two-account experiment suggests.

The test also cannot show whether personalization is the cause or consequence of information environment differences. People who have liberal or conservative political profiles search for different things in the first place — the different results they receive partly reflect their different query choices, not only different personalization of identical queries.

Voices from the Field

"The 'two accounts' experiment is the most effective way I know to communicate what personalization means to people who haven't thought about it. It's not that every search gives you completely different results. It's that for the questions that matter most — questions about politics, policy, contested facts — the information ecosystem you inhabit is subtly but consistently shaped by what you've already engaged with. That's what the filter bubble means in practice."

— Eli Pariser, in a 2018 interview on the ongoing relevance of the filter bubble concept

"Our audits found personalization effects that are real but modest for Google search. Where we found the largest effects was not in search results themselves but in the news integration features — the 'top stories' panels, the 'fact check' annotations, the related articles. These features draw on personalization signals significantly and they're the parts of search most likely to affect what people believe about contested political facts."

— Academic search audit researcher, paraphrased from conference presentation (2019)

Discussion Questions

The Pariser Test methodology has a known limitation: it creates extreme behavioral profiles that no real user fully matches. How should we interpret findings from extreme-profile experiments when thinking about the experiences of actual social media users? Does evidence from extreme profiles overstate or understate filter bubble effects in normal use?
Robertson et al. found that geographic variation produced larger search result differences than political behavioral profile differences. What are the implications of this finding? If geographic targeting creates larger information asymmetries than political personalization, how should this shift our understanding of the filter bubble problem?
The case study notes that personalization becomes more consequential at specific high-stakes moments — searching for information during breaking news, researching candidates before elections. What design choices could platforms make to reduce personalization specifically at high-stakes informational moments while maintaining personalization benefits in lower-stakes contexts?
The Wall Street Journal's "Blue Feed, Red Feed" visualization showed dramatically different Facebook feeds for liberal and conservative profiles, making invisible filter bubble dynamics visible. What were the effects of this visualization on public understanding of filter bubbles? What are the limits of this type of visualization as a tool for communicating filter bubble dynamics?
If personalization effects in search are smaller than popular narrative suggests (as some academic research indicates), but social media feed personalization effects are larger, what are the implications for how we should focus concern about filter bubbles? Should policy attention focus on search, social feeds, or both, and with what different approaches?

What This Means for Users

The Pariser Test's practical value for individual users is primarily epistemic: it demonstrates that what you see when you search is not what "the internet" returns but what an algorithm optimized around your behavioral profile returns. This awareness has several practical implications.

Logged-out search: Many search engines allow logged-out or incognito browsing that reduces (though does not eliminate) personalization effects. Using private browsing for politically sensitive searches provides a closer approximation to a non-personalized result.

Multiple search engine use: Different search engines (Google, Bing, DuckDuckGo, which explicitly minimizes personalization) use different personalization approaches. Consulting multiple search engines for contested political queries and comparing results is a manual version of the Pariser Test that provides some sense of how personalization may be shaping what you see.

Awareness of news integration features: The parts of search most affected by political personalization — news integration panels, featured snippets, and related articles — deserve particular critical attention. These are the features most likely to present personalized framings of contested factual questions.

Source diversification: Deliberately bookmarking and regularly consulting news sources with different editorial perspectives — not just trusting what personalized search surfaces — is the most direct response to political personalization in news search. This requires active effort against the convenience of personalization but maintains the informational breadth that single-account personalized search may not provide.