Case Study 28-1: Bayesian Reasoning and COVID-19 Testing — Understanding False Positives

Overview

The COVID-19 pandemic generated one of the largest rapid deployments of diagnostic testing in human history. Within months of the pathogen being identified, PCR tests, rapid antigen tests, and eventually at-home tests were being administered to billions of people. The interpretation of those tests — particularly the meaning of a positive result — required probabilistic reasoning that most of the public, and much of the press, were unprepared to apply.

This case study works through the Bayesian mathematics of COVID-19 test interpretation across different testing contexts, demonstrates how the same test can have dramatically different predictive value depending on prevalence, and traces how failures of probabilistic reasoning in public communication contributed to both over-reaction and under-reaction at different phases of the pandemic.

Section 1: The Tests and Their Characteristics

PCR Tests

Polymerase Chain Reaction (PCR) tests detect viral RNA by amplifying it until it becomes detectable. They are highly sensitive — able to detect even small amounts of viral material — and were the gold standard for COVID-19 diagnosis throughout the pandemic.

Typical PCR test characteristics: - Sensitivity: 95-98% (some studies report higher in optimally timed samples) - Specificity: 99-99.9% (very low false positive rate due to highly specific primers) - Turnaround time: 12-24 hours (or longer during peak demand)

Rapid Antigen Tests

Rapid antigen tests detect viral protein fragments (antigens) rather than RNA. They are faster and cheaper but less sensitive.

Typical antigen test characteristics: - Sensitivity: 60-85% (varies by viral load; higher sensitivity when viral load is high, i.e., during peak infectiousness) - Specificity: 97-99.5% - Turnaround time: 15-30 minutes

The lower sensitivity of antigen tests has a silver lining: they tend to be positive primarily when viral load is high, which corresponds to the period of peak transmissibility. A positive rapid antigen test is therefore a strong signal of current infectious status.

Section 2: Applying Bayes' Theorem — Three Scenarios

The same test, with the same sensitivity and specificity, has dramatically different predictive value in three different contexts. Let us work through all three systematically.

Scenario A: Asymptomatic Screening, Low Prevalence (Early Pandemic)

Context: March 2020, early in the pandemic. A hospital is screening asymptomatic staff for COVID-19 to protect patients. Community prevalence is estimated at 0.3% (very low, as the pandemic is just beginning in the region).

Test: PCR, sensitivity = 97%, specificity = 99.5%

Calculation: - P(COVID) = 0.003 - P(no COVID) = 0.997 - P(positive | COVID) = 0.97 - P(positive | no COVID) = 0.005

P(positive) = (0.003 × 0.97) + (0.997 × 0.005) = 0.00291 + 0.004985 = 0.007895

P(COVID | positive) = 0.00291 / 0.007895 ≈ 36.9%

Interpretation: A positive PCR test for an asymptomatic healthcare worker in a low-prevalence setting has only about a 37% probability of being a true positive. Nearly two-thirds of positive results in this context would be false positives.

In 10,000 people tested: - 30 have COVID: 29.1 test positive (true positives), 0.9 test negative - 9,970 don't have COVID: 49.9 test positive (false positives), 9,920.1 test negative - Total positive tests: 79 - True positives among positive tests: 29.1/79 ≈ 36.9%

Policy implication: If a hospital excludes every staff member who tests positive from duty, and two-thirds of those exclusions are false positives, this imposes enormous staffing costs during a crisis. The correct response is not to ignore positive tests, but to understand their predictive value in context — possibly requiring confirmatory testing, or adjusting the threshold for staff exclusion based on the clinical risk-benefit calculation.

Scenario B: Symptomatic Testing, Moderate Prevalence (Delta Wave)

Context: August 2021, during the Delta variant wave. A testing clinic sees patients with COVID-like symptoms (fever, cough, loss of smell). Among symptomatic patients presenting at this clinic, the probability of COVID is estimated at 30% (based on local surveillance data and symptom prevalence).

Test: Rapid antigen test, sensitivity = 75%, specificity = 98%

Calculation: - P(COVID) = 0.30 - P(no COVID) = 0.70 - P(positive | COVID) = 0.75 - P(positive | no COVID) = 0.02

P(positive) = (0.30 × 0.75) + (0.70 × 0.02) = 0.225 + 0.014 = 0.239

P(COVID | positive) = 0.225 / 0.239 ≈ 94.1%

Interpretation: A positive antigen test for a symptomatic patient in a high-prevalence testing context has a 94% probability of being a true positive. This is the appropriate context for antigen testing — high prior probability enhances the test's predictive value dramatically.

In 1,000 symptomatic patients tested: - 300 have COVID: 225 test positive (true positives), 75 test negative (false negatives — missed!) - 700 don't have COVID: 14 test positive (false positives), 686 test negative - Total positive tests: 239 - True positives among positive tests: 225/239 ≈ 94.1%

But note: Among the 300 true cases, 75 tested negative (false negatives) — a 25% miss rate for the antigen test. These 75 symptomatic COVID patients were falsely reassured. This is the cost of lower sensitivity: the false negatives send infectious people away without isolation recommendations.

Scenario C: Pre-Event Screening, Elevated Prevalence (Mass Testing)

Context: January 2022, Omicron wave. An organization is screening all employees before a large in-person gathering. Community prevalence is high: an estimated 5% of the region's population is currently infected.

Test: PCR, sensitivity = 97%, specificity = 99.5%

Calculation: - P(COVID) = 0.05 - P(no COVID) = 0.95 - P(positive | COVID) = 0.97 - P(positive | no COVID) = 0.005

P(positive) = (0.05 × 0.97) + (0.95 × 0.005) = 0.0485 + 0.00475 = 0.05325

P(COVID | positive) = 0.0485 / 0.05325 ≈ 91.1%

Interpretation: At 5% prevalence, a positive PCR test has a 91% probability of being a true positive. The high prevalence greatly improves the test's predictive value compared to Scenario A.

In 1,000 employees tested: - 50 have COVID: 48.5 test positive (true positives), 1.5 test negative - 950 don't have COVID: 4.75 test positive (false positives), 945.25 test negative - Total positive tests: 53.25 - True positives: 48.5/53.25 ≈ 91.1% - False positives: 4.75 (less than 5 people wrongly excluded)

Section 3: How Prevalence Changes Everything

The three scenarios demonstrate a critical principle: the same test produces dramatically different posterior probabilities depending on pre-test prevalence.

This is not a quirk of the specific numbers chosen — it is a mathematical consequence of Bayes' Theorem that applies to any diagnostic test. The positive predictive value (PPV) of a test is always a function of prevalence, not just sensitivity and specificity.

The Public Communication Problem

During the pandemic, news coverage of COVID tests rarely provided this context. Headlines reported sensitivity and specificity numbers as if they were fixed predictors of result reliability. Public health communications said things like "this test is 97% accurate" — which is meaningless without specifying the context.

The failure was consequential: - In low-prevalence settings, people with false positive results experienced unnecessary isolation, economic hardship, and severe anxiety - In high-prevalence settings, people with false negative antigen tests (the common case during Omicron) falsely assumed they were negative and did not isolate - Across the pandemic, the asymmetric emphasis on sensitivity (the metric favored by test manufacturers) over specificity created systematic overestimation of positive predictive value

Section 4: The Two-Test Protocol — Sequential Bayesian Updating

A logical response to the false positive problem in low-prevalence settings is sequential testing: using a positive first test result as the new prior for a confirmatory second test.

Scenario: Asymptomatic screening at 0.3% prevalence using PCR test (sensitivity 97%, specificity 99.5%). First PCR test is positive — posterior is 36.9%.

Second PCR test (different lab, independent run):

New prior = 0.369 (from first test) P(COVID) = 0.369; P(no COVID) = 0.631 P(positive | COVID) = 0.97; P(positive | no COVID) = 0.005

P(positive) = (0.369 × 0.97) + (0.631 × 0.005) = 0.3579 + 0.00316 = 0.361

P(COVID | two positives) = 0.3579 / 0.361 ≈ 99.1%

Two independent positive PCR tests in an asymptomatic person, even at very low prevalence, give a 99.1% probability of true infection. This is the clinical logic behind confirmatory testing protocols.

Key assumption: The two tests must be genuinely independent — they cannot share a sample or be from the same laboratory run. If the second test uses the same sample and the false positive arose from a contamination in the sample collection, the second test will also be positive, violating independence.

Section 5: Communication Lessons from the Pandemic

The COVID-19 testing case study generates several lessons for probabilistic communication and media literacy:

Lesson 1: Report context with test results. A positive test result is not a self-interpreting fact. It must be placed in the context of prevalence, symptom status, and testing purpose. "You tested positive" should be accompanied by an estimate of what that means in your specific context.

Lesson 2: Relative metrics need absolute context. "The test is 99% accurate" has no meaning without specifying accurate in which direction (sensitivity? specificity?) and for which population (symptomatic? asymptomatic? which prevalence setting?).

Lesson 3: Natural frequencies aid comprehension. "Of 10,000 asymptomatic people in a low-prevalence community who test positive, approximately 6,300 will be false positives" is dramatically more comprehensible than "the positive predictive value at 0.3% prevalence is 36.9%."

Lesson 4: Uncertainty communication requires the full range. Health authorities who communicated only that positive tests were reliable (appropriate for symptomatic patients in high-prevalence settings) contributed to distress and overcrowding of confirmatory testing services when mass asymptomatic screening began.

Lesson 5: Sequential Bayesian reasoning is teachable. The two-test protocol is an instance of Bayesian updating that can be explained in accessible terms: "A second positive test, from a different kit and tested at a different time, dramatically increases our confidence that the result is accurate."

Questions for Discussion

1. During the pandemic, should public health authorities have communicated the positive predictive value of tests at different prevalence levels, even though this information is more complex than "the test is X% accurate"? What are the risks of oversimplification vs. full probabilistic communication?

2. Many people who received false positive COVID results experienced significant distress and economic harm (lost wages, cancelled travel, broken quarantine by close contacts). Conversely, false negatives may have led to unintended transmission. How should these asymmetric harms be weighed in testing policy design?

3. The two-test sequential protocol dramatically improves positive predictive value at low prevalence. Why wasn't confirmatory testing more systematically recommended during mass asymptomatic screening programs? What organizational and logistical barriers exist?

4. Some jurisdictions during the pandemic announced high daily case counts that turned out to include a significant proportion of false positives from PCR testing at low prevalence. How should this possibility have been communicated to the public, without undermining appropriate public health responses?

5. At-home antigen tests became widely available toward the end of the pandemic. These tests are operated by untrained users, have lower sensitivity than laboratory PCR, and provide results with no clinical context. Design a plain-language guide for interpreting at-home test results that incorporates the Bayesian principles from this case study.