Key Takeaways: Conditional Probability and Bayes' Theorem

One-Sentence Summary

Bayes' theorem is the mathematical engine for updating beliefs with evidence — revealing why even "accurate" tests produce false alarms, why P(A|B) is not the same as P(B|A), and why every AI system from spam filters to self-driving cars runs on conditional probability.

Core Concepts at a Glance

Bayes' Theorem — Three Ways

1. The Formula

$$\boxed{P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B \mid A) \cdot P(A) + P(B \mid \text{not } A) \cdot P(\text{not } A)}}$$

2. The Plain-English Version

$$\text{Posterior} = \frac{\text{Likelihood} \times \text{Prior}}{\text{Total probability of the evidence}}$$

3. The Natural Frequency Approach

1. Start with a large round number (e.g., 10,000 people)
2. Split by the base rate (how many have vs. don't have the condition)
3. Apply the test to each group (true positives, false positives, etc.)
4. Count: "Of everyone who tested positive, how many actually have the condition?"

Conditional Probability Quick Reference

From a Contingency Table

Given this table:

Key insight: For conditional probability, the "given" condition becomes the denominator. You restrict your universe to the column or row of the given condition.

The General Multiplication Rule

$$P(A \text{ and } B) = P(A) \times P(B \mid A) = P(B) \times P(A \mid B)$$

This is the full version of the multiplication rule from Chapter 8. The independent version ($P(A) \times P(B)$) is a special case where $P(B \mid A) = P(B)$.

Medical Testing Vocabulary

Critical: Sensitivity and specificity are properties of the test. PPV and NPV depend on the test AND the base rate.

Decision Guide: When to Use What

What probability question are you asking?
│
├── "Given B happened, what's P(A)?"
│   └── CONDITIONAL PROBABILITY: P(A|B) = P(A and B) / P(B)
│       └── From a table: cell ÷ column or row total
│
├── "I know P(B|A), but I need P(A|B)"
│   └── BAYES' THEOREM: P(A|B) = P(B|A)·P(A) / P(B)
│       └── TIP: Try natural frequencies first
│
├── "Are A and B independent?"
│   └── CHECK: Does P(A|B) = P(A)?
│       ├── YES → Independent
│       └── NO  → Dependent (use general multiplication rule)
│
└── "What's the overall P(B) when there are
     multiple pathways to B?"
    └── LAW OF TOTAL PROBABILITY:
        P(B) = P(B|A)·P(A) + P(B|not A)·P(not A)
        └── TIP: This is the denominator of Bayes' theorem

How PPV Changes with Prevalence

(For a test with 99% sensitivity and 99% specificity)

Takeaway: The rarer the condition, the less a positive test means — even with an excellent test.

Common Misconceptions

The Prosecutor's Fallacy — A Checklist

When someone presents probabilistic evidence, ask:

• [ ] "The probability of WHAT given WHAT?" — Identify which direction the conditional goes.
• [ ] "What's the base rate?" — How common is this event before considering the evidence?
• [ ] "What's the alternative?" — You must compare competing explanations, not evaluate one in isolation.
• [ ] "Was independence assumed?" — If probabilities were multiplied, were the events truly independent?

Python Quick Reference

import pandas as pd
import numpy as np

# --- Conditional probability from a DataFrame ---
# Method 1: pd.crosstab with normalize='index'
cond_probs = pd.crosstab(df['var1'], df['var2'], normalize='index')

# Method 2: Filter and calculate
subset = df[df['condition'] == 'value']
p_conditional = (subset['target'] == 'outcome').mean()

# Method 3: groupby
df.groupby('var1')['var2'].value_counts(normalize=True)

# --- Bayes' theorem function ---
def bayes_theorem(prior, likelihood, false_alarm):
    """P(A|B) from P(A), P(B|A), and P(B|not A)."""
    p_evidence = likelihood * prior + false_alarm * (1 - prior)
    return (likelihood * prior) / p_evidence

# Example: disease screening
posterior = bayes_theorem(
    prior=0.001,       # base rate
    likelihood=0.99,   # sensitivity
    false_alarm=0.02   # 1 - specificity
)

Key Terms

The One Thing to Remember

If you forget everything else from this chapter, remember this:

Probability is not fixed — it changes with evidence. Bayes' theorem is the machine that does the updating: it takes what you believed before (the prior), combines it with what you've just learned (the evidence), and produces what you should believe now (the posterior). But the update depends critically on the base rate — the prior probability you started with. Ignoring the base rate is one of the most common and most dangerous reasoning errors humans make. Every time you encounter a probability claim, ask three questions: "The probability of what?" "Given what?" and "Compared to what?" Those three questions will protect you from the prosecutor's fallacy, the base rate fallacy, and a hundred other traps.