Key Takeaways: Probability — The Foundation of Inference

One-Sentence Summary

Probability provides the mathematical language for quantifying uncertainty, using a small set of rules — complement, addition, and multiplication — that underpin every statistical inference, AI prediction, and data-driven decision that follows.

Core Concepts at a Glance

The Three Approaches to Probability

Probability Rules Quick Reference

Rule 1: Boundaries

$$0 \leq P(A) \leq 1$$

• P(A) = 0 means impossible
• P(A) = 1 means certain

Rule 2: All Outcomes Sum to 1

$$\sum P(\text{all outcomes}) = 1$$

Rule 3: Complement

$$\boxed{P(\text{not } A) = 1 - P(A)}$$

When to use: When calculating "at least one" or "not A" is easier than calculating P(A) directly.

Rule 4: Addition Rule

Mutually exclusive events (no overlap): $$P(A \text{ or } B) = P(A) + P(B)$$

General (any events): $$\boxed{P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)}$$

When to use: Any time you need P(A or B). Always subtract the overlap unless you know the events are mutually exclusive.

Rule 5: Multiplication Rule (Independent Events)

$$\boxed{P(A \text{ and } B) = P(A) \times P(B)}$$

When to use: When events are independent (one doesn't affect the other). Always verify independence before using this rule.

Decision Guide: Which Rule Do I Need?

What probability are you calculating?
│
├── P(not A)?
│   └── COMPLEMENT RULE: P(not A) = 1 − P(A)
│
├── P(A OR B)?
│   ├── Are A and B mutually exclusive?
│   │   ├── YES → P(A or B) = P(A) + P(B)
│   │   └── NO  → P(A or B) = P(A) + P(B) − P(A and B)
│   └── TIP: If you have a contingency table, count directly
│       and divide by the grand total to verify
│
├── P(A AND B)?
│   ├── Are A and B independent?
│   │   ├── YES → P(A and B) = P(A) × P(B)
│   │   └── NO  → Need conditional probability (Ch. 9)
│   └── TIP: In a contingency table, this is cell ÷ grand total
│
└── P(at least one)?
    └── COMPLEMENT TRICK:
        P(at least one) = 1 − P(none)
        Often combine with multiplication rule for P(none)

Key Distinctions

Mutually Exclusive vs. Independent

Critical point: Mutually exclusive events (with non-zero probability) are ALWAYS dependent. Knowing A happened tells you B didn't — that's information, which means dependence.

Contingency Table Probability Cheat Sheet

Given a contingency table with two categorical variables:

Common Misconceptions

The Law of Large Numbers — What It Says and Doesn't Say

Python Quick Reference

import numpy as np
import pandas as pd

# --- Simulation ---
np.random.seed(42)

# Coin flip simulation (1 = heads, 0 = tails)
flips = np.random.choice([0, 1], size=10000)
prop_heads = np.mean(flips)   # Proportion of heads

# Die roll simulation
rolls = np.random.randint(1, 7, size=10000)
prop_six = np.mean(rolls == 6)  # Proportion of sixes

# --- Contingency Tables ---
# Create from a DataFrame
contingency = pd.crosstab(df['var1'], df['var2'], margins=True)

# Joint probabilities (all cells / grand total)
joint_probs = pd.crosstab(df['var1'], df['var2'],
                          margins=True, normalize='all')

# Row-wise proportions (conditional probabilities preview)
row_probs = pd.crosstab(df['var1'], df['var2'],
                        margins=True, normalize='index')

# --- Counting ---
from math import comb, factorial
comb(23, 2)     # "23 choose 2" = 253 (number of pairs)
factorial(5)    # 5! = 120

Key Terms

The One Thing to Remember

If you forget everything else from this chapter, remember this:

Probability is the language of uncertainty — and uncertainty is not a flaw. It's the raw material of every statistical inference you'll ever make. The complement, addition, and multiplication rules are your entire toolkit for basic probability. Master them, and you're ready for everything that follows: conditional probability, distributions, sampling, confidence intervals, hypothesis tests. It all starts here.