Chapter 28 Exercises: Probabilistic Thinking and Uncertainty

These exercises develop probabilistic reasoning skills through calculation, analysis, and reflection. Many problems have numerical answers that can be verified. Show all work.

Part A: Probability Foundations (Exercises 1–6)

Exercise 1: Basic Probability Review A jar contains 30 marbles: 12 red, 10 blue, and 8 green. Without looking, you draw one marble. (a) What is the probability of drawing a red marble? (b) What is the probability of drawing a marble that is not green? (c) What is the probability of drawing either a red or blue marble? (d) If you draw two marbles in a row without replacement, what is the probability that both are red? (e) If you draw two marbles in a row with replacement, what is the probability that both are red?

Exercise 2: Conditional Probability In a school of 500 students, 200 play sports, 150 are in art club, and 80 do both. (a) What is P(sports)? (b) What is P(art club)? (c) What is P(sports AND art club)? (d) What is P(sports OR art club)? (e) What is P(sports | art club)? In words: if you know a student is in art club, what's the probability they also play sports? (f) Are sports and art club participation independent? Show your work.

Exercise 3: Multiplication and Addition Rules A new disease test has these properties: - Sensitivity (true positive rate): 95% - Specificity (true negative rate): 90% - Disease prevalence: 5% in the tested population

For a randomly selected person from this population: (a) What is P(positive test result)? Use the law of total probability. (b) What is P(positive test AND has disease)? (c) What is P(positive test AND no disease)? (d) Verify: P(positive test) = answer (b) + answer (c).

Exercise 4: Independence Assessment For each pair of events below, determine whether they are plausibly independent or correlated (positively or negatively), and explain your reasoning: (a) "Person exercises regularly" and "Person has low blood pressure" (b) "Coin flip 1 is heads" and "Coin flip 2 is heads" (c) "A person reads this newspaper" and "A person votes for this party" (for a strongly partisan newspaper) (d) "A star is very massive" and "A star lives a very long time" (e) "A website was registered last week" and "The website publishes misinformation"

Exercise 5: Complement and Combination (a) If P(a news story is inaccurate) = 0.08, what is P(a news story is accurate)? (b) A fact-checker reviews 5 independent claims. If each has an 85% probability of being accurately stated, what is the probability that ALL FIVE are accurately stated? (c) What is the probability that AT LEAST ONE of the five claims is inaccurate? (Hint: use the complement.) (d) If the probability of each claim being accurate drops to 70%, recalculate (b) and (c). What do you observe?

Exercise 6: The Birthday Problem (Introduction to Surprising Probabilities) The birthday problem: in a room of n people, what is the probability that at least two share a birthday?

The complement is easier: P(all different birthdays) = (365/365) × (364/365) × (363/365) × ... × ((365-n+1)/365)

Calculate P(at least one shared birthday) for n = 10, 23, 30, 50, and 70 people.

(a) At what group size does the probability first exceed 50%? (b) At what group size does it exceed 99%? (c) What does this result tell us about our intuitions for coincidence and "surprising" patterns in large datasets?

Part B: Bayes' Theorem (Exercises 7–14)

Exercise 7: Basic Bayes Calculation A disease affects 2% of the population. A test for this disease has: - Sensitivity = 90% (P(positive | disease) = 0.90) - Specificity = 95% (P(negative | no disease) = 0.95)

Using Bayes' Theorem, calculate P(disease | positive test).

Show your full work: (a) State each component: P(disease), P(no disease), P(positive|disease), P(positive|no disease) (b) Calculate P(positive test) using the law of total probability (c) Apply Bayes' formula (d) Express the answer as a percentage (e) Interpret: out of 1,000 people who test positive, how many actually have the disease?

Exercise 8: The Base Rate Effect Using the same test as Exercise 7 (sensitivity = 90%, specificity = 95%), calculate P(disease | positive test) for three different prevalence rates: (a) Prevalence = 0.1% (a rare disease) (b) Prevalence = 2% (as in Exercise 7) (c) Prevalence = 20% (a common condition) (d) Prevalence = 50% (equal probability either way)

Create a table showing prevalence vs. P(disease | positive test). What pattern do you observe? What does this tell us about the context-dependence of test result interpretation?

Exercise 9: Bayesian Updating with Multiple Tests A patient tests positive on the initial screening test (sensitivity 90%, specificity 95%, disease prevalence 2%). Their doctor orders a confirmatory test (different mechanism: sensitivity 92%, specificity 98%).

(a) Using your result from Exercise 7 as the new prior, calculate P(disease | positive confirmatory test). (b) Now suppose the confirmatory test comes back negative. Calculate P(disease | negative confirmatory test), using the same posterior from Exercise 7 as prior. (c) What does this exercise demonstrate about the value of sequential testing?

Exercise 10: Likelihood Ratios A simpler formulation of Bayes uses likelihood ratios (LR) and odds:

Prior odds = P(hypothesis) / P(not hypothesis) Posterior odds = Prior odds × Likelihood Ratio LR = P(evidence | hypothesis) / P(evidence | not hypothesis)

(a) Convert a prior probability of 0.05 to prior odds. (b) If the LR for a positive test result is 15, what are the posterior odds? (c) Convert the posterior odds back to probability. (d) Verify this against the standard Bayes formula (use P(positive|disease) = 0.90, P(positive|no disease) = 0.06 to get LR = 15; use P(disease) = 0.05). (e) Calculate the posterior probability after a second independent piece of evidence with LR = 8.

Exercise 11: The Prosecutor's Fallacy In a homicide case, a detective argues: "The probability that an innocent person would be in this location at this time of day is less than 1 in 1,000. Therefore, the probability that the suspect is innocent is less than 1 in 1,000."

(a) Identify the formal error in this reasoning (which conditional probability is being confused with which?). (b) There are 400,000 people in the city. How many would be expected to pass through this location at this time of day if the 1/1,000 rate is correct? (c) If only one of those people is the actual killer, what is the actual probability the suspect is guilty, given only this evidence? (d) Write a one-paragraph explanation of the prosecutor's fallacy for a general audience (imagine you are writing for a juror education pamphlet).

Exercise 12: The Linda Problem and Conjunction Fallacy The following problem is adapted from Kahneman and Tversky (1983).

Michael is 35 years old. He studied economics and history at a prestigious university. As a student, he was very involved in debate club and invested in stocks as a hobby. He now works in finance.

Rank the following from most to least probable: 1. Michael is a Republican. 2. Michael is a Republican and attends church regularly. 3. Michael donates to environmental charities. 4. Michael is a Republican who attends church regularly and has three children. 5. Michael works in investment banking.

(a) After ranking them, identify any pairs where you may have committed the conjunction fallacy. (b) Prove mathematically why any conjunction must be less probable than either of its parts. (c) Why does the profile description make the conjunction fallacy appealing? What cognitive process is at work?

Exercise 13: Medical Test Design Trade-offs You are designing a disease screening program. The disease has a prevalence of 0.5% in the general population. You have three test designs to choose from:

Test	Sensitivity	Specificity	Cost
A	95%	90%	$10
B	85%	99%	$25
C	99.5%	80%	$5

For each test, calculate: (a) P(disease | positive test) for a general population screening (b) The expected number of true and false positives per 100,000 people screened (c) The cost of identifying each true positive (total cost / number of true positives)

(d) Which test would you recommend for a mass screening program, and why? (e) Which would you recommend if the disease is immediately fatal if untreated and treatment has no side effects? (f) Which would you recommend if the treatment is dangerous and should only be given to confirmed cases?

Exercise 14: Bayes in a News Context A partisan news outlet publishes a story alleging corruption by a political official. You are trying to assess the probability that the corruption actually occurred.

Prior: Based on general base rates and your background knowledge, you estimate a 15% prior probability that this type of allegation is substantiated.

Evidence piece 1: The story is published by a news outlet with a known history of publishing false allegations (only 20% of their similar allegations have historically been confirmed). LR for this source = ?

Evidence piece 2: Two independent mainstream news organizations have also reported corroborating details. LR for this corroboration = ? (assume mainstream corroboration has a LR of approximately 8)

Evidence piece 3: The official's office has denied the allegation without providing specific counter-evidence. LR for this type of denial = ? (assume non-specific denials have a LR of approximately 0.7)

(a) Calculate the prior odds from 15% prior probability. (b) Calculate LR for Evidence 1 (the partisan source): if 20% of such allegations are confirmed, and such a source might publish regardless of truth, what is a reasonable LR? (c) Apply all three pieces of evidence sequentially to compute a posterior probability. (d) Discuss: this exercise requires you to estimate likelihood ratios without precise data. How does this limitation affect the usefulness of the Bayesian framework in practical reasoning?

Part C: Superforecasting and Calibration (Exercises 15–20)

Exercise 15: Calibration Measurement You will make 20 binary predictions (yes/no questions) with stated confidence levels, then track outcomes.

For each of the following questions, state your answer (yes/no) and your confidence level (50%, 60%, 70%, 80%, 90%, or 99%):

The next US Presidential election will be won by the Democratic candidate.
Global average temperature in 2030 will be higher than in 2020.
At least one country will have legalized recreational cannabis in Asia by 2028.
Social media usage will be higher in 2030 than today for teenagers.
A human will have landed on Mars by 2035. (Add 15 more current-events predictions as assigned by instructor)

After outcomes are known, calculate: (a) For all questions where you said 70%: what fraction were correct? (b) Plot a calibration curve for your predictions across all confidence levels. (c) Compute your Brier scores for each prediction. (d) What is your average Brier score? Compare to class average.

Exercise 16: The Premortem Exercise Choose a belief you currently hold with high confidence (at least 80%): a factual belief, a prediction about current events, or a belief about how something works.

(a) Describe the belief and rate your confidence: % certain. (b) Conduct a premortem: Write a 200-word scenario in which your belief turns out to be WRONG. What would the explanation be? What evidence would you have missed or misinterpreted? (c) After conducting the premortem, re-rate your confidence: % certain. (d) Did your confidence change? By how much? Why or why not?

Exercise 17: Reference Class Forecasting Apply reference class forecasting to three predictions:

(a) "How many of the restaurants that opened in a major city this year will still be operating in five years?" Base your prediction on the reference class of new restaurant openings generally, then adjust for any specific factors.

(b) "How likely is it that a first-time fiction author who submits a manuscript to a traditional publisher will get published?" Find or estimate the reference class rate, then adjust.

(c) "What is the probability that a new pharmaceutical drug in Phase 2 clinical trials will eventually receive regulatory approval?" Research the reference class rate (approximate) and state your estimate.

For each: identify the reference class, state the reference class base rate, and explain what adjustments (up or down) you would make for the specific situation.

Exercise 18: Comparing Confidence Intervals A news article reports: "Economists surveyed expect GDP growth of between 1.5% and 4.0% next year."

(a) What confidence level is implied by this range (assume 90% confidence interval)? (b) The actual result was 0.8% growth. Was this result "surprising" given the range? What does this tell us about stated confidence intervals? (c) Construct your own 90% confidence interval for each of the following: - The high temperature in your city next January - The number of films nominated for Best Picture at the next Oscars - The world record marathon time in 10 years (in hours and minutes) (d) Share your confidence intervals with classmates. What fraction of the class's 90% confidence intervals would you expect to be wrong? Compare this to the actual number.

Exercise 19: Brier Score Workshop The Brier score measures forecast accuracy. For a binary event: BS = (probability - outcome)²

(a) Calculate Brier scores for the following forecasts: - Forecast: 90% confident. Outcome: event occurred. BS = ? - Forecast: 90% confident. Outcome: event did NOT occur. BS = ? - Forecast: 50% confident. Outcome: event occurred. BS = ? - Forecast: 10% confident. Outcome: event occurred. BS = ?

(b) A weather forecaster says "30% chance of rain" every day for a week. It rains on 2 of those days. Calculate the forecaster's Brier scores for each day and the weekly average.

(c) Compare two strategies: - Strategy A: Always say 50% (never commits) - Strategy B: Says actual probabilities based on meteorological data

Show that Strategy B with good calibration will outperform Strategy A on average Brier score, even if Strategy B makes some errors.

Exercise 20: Superforecasting Traits Self-Assessment Using the characteristics of superforecasters identified by Tetlock (listed in Section 28.6), conduct a self-assessment:

For each trait, rate yourself on a scale of 1-5: (1 = this is not characteristic of me, 5 = this strongly describes me)

Probabilistic thinking (do I naturally think in probabilities vs. binary categories?)
Active open-mindedness (do I actively seek disconfirming evidence?)
Calibration focus (do I care about accuracy of my confidence levels?)
Granularity (do I make fine-grained distinctions like 63% vs. 65%?)
Willingness to update (do I revise my views readily when evidence changes?)
Self-awareness of biases (do I actively work to counteract my biases?)
Dragonfly eye view (do I integrate multiple perspectives before deciding?)

(a) Sum your scores and identify your two lowest-rated traits. (b) For each low-rated trait, design a specific practice you could adopt to improve it. (c) Tetlock's research suggests forecasting skill can be improved with practice. Design a 30-day personal forecasting practice.

Part D: Communicating Uncertainty and Decision-Making (Exercises 21–28)

Exercise 21: Words to Numbers Translation Survey 10 people by asking them: "What percentage probability would you associate with each of the following phrases?" (Phrases: "certain," "highly likely," "likely," "possible," "unlikely," "rare," "impossible.")

(a) Compile the responses into a table. (b) Calculate the mean and range for each phrase. (c) Compare your results to the IPCC uncertainty language definitions from Section 28.8. (d) What are the implications of this variability for science communication? Write a 150-word response.

Exercise 22: IPCC Language Analysis Find three quotes from IPCC Summary for Policymakers reports that use formal uncertainty language. For each: (a) Identify the uncertainty term used (e.g., "likely," "very likely") (b) What does this term mean in the IPCC system (what numerical probability range)? (c) How would a typical non-specialist reader likely interpret this term? (d) Does the gap between official meaning and typical interpretation have consequences for policy understanding?

Exercise 23: Expected Value Calculations Calculate the expected value for each scenario:

(a) A lottery ticket costs $5. There is a 1 in 1,000,000 chance of winning $1,000,000 and a 1 in 1,000 chance of winning $100. All other tickets win nothing. Is the ticket "worth" buying in expected value terms?

(b) You can purchase cybersecurity insurance for $200/year. There is a 5% annual probability of a cyberattack that would cost $5,000 to remediate. What is the EV of buying insurance? Of not buying it?

(c) A vaccine has a 0.001% chance of causing a severe adverse reaction (costing 1,000 QALYs of health loss) and prevents infection with 80% probability, where infection has a 0.5% chance of causing severe complications (costing 10 QALYs) and a 0.01% chance of death (costing 40 remaining QALYs). Calculate the expected health cost of vaccination vs. non-vaccination.

Exercise 24: The Precautionary Principle in Practice For each of the following scenarios, analyze the risk-benefit structure and determine whether application of the precautionary principle is warranted:

(a) A proposed new food additive has undergone 5 years of safety testing with no adverse effects found, but some animal studies show effects at very high doses.

(b) A new gene editing technology could cure a fatal inherited disease in infants, but its long-term effects on non-targeted genes are uncertain.

(c) A proposed regulation would require coal plants to reduce particulate emissions, imposing economic costs, but the health benefits of reduced air pollution are well-documented.

(d) A claim circulating on social media suggests that a common household chemical combination is dangerous. The chemistry is plausible but no human cases of harm have been documented.

For each: identify the uncertain harm, the cost of precaution, and the cost of inaction, then make a recommendation.

Exercise 25: Manufactured Uncertainty Recognition Read the following excerpts and identify the specific manufactured uncertainty techniques being used:

(a) "Scientists still don't know exactly how much CO2 is in the atmosphere or exactly when warming effects will be felt, so we certainly shouldn't rush into expensive regulations."

(b) "There are many reputable scientists who question the vaccine schedule. Parents deserve to make informed choices, and the debate continues among experts."

(c) "While this study appears to show an effect, it was conducted on rats, and many animal studies don't translate to humans. Much more research is needed before any conclusions can be drawn."

(d) "The WHO itself admits that the precise transmission route isn't fully understood. Given this uncertainty, how can authorities justify these restrictions?"

For each: (i) Identify the technique (false symmetry, uncertainty laundering, demanding certainty, exploiting evidential asymmetry), (ii) Describe what genuine uncertainty exists vs. what is being exaggerated, (iii) Write a calibrated counter-response.

Exercise 26: Base Rate Research Project Base rate neglect often occurs because people don't know the base rate. For each of the following scenarios, research the actual base rate and explain how knowing it changes interpretation:

(a) A story reports "25 cases of autism reported among children who received the MMR vaccine in City X." What is the baseline expectation for autism diagnosis in a similar-sized population of children this age?

(b) A headline reads "Airline safety emergency: 50 near-misses this year." What is the base rate for near-miss incidents per million flights?

(c) "Drug X linked to 200 cases of liver damage." What is the base rate for liver damage of this type in people not taking drug X?

(d) "Breakthrough cancer rate in vaccinated population exceeds baseline." What would the expected breakthrough rate be even for a 95% effective vaccine in a population with this infection rate?

Exercise 27: Calibration Training Practice Round Your instructor will provide 30 questions with verifiable answers (trivia, statistics, historical facts). For each question: (a) Write your best guess at the answer (b) Provide a 90% confidence interval (a range you are 90% sure contains the true answer)

After all answers are revealed: (c) Count how many of your 90% CI were correct (should be ~27 if perfectly calibrated) (d) If fewer than 23 were correct, you are overconfident. If more than 29 were correct, you may be underconfident. (e) Repeat the exercise two weeks later with a new question set. Has your calibration improved?

Exercise 28: Integrated Analysis — Probabilistic Reasoning Applied to a Misinformation Case Select a health-related misinformation claim currently circulating online (approved by instructor or selected from a current fact-check database).

Write a 600-800 word analysis that applies probabilistic reasoning to the claim:

(a) Prior probability assessment: Based on background knowledge, what is a reasonable prior probability that this claim is true? What comparable claims and base rates inform this prior?

(b) Evidence evaluation: What evidence supports the claim? What is the quality (study design, sample size, replication status) of this evidence? Assign approximate likelihood ratios.

(d) Bayesian update: Describe (qualitatively or quantitatively) how the evidence updates the prior.

(e) Uncertainty communication: If you were to communicate your uncertainty about this claim to a general audience, what language would you use? How would you avoid both overconfidence and manufactured-uncertainty-style hedging?

(f) Decision implications: If a person used your assessment to make a health decision (to adopt or avoid the practice, to seek or decline a treatment), what decision would follow from your posterior probability? Is this the same decision that would follow from naive binary acceptance or rejection of the claim?

Reflection Questions

After completing the exercises in this chapter:

Which probability concept did you find most counterintuitive? Did working through the mathematics change your intuition, or do you find the intuition persists even after intellectual understanding?
Tetlock distinguishes between "foxes" (those who draw on many frameworks) and "hedgehogs" (those who explain everything through one big idea). Which describes your own thinking style? What would it take to develop more of the other?
Identify a domain in your life — health decisions, financial decisions, relationship choices — where you regularly make decisions under uncertainty. How would explicitly probabilistic reasoning change how you approach these decisions?
Is there a risk of probabilistic thinking becoming a form of epistemic cowardice — always hedging, never committing, using uncertainty as an excuse to avoid taking a stand? Where is the line between appropriate uncertainty and failure to act on available evidence?