Chapter 4 Quiz: Thinking Like a Political Analyst

Answer all questions. Multiple-choice questions have exactly one correct answer unless noted.

Part A: Multiple Choice

1. A campaign analyst summarizes the age distribution of all registered voters in the state voter file. This is an example of:

a) Inferential analysis b) Descriptive analysis c) Causal analysis d) Experimental analysis

2. A pollster surveys 900 likely voters and reports that Garza leads Whitfield by 5 points. The claim that "Garza leads among all likely voters in the state" is an example of:

a) Descriptive analysis, because it describes the poll result b) Inferential analysis, because it uses a sample to make a claim about a population c) Causal analysis, because it implies Garza's strategy is causing the lead d) Predictive analysis, because it predicts the election outcome

3. The "fundamental problem of causal inference" refers to:

a) The difficulty of collecting reliable data in political contexts b) The fact that for any unit, you can only observe one of the potential outcomes, never the counterfactual c) The inability of regression analysis to control for all confounds d) The systematic undersampling of minority communities in political surveys

4. Which of the following best illustrates the ecological fallacy?

a) Concluding from a single case that a general pattern holds b) Using aggregate county-level data to draw conclusions about individual voter behavior c) Assuming that correlation implies causation d) Treating a model's prediction as a certainty

5. Nadia finds that precincts with more campaign canvassing show higher support for Garza. Before concluding that canvassing increases support, the most important alternative explanation to consider is:

a) That the canvassers were unusually persuasive communicators b) That canvassers were directed to precincts already identified as higher-support c) That voters who answer the door are different from those who do not d) That the polling in those precincts was conducted by a pro-Garza firm

6. A "base rate" in political analysis refers to:

a) The lowest possible polling number a candidate can receive b) The background probability of an event in the relevant reference class c) The minimum sample size required for a reliable poll d) The percentage of voters who always vote for the same party

7. In the context of Bayesian updating, your "prior" should ideally be based on:

a) The most recent poll showing your candidate's position b) Structural features of the race, historical patterns, and fundamentals c) The campaign consultant's gut assessment d) The national polling average for the current cycle

8. Which question from the analyst's decision tree is most directly designed to prevent confirmation bias?

a) "What decision will this analysis inform?" b) "What is the right unit of analysis?" c) "What would change your mind?" d) "What precision does the decision require?"

9. A campaign's predictive voter support model correctly classifies 82% of voters in the training dataset. A member of the campaign leadership team concludes that the model is "82% accurate for this election." This conclusion:

a) Is correct if the training dataset was a representative sample of all voters b) Ignores the distinction between in-sample performance and out-of-sample prediction c) Is slightly overconfident but roughly accurate d) Should be reported as 82% ± 4% to account for the margin of error

10. Jake Rourke vividly remembers a single mailer that swung a rural county by 8 points in 2014, and heavily weights this when estimating the effectiveness of mailers in 2026. This illustrates:

a) The anchoring bias b) The availability heuristic c) The ecological fallacy d) Selection bias

11. The primary difference between observational and experimental analysis is:

a) Observational analysis uses quantitative data; experimental analysis uses qualitative data b) In experimental analysis, units are randomly assigned to conditions; in observational analysis, they are not c) Experimental analysis is appropriate for large samples; observational analysis for small ones d) Observational analysis is conducted in the field; experimental analysis in the laboratory

12. A calibrated forecaster is one who:

a) Is right at least 75% of the time b) Never expresses more than 60% confidence in any prediction c) Has stated confidence levels that match their actual accuracy over many predictions d) Uses formal Bayesian updating for every forecast

Part B: Short Answer

13. Define the "ecological fallacy" in your own words and provide a political example not mentioned in the chapter.

14. Explain the difference between predictive and explanatory analysis. Give one concrete example of a political question where prediction is the appropriate goal and one where explanation is more appropriate.

15. What is a "pre-mortem" and why is it a useful analytical tool? In two to three sentences, explain the cognitive function it serves.

Part C: Application

16. Read the following scenario and answer the questions below.

A political analyst working for a Senate campaign notices that counties where the unemployment rate increased over the past two years show higher support for the challenger candidate. The analyst presents this finding to campaign leadership as evidence that "the economy is driving voters to the challenger."

a) Is this claim justified by the finding? Explain the analytical problem with the conclusion.

b) List two alternative explanations for the pattern.

c) What type of additional analysis or data would help the analyst make a stronger claim?

17. Suppose you observe that states where presidential candidates held more campaign rallies in the final month also showed higher voter turnout. Apply the analyst's decision tree to evaluate whether this relationship represents a causal effect of rallies on turnout.

Answer Key

Part A: 1-b | 2-b | 3-b | 4-b | 5-b | 6-b | 7-b | 8-c | 9-b | 10-b | 11-b | 12-c

Part B — Scoring Rubric:

Question 13: Full credit requires defining the fallacy (drawing individual-level conclusions from aggregate data) and providing a valid political example. Half credit if definition is correct but example illustrates a different fallacy.

Question 14: Full credit requires clear articulation of the trade-off (prediction optimizes accuracy; explanation optimizes interpretability) and two appropriate examples — one where the outcome variable is the goal (prediction) and one where understanding mechanisms matters (explanation).

Question 15: Full credit requires both defining the tool (imagining the recommendation failed, then working backward to identify causes) and explaining the cognitive function (surfaces confirmation bias and motivated reasoning before decisions are made).

Part C — Scoring Rubric:

Question 16: (a) Full credit for identifying confounding — counties with rising unemployment likely differ from others in multiple ways correlated with challenger support. (b) Full credit for any two valid confounds (incumbency effects, demographic composition, prior voting patterns, media coverage, etc.). (c) Full credit for proposing a valid approach: matched comparison of similar counties, regression controlling for demographic variables, or natural experiment design.

Question 17: Full credit for working through at least four of the seven decision-tree questions with appropriate specificity. Look for: the selection problem (campaigns choose where to hold rallies), the reverse causality possibility (rallies go where the race is competitive, which predicts turnout independently), and appropriate methods for addressing confounding.