Case Study 21.1: Nadia's Model vs. Jake's Model — Two Campaigns, One Race

Background

Every competitive Senate race has two forecasting operations running simultaneously — one for each campaign. They look at the same polls, the same economic data, and the same voter files, but they often arrive at different conclusions. This case study examines what happens when two sophisticated analytics operations analyze the same race with different assumptions.

Nadia Osei, Garza's analytics director, has built the model described in Chapter 21: a weighted poll aggregator blended with a fundamentals prior, using a systematic error SD of 2.0 points and a poll weight of 0.75. Her final output: Garza wins with 61 percent probability.

Jake Rourke, Whitfield's campaign manager, has his own internal analytics operation. Jake has spent twenty-two years in Republican politics, has lived through 2016 and 2020, and has developed a specific skepticism about polls in competitive states. His internal forecaster has built a model with the following differences:

• Poll weight: 0.50 (Jake trusts fundamentals more)
• Systematic error SD: 3.5 (Jake assumes larger potential polling bias, based on 2016/2020)
• State lean adjustment: +2.5 R (Jake believes Ordana's recent Republican trend has been underestimated in historical Cook PVI)

Comparing the Two Models

The table below shows how each model's parameters differ and what they produce:

Both models say Garza is ahead. But Nadia is presenting a comfortable favorite scenario; Jake is presenting an essentially tied race.

What Drives the Divergence?

Difference 1: State Lean Assumption

Jake has updated his state lean to R+2.5, arguing that Ordana's 2020 presidential result (which went R by 3.8 points) and 2022 midterm results (which showed continued Republican gains in rural counties) justify a more aggressive Republican lean than the historical Cook PVI of R+1.8.

Nadia uses the standard Cook PVI, arguing that a single cycle's performance should not revise a multi-cycle average without more evidence, and that 2020 was an unusual election in Ordana due to a specific local economic shock.

Who is right? This is genuinely uncertain, and the difference — 0.7 points in the state lean — contributes approximately 0.17 to the point estimate difference (given 25-50% fundamentals weight).

Difference 2: Poll Weight

Jake's higher fundamentals weight (0.50 vs. 0.25) reflects a principled skepticism about current polls. He has seen two consecutive cycles in which Republican support was underestimated in states like Ordana, and he is unwilling to place 75 percent of his point estimate on polling data he does not fully trust.

Nadia argues that placing equal weight on polls and fundamentals when seventeen polls are available, several from highly rated firms, leaves too much forecasting power with a historical-baseline model that cannot capture current-cycle dynamics.

The poll weight difference contributes approximately 0.7 points to the point estimate difference.

Difference 3: Systematic Error SD

Jake's assumption of a 3.5-point systematic error SD (vs. Nadia's 2.0) substantially widens the confidence interval. Even if both models had the same point estimate, Jake's model would show a lower win probability for Garza because his wider uncertainty distribution puts more probability mass in the tails.

At the same point estimate of +2.1, Jake's systematic error assumption would reduce Garza's win probability from approximately 61 percent to approximately 53 percent. The rest of the gap (53 percent vs. 52 percent) comes from the different point estimate.

The Decision Implications

Both campaigns are using their models to make resource allocation decisions in the final three weeks of the race.

Nadia's decision from a 61% Garza forecast: This state is a soft hold. The campaign should maintain current spending levels but could potentially redirect marginal resources to other states where Garza allies are in tighter races. At 61%, the race is competitive enough to not abandon but comfortable enough to not require emergency investment.

Jake's decision from a 52% Whitfield forecast: This is essentially a coin flip. Every dollar spent here is justified; this is a winnable race that requires maximum effort. Jake is likely to recommend pulling in national party resources, scheduling additional candidate appearances, and increasing turnout operation spending.

Notably, both campaigns' decisions are internally consistent with their models. The problem is that the models cannot both be right about the true underlying probability — and neither campaign knows whose model is more accurate.

The Resolution

Six weeks later, the actual election result: Garza wins by 1.9 points.

Both models were directionally correct (Garza wins). Nadia's point estimate of +2.1 was nearly exactly right. Jake's point estimate of +0.8 missed by about 1.1 points.

The primary sources of Jake's error: 1. The state lean revision to R+2.5 was not justified by the evidence — Ordana's 2020 swing toward Republicans was partly cycle-specific. 2. The high fundamentals weight amplified the effect of the overestimated state lean. 3. The high systematic error SD, while individually defensible, combined with the biased point estimate to produce a picture closer to a toss-up than the underlying data supported.

Nadia's model was right for a combination of correct assumptions (standard state lean) and good data management (17 polls from reliable firms, appropriately weighted).

Discussion Questions

1. Jake's decision to revise the state lean from R+1.8 to R+2.5 based on 2020 results illustrates the challenge of deciding when to override historical baselines with recent evidence. What decision rule would you use to determine when to update a historical partisan lean estimate? How many cycles of data would you need?

2. Both models were directionally correct but Nadia's was more accurate. Does this mean Nadia's model is generally better? What scenario could be constructed in which Jake's model would be more accurate and Nadia's model would give badly wrong guidance?

3. Jake's campaign made a decision to invest maximum resources in Ordana because they saw it as a near-toss-up. Nadia's campaign made a decision to treat it as a soft hold. Who made the better resource allocation decision, given what was known at the time (before the result)? Does the outcome change your assessment?

4. If you were advising a Senate campaign committee that needs to decide how to allocate $20 million across eight competitive races, and you had access to both Nadia's and Jake's models for each race (but not the actual outcome), how would you resolve the disagreement between them for each race? What meta-model or judgment call would you apply?

5. After the election, Nadia is preparing her postmortem. Jake is preparing his. What should each postmortem say about the methodology? Were either of their errors within expected variance given their models' stated uncertainty?