Appendix F: How to Read a Poll

A practical primer for evaluating survey research in American politics.

A poll is a measurement tool. Like any measurement tool, it has a manufacturer, a method, an error band, and a set of assumptions baked into its design. A reader who treats a poll as "what the country thinks" — full stop — will be misled. A reader who treats every poll as worthless propaganda will also be misled. The truth is in between, and learning to read polls well is a basic civic skill.

This appendix teaches you how to read a poll release the way a methodologist reads it: looking for what the poll actually says, what it leaves out, and what its sponsor wants you to think it says.

This appendix is referenced from Chapter 17 (Public Opinion and Polling), Chapter 21 (Campaign Operations), and Chapter 33 (The Policy Process). Use it whenever a number with a percent sign appears in a political news story.

1. The anatomy of a poll release

Every responsible poll release contains a methodology statement. If a release doesn't, that is itself a finding — usually a bad one. Here is what to look for:

• Sample size (n). How many people were interviewed. Bigger is better, but with diminishing returns. A national poll of 1,000 adults has a margin of error around ±3 points; a poll of 4,000 around ±1.5. State-level polls often have smaller samples (600–800), with correspondingly larger margins.
• Sample population. Who was interviewed. The three big categories:
• All adults. A general-population sample. Useful for measuring opinion on issues but not for predicting elections (because not all adults vote).
• Registered voters (RV). People on the voter rolls. Closer to electoral relevance but still includes many non-voters.
• Likely voters (LV). A subset of registered voters whom the pollster's model predicts will actually turn out. The most predictive — and the most contested. (See section 3.)
• Sampling method. How respondents were reached. The four main methods today:
• Live telephone (landline + cell). The traditional gold standard. Expensive, declining response rates.
• Interactive Voice Response (IVR / "robocall"). Recorded voice; respondents press buttons. Cheaper, can't legally call cell phones in most states without separate live-call protocols, so often supplemented with online.
• Online panel. Pre-recruited internet panels (YouGov, Ipsos KnowledgePanel, NORC AmeriSpeak, SSRS, Morning Consult). Now the dominant method for most public polls.
• Text-to-web. Recruit by SMS, complete the survey online. Growing rapidly.
• Field dates. When respondents were interviewed. Polls fielded right before a debate are stale the day after the debate. A poll fielded over a single weekend is more vulnerable to weekend-effect noise than one fielded across a full week.
• Sponsor. Who paid for the poll. Pew Research, Gallup, and ANES are non-partisan. So are most major newspaper polls (NYT/Siena, WaPo/ABC, NBC News). Some campaign-affiliated polls (the Trafalgar Group, Public Policy Polling) lean toward their political sponsors. A campaign-internal poll released by a campaign is partisan information by definition.
• Margin of error (MoE). The statistical uncertainty band. (See section 2.)
• Weighting variables. Which demographic characteristics were used to make the sample resemble the target population. (See section 4.)

If a poll release omits any of these, it is incomplete. The American Association for Public Opinion Research (AAPOR) Transparency Initiative maintains a database of pollsters who commit to disclosing methodology; if a poll's sponsor isn't on that list, treat its numbers more skeptically.

2. The margin of error and what it actually means

The margin of error is the most-cited and most-misunderstood number in polling.

What it is

A 95% confidence interval. If a poll reports "Smith 49%, Jones 47%, MoE ±3 points," the technical claim is: if you ran this exact poll many times under the same conditions, in 95% of those repetitions, the true population value for Smith would fall between 46% and 52%, and the true value for Jones between 44% and 50%. The MoE describes the uncertainty of each candidate's number separately.

Why a 3-point margin means a 6-point swing range

If Smith's "true" support could be anywhere from 46 to 52 (a 6-point range), and Jones's could be anywhere from 44 to 50 (also 6 points), then the actual race could be anywhere from "Jones up by 4" to "Smith up by 8." The MoE on a single number is ±3, but the plausible range of outcomes spans 12 points if you take both extremes.

Margin of error of the difference

The relevant statistic for a two-candidate race is the margin of error of the difference between the candidates, which is roughly 1.7 times the headline MoE. A poll with ±3 MoE has a difference MoE of about ±5. So when Smith is at 49 and Jones at 47, the lead is 2 points, well inside the 5-point difference MoE: statistically, this is a tied race. Reporters who write "Smith leads by 2" without noting the MoE are mis-stating the finding.

The phrase "statistical tie" is journalist shorthand for "the lead is smaller than the difference MoE." Use it. Look for it. Write it on the corner of every horse-race headline you read.

What MoE does not include

MoE captures only sampling error — the random variation that comes from interviewing 1,000 people instead of all 250 million voting-age Americans. It does not include:

• Coverage error (the people you couldn't reach because they don't have phones, don't answer unknown numbers, don't take internet surveys, etc.).
• Non-response bias (the people who could have been reached but refused — and who systematically differ from those who said yes).
• Measurement error (people misunderstanding the question, lying, misremembering, picking an answer because it sounds polite).
• Specification error (likely-voter screen too tight, wrong demographic targets in weighting).

These can easily exceed sampling error in size. The 2016 polls had MoEs around ±3, but the actual error in key state polling was 4–8 points. The MoE was not the source of the miss.

A useful rule of thumb: the true uncertainty of a poll is roughly double its reported MoE.

3. Sample population and the likely-voter screen

Whether a poll surveys "all adults," "registered voters," or "likely voters" can swing the headline number 4–7 points — sometimes more.

Why the difference matters

The American electorate is older, whiter, more educated, and higher-income than the American adult population. So a poll of all adults will look more Democratic on most issues than a poll of likely voters, simply because the people who don't vote are demographically tilted toward the Democratic coalition. (This is a generalization with many exceptions — see Chapter 22 on voting behavior — but it captures the systematic pattern.)

If a pollster uses a tight likely-voter screen (asking respondents whether they voted in the last election, whether they know where their polling place is, how interested they are in the campaign), they get an older, whiter, more partisan-engaged sample. If they use a loose screen, they get a younger, more diverse, less politically engaged sample.

The discretion problem

The likely-voter screen is the single biggest discretionary lever a pollster has. There is no objectively "correct" turnout assumption. Every cycle, pollsters look at recent turnout patterns and estimate who will show up. They are guessing.

In 2022, many pollsters expected Republican turnout to surge (the "red wave" narrative). They built that assumption into their likely-voter models. The wave didn't materialize — Democrats held the Senate and lost the House by less than expected — and the polls overstated Republican support by 2–4 points across most close races. Likely-voter assumptions were a meaningful piece of the miss.

In 2024, pollsters corrected for the 2022 over-estimate of Republicans. Some of them corrected too far. The 2024 polling was closer to the final result than 2022 in some places and missed in different directions in others. (The post-mortem is ongoing as of 2026.)

The lesson: when reading a poll, ask which sample population was surveyed. If the release says "likely voters," ask what the screen was. If a campaign release shows their candidate doing better with likely voters than with all adults, that is normal — but the spread tells you what the pollster's turnout assumption is doing.

4. Weighting

When a pollster interviews 1,000 people, the raw sample is virtually never demographically representative of the target population. There will be too many college graduates, too many older respondents (because they answer phones), too few Hispanic respondents, too few rural respondents, etc. Pollsters fix this by weighting: counting some respondents more than others to make the demographic profile of the (weighted) sample match what the analyst believes the target population looks like.

The standard weighting variables

• Age
• Race / ethnicity (often: white, Black, Hispanic, Asian, other)
• Gender
• Education (college degree vs. no college degree)
• Region or census division
• Sometimes: household income, marital status, religious affiliation

Education weighting after 2016

Before 2016, most pollsters did not weight by education. The 2016 polls under-counted non-college white voters, who broke heavily for Trump, leading to a systematic under-estimate of Republican support in industrial Midwestern states. After 2016, most pollsters added education weighting. After 2020, most pollsters added it more aggressively, especially for state-level polling.

This is a real methodological improvement. It's also a methodological assumption: the pollster has to decide what the target distribution is — what percentage of voters in 2028 will be non-college whites? — and that target is itself a guess. Different pollsters use different targets. Their results will diverge.

How to spot a heavily weighted poll

A poll is "heavily weighted" if a small number of raw respondents are doing a lot of the work. The clue is in the effective sample size, sometimes reported as the "design effect" or "n effective." If a poll has 1,000 raw respondents but an effective sample size of 600, the design effect is high — meaning that some demographic slices were small in the raw sample and got big weights, which inflates the MoE.

Most pollsters do not disclose this, unfortunately. AAPOR-transparency-compliant pollsters do.

5. House effects

Different pollsters consistently produce results that lean a few points one way or the other. This is called the house effect, and it is empirically well-documented.

Examples

• Rasmussen Reports has historically polled 3–5 points to the right of the polling average. (Rasmussen's methodology is opaque; they do not participate in AAPOR transparency.)
• The Trafalgar Group also tends to show Republicans 2–4 points stronger than the average; they are one of the few firms to have correctly projected Trump's 2016 Pennsylvania and Michigan wins, which gave them credibility, though their 2020 and 2022 results were mixed.
• Quinnipiac University has tended to lean 1–3 points more favorable to Democrats, though their house effect has shrunk in recent cycles.
• Public Policy Polling (PPP) is openly Democratic-affiliated.
• Fox News polling (now run by Beacon Research and Shaw & Co. in partnership) is a mainstream non-partisan operation despite the network's editorial lean — its polls are generally well-regarded.

Why house effects exist

A pollster's choices on sampling frame, likely-voter screen, weighting targets, and question wording all push the result a small amount in some direction. A pollster who consistently uses a tight likely-voter screen will show Republicans doing 1–2 points better than a pollster using a loose screen. None of this is fraud — it is methodology making different bets.

What to do about house effects

Don't chase a single poll. Look at the average across many pollsters (next section). If a single pollster shows your preferred candidate 5 points up and the average shows them tied, the single pollster is probably wrong; the average is probably closer to truth.

6. Aggregators

The remedy for any single poll's noise and house effect is to average across many polls. This is what poll aggregators do.

Major aggregators

• FiveThirtyEight (now ABC News–owned; Nate Silver departed in 2023; the brand continues but with new staff and a modified model). Historically the most influential aggregator.
• Silver Bulletin (Nate Silver's Substack-based forecasting operation, launched 2024). Continues the FiveThirtyEight forecasting tradition under his direct control.
• The Economist runs an election forecasting model (originally led by Andrew Gelman's group). Generally well-regarded.
• RealClearPolitics (RCP) averages polls without modeling. Less sophisticated but more transparent — you can see exactly which polls are in the average. RCP averages tend to be a bit more Republican-friendly than 538-style averages because RCP weights all polls equally including some that 538 excludes.
• Decision Desk HQ runs a forecast and provides election-night calls for several news organizations.
• Split Ticket is an analytics outfit doing Senate, House, and state-level forecasting.
• 270toWin doesn't run forecasts but provides an interactive electoral-college map and aggregates state polling.

What aggregators do (and don't)

A simple aggregator computes a rolling average of recent polls, sometimes weighted by recency, pollster quality, and sample size. A forecasting model goes further: combining polls with "fundamentals" (economic indicators, presidential approval, incumbency, partisan lean of the state) to project a probability of each outcome.

Aggregators reduce the noise of any single poll. They cannot remove systematic bias that affects all pollsters the same way. If most pollsters use a similar likely-voter screen and that screen is wrong in the same direction, the aggregate will be wrong too. This is what happened in 2016 in the upper Midwest.

A forecasting model that says "Candidate A has a 70% chance of winning" is not predicting the outcome. It is saying: in 100 simulations of the election based on the available evidence, the model thinks A wins about 70 of them. Candidate B winning is not a "shock"; it is the 30% case happening.

7. Online panels vs. RDD vs. live caller

The state of polling methodology has changed dramatically since 2010.

Live caller (random digit dialing — RDD)

Traditional live-caller polling dials random phone numbers — both landline and cell — and conducts the interview live. Response rates have collapsed: in the 1980s, around 60–70% of dialed numbers produced an interview. Today it is under 5%. Most live-caller polls require 20,000+ dials to get 1,000 interviews. The cost has risen accordingly. Live-caller polls remain the methodological gold standard for many academic surveys (ANES, GSS) but are economically untenable for routine media polling.

Online panels

A pre-recruited group of respondents who agreed to take surveys in exchange for small incentives. The largest panels (Ipsos KnowledgePanel, NORC AmeriSpeak) are recruited via probability-based methods and are roughly comparable to RDD in coverage. Most others (YouGov, Morning Consult, Civiqs) are recruited via opt-in online ads and rely heavily on weighting to approximate a representative sample.

Online panels are cheaper, faster, and now produce most public polling. They have generally performed comparably to live-caller polls in election forecasting, though there are concerns about panel attrition (long-time panelists may differ from the general population) and about how well opt-in panels reach low-political-engagement respondents.

Text-to-web

Surveyors send text messages that link to a web survey. Cheap, fast, and growing, especially for state-level polling.

The bottom line: No single method is now the gold standard. A serious methodologist would ideally compare results across methods.

8. The 2016, 2020, 2022, and 2024 polling errors

2016

National polls showed Hillary Clinton up about 3 points; she won the popular vote by 2.1. So national polls were close. State polls in the upper Midwest were systematically off by 3–5 points, mostly because of insufficient education weighting. Pennsylvania, Michigan, and Wisconsin were the misses.

2020

National polls showed Biden up about 8 points; he won by 4.5. State polling underestimated Trump again, this time by 1–4 points. Education weighting had improved; non-response was the new culprit (it appears Trump-supporting voters were less likely to respond to surveys). Some pollsters added partisan weighting (counting registered Republicans more heavily) after 2020.

2022

Polls overstated Republican strength in the Senate by 2–4 points. The "red wave" failed to materialize. Likely-voter models built on assumptions of low Democratic turnout didn't pan out — Democrats were unusually motivated by the post-Dobbs environment and by abortion ballot initiatives.

2024

The polling-error story is more complicated. National polls were quite close to the final popular-vote margin. State polls in the seven battlegrounds were generally tighter than the final results, with most showing the race "in the MoE" right up to election day. The result — Trump winning the popular vote and clean-sweeping the seven battlegrounds — was inside the polls' uncertainty bands but on the edge. The polling industry's post-mortem (still being written) suggests another small miss in the Trump-favorable direction, smaller than 2016 or 2020 but still systematic.

The pattern: Trump-era polling has had a small but persistent under-estimate of Trump support, attributable largely to differential non-response. Pollsters have made progress; the problem hasn't fully gone away.

9. Partisan and non-partisan polls

Identifying a partisan poll

Look at the sponsor. If a poll is sponsored by a campaign, a party committee, a 501(c)(4) advocacy organization, or an explicitly partisan think tank, it is a partisan poll. The numbers may still be technically accurate, but the release decision — what to publish, what to suppress — is strategic. Campaigns release polls that show their candidate doing well. They keep polls that show them losing.

A non-partisan poll is sponsored by a media outlet, a university, or a non-partisan research foundation (Pew, Gallup, ANES, GSS) and is released regardless of whether the result helps a particular side.

Internal vs. public

Campaign internal polls (often "tracking polls" run nightly) are tools for campaign decision-making. They are sometimes leaked to reporters strategically. Treat any "campaign internal poll" leaked to a reporter as an act of campaign messaging. It may or may not be representative of the campaign's full polling.

Push polls

A push poll is not a poll at all. It is a phone bank that calls voters and presents negative information about a candidate as part of a "survey." The questions are designed to suggest, not to measure: "If you knew that Senator Smith voted to raise your taxes by 50%, would you be more or less likely to vote for him?" These calls tend to be brief (real polls take 8–15 minutes), reach huge numbers of voters (real polls reach 500–2,000), and are usually run in the final week of a campaign. Push polls are sometimes confused with negative-message testing, which is a real polling technique done with small samples; push polls are messaging, not research.

If you receive a "poll" that asks 2–3 leading questions and ends in under 90 seconds, you got push-polled.

10. Approval polling and issue polling

Approval

Presidential approval is the most-tracked single number in American politics. It moves slowly, mostly within a 10-point band for any given administration, and reflects partisan loyalty more than current events. The headline approval number conceals huge intra-party uniformity: about 90% of in-party voters approve, about 90% of out-party voters disapprove, and the meaningful variation is in the 10–25% of voters with weaker partisan attachment.

Approval polling is most informative about changes (a 5-point drop is news; a 5-point absolute level is not).

Different pollsters' approval numbers diverge by 4–8 points at any given moment because of house effects. The Gallup numbers are typically a few points lower than RealClearPolitics averages; both are within their own reasonable methodological choices.

Issue polling

Issue polling has a problem that horse-race polling does not: voters often hold inconsistent positions, and the way you ask the question determines the answer.

Examples: - "Do you favor or oppose government spending more on healthcare?" → Strong majority favor. - "Do you favor or oppose raising taxes to spend more on healthcare?" → Plurality oppose. - "Do you favor or oppose lowering the federal deficit?" → Strong majority favor. - "Do you favor or oppose cutting Medicare to lower the federal deficit?" → Strong majority oppose.

These contradictions are not respondents being stupid. They are real: voters want both more services and lower taxes; both lower deficits and protected entitlements. Issue polling that doesn't force the trade-off can produce nonsense. Issue polling that does force the trade-off (forced-choice questions) produces more realistic but also more contentious data.

Question wording also matters enormously. "Estate tax" vs. "death tax" gets different answers. "Welfare" vs. "assistance to poor families" gets different answers. Read the actual question.

11. What polls cannot tell you

A poll is a snapshot of self-reported attitudes at one moment. It is not:

• A predictor of future behavior. People's stated voting intention months out is not their actual behavior. Voters lie to themselves about whether they will turn out. They change their minds. New information arrives.
• A measure of intensity. A 60–40 issue split where the 40 are intensely motivated and the 60 are mildly supportive will produce different political outcomes than a 60–40 split where the 60 are intense and the 40 mildly oppose. Polls rarely measure intensity well.
• A measure of "the will of the people" in a normative sense. A majority preference for a policy is data; whether a representative should vote for it depends on a theory of representation (delegate, trustee, partisan) that polling cannot resolve.
• Free of social desirability bias. People give answers they think the interviewer wants to hear, especially on sensitive topics (race, religion, sexual behavior, drug use, voting). Online and self-administered surveys reduce this bias somewhat; live-caller polls amplify it.

12. The 30-second checklist

When you see a poll headline, run through this list before believing or sharing:

1. Who paid for it? Non-partisan sponsor: weight more. Campaign or partisan sponsor: weight less.
2. Who was sampled? All adults, registered voters, or likely voters? On a horse race, only LV is informative.
3. What's the sample size? Under 500: large MoE, treat skeptically. Over 1,000: standard.
4. What's the MoE? And: what's the lead? If lead is less than 1.7× MoE, it's a statistical tie.
5. When was it fielded? Pre- or post-major event? Within last 7 days?
6. Is it consistent with the polling average? If a single poll diverges 5+ points from the average, treat it as an outlier until corroborated.
7. What's the trend? A single poll is noise; a trend across pollsters is signal.

13. Where to find polls and methodology resources

Polling data and aggregates

• FiveThirtyEight (fivethirtyeight.com) — aggregator, model, pollster ratings.
• Silver Bulletin (natesilver.net) — Nate Silver's forecasting Substack.
• RealClearPolitics (realclearpolitics.com) — polling averages.
• The Economist election models — usually published as interactive forecasts.
• 270toWin (270towin.com) — electoral-college map and state polling.
• Polling Report (pollingreport.com) — long-running collection of issue polling.

Long-running survey programs

• Pew Research Center (pewresearch.org) — issue and demographic polling.
• Gallup (news.gallup.com) — historical approval and issue tracking.
• American National Election Studies (ANES) (electionstudies.org) — academic pre/post-election survey, gold standard for political-science research.
• General Social Survey (GSS) (gss.norc.org) — broader social attitudes since 1972.
• Cooperative Election Study (CES, formerly CCES) (cces.gov.harvard.edu) — large-N (60,000+) academic survey.

Forecasts and race ratings

• Cook Political Report (cookpolitical.com) — race ratings, PVI, election analysis.
• Sabato's Crystal Ball (centerforpolitics.org) — University of Virginia forecasting.
• Inside Elections (insideelections.com) — Nathan Gonzales's analysis.

Methodology resources

• AAPOR (American Association for Public Opinion Research, aapor.org) — standards, ethics, transparency database.
• Pew Research Center methodology section (pewresearch.org/methods) — accessible explainers on weighting, sampling, MoE.
• Andrew Gelman's blog (statmodeling.stat.columbia.edu) — statistician's running commentary on polling and political-data issues.

A final word

Reading polls well is a skill. It takes practice. Watch the same races and pollsters across a cycle and you will start to recognize the house effects, the methodological choices, the predictable patterns of error. You will also start to notice which reporters write about polls accurately and which ones ignore the MoE.

Polls are not a magical window into the public mind. They are a measurement, with all the limits of measurement. Treat them as data, not as oracle. Compare them. Average them. Doubt the outliers. Trust the trend. And never repeat a single poll as if it were "what America thinks."

That last instruction will keep you ahead of 90% of the political commentary you will encounter.