Measuring Public Opinion

Democracy requires knowing what the people want. But how do we know what the people want?

Why Public Opinion Matters

In theory, representatives respond to constituent preferences. In practice, they need to know what those preferences are.

Public opinion influences:

• Electoral outcomes
• Policy decisions
• Media coverage
• Interest group strategies

Politicians obsessively track polls. Understanding how those polls work (and fail) is essential.

Scientific Polling

The Basics

A well-designed poll uses a random sample of a population to estimate the views of the whole population.

Key concepts:

Population: The group you want to learn about (e.g., American adults, likely voters in Michigan)

Sample: The subset you actually survey

Random sampling: Every member of the population has an equal chance of being selected. This is what makes inference possible.

Sample size: Larger samples are more precise. Most national polls use 800-1,500 respondents.

Margin of error: The range within which the true value likely falls. A poll showing 52% support with a 3% margin of error means support is probably between 49% and 55%.

Sampling Methods

Probability sampling: Random selection. The gold standard. Expensive and difficult.

Convenience sampling: Survey whoever’s available. Cheap but unreliable.

Quota sampling: Ensure the sample matches the population on key demographics. Better than convenience, not as good as probability.

Online panels: Recruit large groups, sample from them. Increasingly common as phone response rates collapse.

The Response Rate Problem

In the 1990s, about 35% of people contacted agreed to be polled. Today it’s around 5%.

Why it matters: If the 5% who respond differ systematically from the 95% who don’t, the poll is biased.

Weighting: Pollsters adjust results so the sample matches the population demographically. If young people are underrepresented, their responses count more. This helps but can’t fully solve the problem.

Question Design

How you ask shapes what you learn.

Question Wording Effects

Leading questions: “Do you support the president’s irresponsible plan?” will get different results than “Do you support the president’s plan?”

Social desirability bias: People underreport socially undesirable views (racism, not voting) and overreport desirable ones (charitable giving, recycling).

Framing effects: “Estate tax” polls differently than “death tax.” Same policy, different words, different results.

Question Order Effects

Earlier questions prime responses to later ones.

Ask about crime, then ask about race: different results than race, then crime.

Good pollsters randomize question order to minimize these effects.

Open vs. Closed Questions

Closed (multiple choice): Easier to analyze, may force false choices

Open (write your own answer): Richer information, harder to code, prone to non-response

Most polls use closed questions for statistical analysis.

Types of Polls

Benchmark Polls

Early campaign polls establishing baseline support. “Where are we starting?”

Tracking Polls

Continuous polling measuring change over time. Rolling samples allow real-time monitoring.

Exit Polls

Surveys of voters as they leave polling places. Used to project winners and analyze the electorate. (The 2000 Florida exit poll debacle damaged their reputation.)

Push Polls

Not real polls. Propaganda disguised as research. “Would you be more or less likely to vote for Candidate X if you knew they beat their spouse?”

Designed to spread negative information, not gather data.

Focus Groups

Small group discussions, not statistical samples. Used for qualitative insight: how people talk about issues, what arguments resonate.

Evaluating Poll Quality

Who Conducted It?

Academic and nonpartisan organizations (Pew, Gallup, university polls) are generally more reliable than partisan pollsters or media organizations under deadline pressure.

What’s the Population?

“Adults” is easier to poll than “likely voters.” But likely voters predict elections better. Definitions of “likely voter” vary and affect results.

What’s the Methodology?

Phone (landline vs. cell), online, in-person. Each has different biases. Response rates matter.

What’s the Margin of Error?

Standard is about 3%. Beware polls that don’t report it.

What Were the Actual Questions?

Reputable pollsters release full questionnaires. If they don’t, be suspicious.

Poll Aggregation

Individual polls have error. Aggregating multiple polls reduces error.

Sites like FiveThirtyEight and RealClearPolitics combine polls, weighting by quality and recency. Aggregates are more reliable than individual polls.

The 2016 and 2020 Polling Misses

2016

National polls were roughly accurate (Clinton won the popular vote). State polls in the Midwest systematically underestimated Trump support.

Possible causes:

• Late-deciding voters broke for Trump
• Education weighting was inadequate (non-college whites were undersampled)
• Social desirability bias (shy Trump voters)

2020

Polls overestimated Biden’s lead, especially in battleground states. Trump outperformed polls again.

Possible causes:

• Differential non-response (Trump supporters less willing to take polls)
• COVID-related turnout changes
• Late campaign shifts

Polling is harder than it used to be. The industry is adapting but hasn’t solved the problem.

Ideology and Public Opinion

What Americans Believe

Most Americans are ideologically innocent. They don’t have coherent liberal or conservative belief systems. They have opinions on specific issues that don’t fit neat categories.

Symbolic ideology: About 35-40% identify as conservative, 25% as liberal, the rest as moderate.

Operational ideology: When asked about specific policies (Social Security, Medicare, environmental protection), majorities often support liberal positions.

Americans are “symbolically conservative but operationally liberal.”

Policy Preferences

On most issues, Americans cluster in the center. Strong partisans are vocal minorities.

Exceptions: Abortion, guns, and immigration show genuine polarization with fewer people in the middle.

The Takeaway

Public opinion polling is essential to democracy but inherently imperfect.

Understanding polls means understanding:

• How random sampling works
• Why question wording matters
• How to evaluate poll quality
• Why polls sometimes miss

The gap between what polls measure and what the public actually thinks is why elections still surprise us.