Which Midterm Polls Should We Be Taking With a Grain of Salt?

Which Midterm Polls Should We Be Taking With a Grain of Salt?

Frank Bruni, a contributing Opinion writer, hosted a written online conversation with Amy Walter of The Cook Political Report and Patrick Ruffini, a Republican pollster, to discuss the state of polling and of Democratic anxiety about polls ahead of the midterms.

Frank Bruni: Amy, Patrick, as if the people over at Politico knew that the three of us would be huddling to discuss polling, it just published a long article about the midterms with the gloomy, spooky headline “Pollsters Fear They’re Blowing It Again in 2022.”

Do you two fear that pollsters are blowing it again in 2022?

Sign up for The Morning newsletter from The New York Times

Patrick Ruffini: It’s certainly possible that they could. The best evidence we have so far that something might be afoot comes from The Times’s own Nate Cohn, who finds that some of the Democratic overperformances seem to be coming in states that saw large polling errors in 2016 and 2020.

Amy Walter: I do worry that we are asking more from polling than it is able to provide. Many competitive Senate races are in states — like Arizona, Georgia, Pennsylvania and Wisconsin — that Joe Biden won by supernarrow margins in 2020. The reality is that they are going to be very close again. And so an error of just three to four points is the difference between Democratic and Republican control of the Senate.

Ruffini: This also doesn’t mean we can predict that polls will miss in any given direction. But it does suggest taking polls in states like Ohio, which Donald Trump won comfortably but where the Republican J.D. Vance is tied or slightly behind, with a grain of salt.

Bruni: So what would you say specifically to Democrats? Are they getting their hopes up — again — in a reckless fashion?

Walter: Democrats are definitely suffering from political PTSD. After 2016 and 2020, I don’t think Democrats are getting their hopes up. In fact, the ones I talk with are hoping for the best but not expecting such.

Ruffini: In any election, you have the polls themselves, and then you have the polls as filtered through the partisan media environment. Those aren’t necessarily the same thing. On Twitter, there’s a huge incentive to hype individual polling results that are good for your side while ignoring the average. I don’t expect this to let up, because maintaining this hype is important for low-dollar fund-raising. But I do think this has led to a perhaps exaggerated sense of Democratic optimism.

Bruni: Great point, Patrick — in these fractured and hyperpartisan times of information curation, polls aren’t so much sets of numbers as they are Rorschachs.

But I want to pick up on something else that you said — “polls will miss in any given direction” — to ask why the worry seems only to be about overstatement of Democratic support and prospects. Is it possible that the error could be in the other direction and we are understating Republican problems and worries?

Ruffini: In politics, we always tend to fight the last war. Historically, polling misses have been pretty random, happening about equally on both sides. But the last big example of them missing in a pro-Republican direction was 2012. The more recent examples stick in our minds, 2020 specifically, which was actually worse in percentage terms than 2016.

Walter: Patrick’s point about the last war is so important. This is especially true when we are living in a time when we have little overlap with people from different political tribes. The two sides have very little appreciation for what motivates, interests or worries the other side, so the two sides over- or underestimate each other a lot.

As our politics continue to break along educational attainment — those who have a college degree are increasingly more Democratic-leaning, those with less education increasingly more Republican-leaning — polls are likely to overstate the Democratic advantage, since we know that there’s a really clear connection between civic voting behavior and education levels.

Ruffini: And we may be missing a certain kind of Trump voter, who may not be answering polls out of a distrust for the media, polling and institutions generally.

Bruni: Regarding 2016 and 2020, Trump was on the ballot both of those years. He’s not — um, technically — this time around. So is there a greater possibility of accuracy, of a repeat of 2018, when polling came closer to the mark?

Ruffini: The frustrating thing about all of this is that we just don’t have a very good sample size to answer this. In polls, that’s called an n size, like n = 1,000 registered voters. There have been n = 2 elections where Trump has been on the ballot and n = 1 midterm election in the Trump era. That’s not a lot.

Bruni: We’ve mentioned 2016 and 2020 versus 2018. Are there reasons to believe that none of those points of reference are all that illuminating — that 2022 is entirely its own cat, with its own inimitable wrinkles? There are cats that have wrinkles, right? I’m a dog guy, but I feel certain that I’ve seen shar-pei-style cats in pictures.

Walter: First, let’s be clear. Dogs are the best. So let’s change this to “Is this an entirely different breed?”

I’m a big believer in the aphorism that history doesn’t repeat, but it does rhyme.

Ruffini: Right. Every election is different, and seeing each new election through the lens of the previous election is usually a bad analytical strategy.

Walter: But there are important fundamentals that can’t be dismissed. Midterms are about the party in charge. It is hard to make a midterm election about the out-party — the party not in charge — especially when Democrats control not just the White House but the House and Senate as well.

However, the combination of overturning Roe v. Wade plus the ubiquitous presence of Trump has indeed made the out-party — the G.O.P. — a key element of this election. To me, the question is whether that focus on the stuff the Republicans are doing and have done is enough to counter frustration with the Democrats.

Ruffini: 2022 is unique in that it’s a midterm cycle where both sides have reasons to be energized — Republicans by running against an unpopular president in a time of high economic uncertainty and Democrats by the Dobbs v. Jackson Women’s Health Organization decision overturning Roe. It’s really unique in the sweep of midterm elections historically. To the extent there is still an energized Republican base, polls could miss if they aren’t capturing this new kind of non-college, low-turnout voter that Trump brought into the process.

Bruni: Patrick, this one’s for you, as you’re the one among us who’s actually in the polling business. In the context of Amy’s terrific observation about education levels and the Democratic Party and who’s more readily responsive to pollsters, what are you and what is your firm doing to make sure you reach and sample enough Republican and Trump-inclined voters?

Ruffini: That’s a great question. Nearly all of our polls are off the voter file, which means we have a much larger set of variables — like voting history and partisan primary participation — to weight on than you might typically see in a media poll (with the exception of the Times/Siena polls, which do a great job in this regard). We’ve developed targets for the right number of college or non-college voters among likely voters in each congressional district. We’re also making sure that our samples have the right proportions of people who have registered with either party or have participated in a specific party’s primary before.

But none of this is a silver bullet. After 2016, pollsters figured out we needed to weight on education. In 2020 we weighted on education — and we got a worse polling error. All the correct weighting decisions won’t matter if the non-college or low-turnout voter you’re getting to take surveys isn’t representative of those people who will actually show up to vote.

Bruni: Does the taking of polls and the reporting on polls and the consciousness of polls inevitably queer what would have happened in their absence? I will go to my grave believing that if so many voters hadn’t thought that Hillary Clinton had victory in the bag, she would have won. Some 77,000 votes in Pennsylvania, Michigan and Wisconsin — the margin of her Electoral College loss — are easily accounted for by overconfident, complacent Clinton supporters.

Walter: In 2016, there were two key groups of people that determined the election. Those who never liked Clinton and those who disliked Trump and Clinton equally. At the end, those who disliked both equally broke overwhelmingly for Trump. And, those Democratic-leaning voters who didn’t like her at all were never fully convinced that she was a worthy candidate.

Ruffini: I don’t worry about this too much since the people most likely to be paying attention to the daily movement of the polls are people who are 100 percent sure to vote. It can also work in the other direction. If the polls are showing a race in a red or blue state is close, that can motivate a majority of the party’s voters to get out and vote, and that might be why close races in those states usually resolve to the state fundamentals.

Bruni: Evaluate the news media in all of this, and be brutal if you like. For as long as I’ve been a reporter, I’ve listened to news leaders say our political coverage should be less attentive to polls. It remains plenty attentive to polls. Should we reform? Is there any hope of that? Does it matter?

Ruffini: I don’t think there’s any hope of this getting better, and that’s not the media’s fault. It’s the fault of readers (sorry, readers!) who have an insatiable appetite for staring at the scoreboard.

Walter: We do pay too much attention to polls, but polls are the tool we have to capture the opinions of an incredibly diverse society. A reporter could go knock on 3,000 doors and miss a lot because they weren’t able to get the kind of cross-section of voters a poll does.

Ruffini: Where I do hope the media gets better is in conducting more polls the way campaigns conduct them, which are not mostly about who is winning but showing a candidate how to win.

In those polls, we test the impact of messages on the electorate and show how their standing moved as a result. It’s possible to do this in a balanced way, and it would be illuminating for readers to see, starting with “Here’s where the race stands today, but here’s the impact of this Democratic attack or this Republican response,” etc.

Bruni: Let’s finish with a lightning round. Please answer these quickly and in a sentence or less, starting with this: Which issue will ultimately have greater effect, even if just by a bit, in the outcome of the midterms — abortion or gas prices?

Walter: Abortion. Only because gas prices are linked to overall economic worries.

Ruffini: Gas prices, because they’re a microcosm about concerns about inflation. When we asked voters a head-to-head about what’s more important to their vote, reducing inflation comes out ahead of protecting abortion rights by 67 to 29 percent.

Bruni: Which of the competitive Senate races will have an outcome that’s most tightly tethered to — and thus most indicative of — the country’s mood and leanings right now?

Walter: Arizona and Georgia were the two closest races for Senate and president in 2020. They should both be indicative. But Georgia is much closer because the G.O.P. candidate, Herschel Walker, while he’s still got some problems, has much less baggage and much better name recognition than the G.O.P. candidate in Arizona, Blake Masters.

Ruffini: If Republicans are going to flip the Senate, Georgia is most likely to be the tipping-point state.

Bruni: If there’s a Senate upset, which race is it? Who’s the unpredicted victor?

Walter: For Republicans, it would be Don Bolduc in New Hampshire. They’ve argued that the incumbent, Senator Maggie Hassan, has low approval ratings and is very weak. It would be an upset because Bolduc is a flawed candidate with very little money or history of strong fund-raising.

Ruffini: I’d agree about New Hampshire. The polling has shown a single-digit race. Republicans are also hoping they can execute a bit of a sneak attack in Colorado with Joe O’Dea, though the state fundamentals look more challenging.

Bruni: You (hypothetically) have to place a bet with serious money on the line. Is the Republican presidential nominee in 2024 Donald Trump, Ron DeSantis or “other”?

Walter: It’s always a safer bet to pick “other.” One of the most difficult things to do in politics is what DeSantis is trying to do: not just to upend someone like Trump but to remain a front-runner for another year-plus.

Ruffini: I’d place some money on DeSantis and some on “other.” DeSantis is in a strong position right now, relative to the other non-Trumps, but he hasn’t taken many punches. And Trump’s position is soft for a former president who’s supposedly loved by the base and who has remained in the fray. Time has not been his friend. About as many Republicans in the ABC/Washington Post poll this weekend said they didn’t want him to run as did.

Bruni: Same deal with the Democratic presidential nominee — but don’t be safe. Live large. To the daredevil go the spoils. Joe Biden, Kamala Harris or “other”?

Walter: History tells us that Biden will run. If he doesn’t, history tells us that it will be Harris. But I feel very uncomfortable with either answer right now.

Ruffini: “Other.” Our own polling shows Biden in a weaker position for renomination than Trump and Democrats less sure about who the alternative would be if he doesn’t run. I also think we’re underestimating the possibility that he doesn’t run at the age of 81.

Bruni: OK, final question. Name a politician, on either side of the aisle, who has not yet been mentioned in our conversation but whose future is much brighter than most people realize.

Walter: If you talk to Republicans, Representative Patrick McHenry is someone they see as perhaps the next leader for the party. There’s a lot of focus on Kevin McCarthy now, but many people see McHenry as a speaker in waiting.

Ruffini: He’s stayed out of the presidential conversation (probably wisely until Trump has passed from the scene), but I think Dan Crenshaw remains an enormously compelling future leader for the G.O.P. Also in Texas, should we see Republicans capitalize on their gains with Hispanic voters and take at least one seat in the Rio Grande Valley, one of those candidates — Mayra Flores, Monica De La Cruz or Cassy Garcia — will easily be in the conversation for statewide office.

Bruni: Thank you, both. I just took a poll, and 90 percent of respondents said they’d want to read your thoughts at twice this length. Then again, the margin of error was plus or minus 50 percent, and I’m not sure I sampled enough rural voters in the West.

The Times is committed to publishing a diversity of letters to the editor. We’d like to hear what you think about this or any of our articles. Here are some tips. And here’s our email: letters@nytimes.com.

Follow The New York Times Opinion section on Facebook, Twitter (@NYTopinion) and Instagram.

Patrick Ruffini (@PatrickRuffini) is a co-founder of the Republican research firm Echelon Insights.

Amy Walter (@amyewalter) is the publisher and editor in chief of The Cook Political Report.

NEWS RELATED

Bavarian Podcast Works S5E22: Spain 1-1 Germany postgame analysis + Is Bayern Munich having a terrible World Cup?

Bavarian Podcast Works S5E22: Spain 1-1 Germany postgame analysis + Is Bayern Munich having a terrible World Cup? Another game on the books and Germany are still alive in the World Cup. In some ways, a draw versus an in-form Spain team looks good on paper, but Hansi Flick ...

View more: Bavarian Podcast Works S5E22: Spain 1-1 Germany postgame analysis + Is Bayern Munich having a terrible World Cup?

Slumping Ducks fall to streaking Kraken

Matty Beniers had a goal and two assists, Daniel Sprong scored the tiebreaking goal early in the third period and the Seattle Kraken tied a franchise record with their fifth consecutive victory, 5-4 over the Ducks on Sunday night. Jared McCann, Alex Wennberg and Vince Dunn scored in the ...

View more: Slumping Ducks fall to streaking Kraken

Rams, Sean McVay take another blow in lopsided loss to Chiefs

KANSAS CITY —  During this lost season, Rams coach Sean McVay absorbed numerous shots to his mental state. His offense no longer soars, and his star quarterback has twice been in the concussion protocol. His star wide receiver also is out because of injury. And the Rams’ offensive line continues to ...

View more: Rams, Sean McVay take another blow in lopsided loss to Chiefs

Dua Lipa Granted Albanian Citizenship Ahead of Country's 110th Anniversary of Independence

The pop superstar was born in London in 1995 to immigrant Albanian parents Anesa and Dukagjin Lipa from Kosovo.

View more: Dua Lipa Granted Albanian Citizenship Ahead of Country's 110th Anniversary of Independence

Small Plane Dangles From Power Transmission Tower in Maryland, Thousands Without Power

Link video: http://nbclim-download.edgesuite.net/Prod/NBCU_LM_VMS_-_WRC/23/483/19813529463-1080pnbcstations.mp4 NBC Universal, Inc. A small plane with two people aboard slammed into a power transmission tower in Montgomery County, Maryland, on Sunday evening, authorities said. Two people were left stranded in the crumpled aircraft 100 feet from the ground and the crash left large swaths of ...

View more: Small Plane Dangles From Power Transmission Tower in Maryland, Thousands Without Power

Alba: Spain did a ‘good job’ against Germany

Jordi Alba felt Spain did a good job against Germany despite being held to a 1-1 draw by Hansi Flick’s side on Sunday. Spain took the lead through Alvaro Morata but were denied victory when Niclas Fullkrug struck to earn Germany their first point. Alba shared his thoughts after ...

View more: Alba: Spain did a ‘good job’ against Germany

The Holiday Train rolls into the Capital Region, helping local food banks

The Holiday Train rolls into the Capital Region, helping local food banks The holiday season is a time of giving. That’s the inspiration behind the annual Holiday Train fundraising tradition. Hundreds of people weathered the storm as they showed up to see the festive train make its grand entrance ...

View more: The Holiday Train rolls into the Capital Region, helping local food banks

Raiders' Josh Jacobs caps 300-yard game with walk-off TD in OT

Raiders' Josh Jacobs caps 300-yard game with walk-off TD in OT SEATTLE — Josh Jacobs wasn't sure he'd play Sunday until about two hours before kickoff. The Las Vegas Raiders' fourth-year running back had to test out the left calf he tweaked in a noncontact drill in Friday practice. ...

View more: Raiders' Josh Jacobs caps 300-yard game with walk-off TD in OT

Singer Dua Lipa Granted Albanian Citizenship

Woman’s current boyfriend opens fire on her child’s father, his mother, his brother during custody exchange in NE Houston, police say

'I've Been Having An Affair With My Stepmother' — Man Shares His Regret After Dad's Death

Young Crypto Founder Shocks Industry With Sudden Death at 30 in His Sleep

Ex-AKB48 member Miki Nishino marries comedian 31 years her senior

Four Texas A&M players enter the transfer portal

Small Plane Dangles From Power Transmission Tower in Montgomery County; Thousands Without Power

Melissa Highsmith, missing for 51 years, found in Fort Worth

NFL star Odell Beckham Jr. removed from plane by police

The Frozen Trader Joe's Shrimp Fans Swear By

South Dakota will play No. 5 Houston in NCAA Tournament First Round

Plane that took off from Westchester County crashes into power lines in Maryland, FAA reports

OTHER NEWS

Breaking thailand news, thai news, thailand news Verified News Story Network