Keeping up with the latest election polls is more challenging than ever.
Polling has evolved considerably even in the past four years, and methodologies have shifted fast, with new attempts to target the previously hard-to-reach, potentially pivotal fraction of former president Donald Trump’s coalition.
We also need to alter our perspective on the matter.
Here, on the eve of election year, is your roadmap to the next year of polling mayhem: a rundown of what any astute political watcher worth their salt should keep an eye out for. With this in mind, you’ll be able to discern which polls of 2024 are worth your time and attention, and which ones you can safely disregard. Also, those who read the poll (and write about it on social media in particular) need to understand what the findings represent and what they don’t.
Paying close attention to a poll’s margin of error and who commissioned it are two fundamental fundamentals that have not changed. Unfortunately, there is now even more work involved in being a well-informed voter. The process of interviewing and selecting poll participants should be well documented, and more transparency should be demanded regarding it. The meaning of “leading” a candidate is murky in a politically divided nation, where ultra-important races often come down to the wire.
The arguments concerning Trump and President Joe Biden’s polls have begun even if it is not until 2024. Now you know how to interpret the polls, and here’s what pollsters owe you:
Pollsters are modifying their methods of interviewing
Online surveys and phone polls were the vast majority of 2020 election polls just four years ago.
In order to piece together representative samples, pollsters are now not just adopting new procedures, but also combining methods within specific surveys.
Whether respondents are contacted by text or email, are current members of an online panel, or complete the survey after clicking on an ad on another website, many polls now combine web-based approaches with phone interviews. However, there are still some polls that only employ one methodology.
Respondents for CNN’s most recent survey were chosen by mail and then interviewed over the phone and online (more on this below). The latest survey by the Wall Street Journal included both online responses from respondents contacted via text message and phone interviews.
There may not be a universally accepted method of polling anymore due to changes in Americans’ communication patterns. We won’t know the full extent of the benefits and drawbacks of any methodology until after the 2024 election.
That’s why it’s crucial to know the interview process. The “mode” or interview approach used in a public poll should be demanded and noted by those reading the poll.
Alternate methods of communicating with individuals
How participants were selected to take part is one aspect of the significant methodological shifts in polling.
Twenty years ago, practically all public surveys relied on randomly dialling phone numbers to obtain a representative sample. This method relied on the area code and exchange to identify specific geographic areas. However, that was the era when landline phones were in almost every American home.
Now, surveys carried out in such a manner constitute a clear minority. On the contrary, a large number of pollsters rely on voter rolls, a tactic originally employed by internal campaign pollsters who have long aimed to reach out to individuals they are aware are registered to vote.
Even if some pollsters are utilising other approaches, that can still be in the form of phone surveys. CNN uses the home addresses recorded in the voter file to find some of the respondents for its state polls. After that, they’ll get an online survey invitation.
In contrast, the vast majority of online polls draw on preexisting databases of survey takers, who may or may not be registered voters. Like the CNN poll, some panels are put together at random, a process called probability sampling. Rather than randomly recruiting users to fill out surveys, some use “opt-in” panels of people who have already offered to do so, such as the AWN|Morning Consult poll.
So, how should elections be conducted? Most pollsters believe that surveys called from the voter database are superior to those that use randomly generated phone numbers when it comes to phone polling. Voter records contain a wealth of information, including gender, racial identity, party affiliation (in some states), and voting history.
Online surveys constructed using probability samples outperformed those derived from opt-in panels on the majority of metrics, according to a study conducted by the Pew Research Centre earlier this year. On the other hand, the probability polls performed worse on one measure? Participation in the 2020 presidential election, implying that the benefits of probability online surveys aren’t always applicable to this type of survey.
Correcting the mistakes made in 2016 and 2020
Pollsters have another, more direct tactic at their disposal: asking individuals to recall their vote for the president from the previous election. This is in addition to their efforts to contact large segments of the public.
Pollsters are increasingly using weighting surveys to match the results of the 2020 election. This is particularly true when trying to explain why Trump and the GOP were underrated in the previous two presidential contests. For instance, Republicans who take the time to fill out surveys tend to have lower levels of support for Trump compared to those who don’t, suggesting that conventional methods of gauging party identity might not be enough.
While it does assist, most pollsters agree that it isn’t a miracle cure. The report claims that if the New York Times had given more weight to respondents’ recollection of their 2020 presidential vote, the findings of the 2022 polls conducted with Siena College would have been less accurate.
Another reason to pay attention to whether pollsters are considering previous votes is that this approach can reduce volatility by reducing survey-to-survey jumps.
Define “lead.”
It is already difficult to interpret election polls because to the narrow margins that have determined presidential elections in recent years.
That is why the margin of error and the strength of the lead between the two candidates must be carefully considered. This week’s New York Times/Siena College poll, which showed President Joe Biden ahead of Trump by 2 points, is not statistically significant.
In my opinion, a fairly straightforward rule of thumb is that a candidate is not clearly leading if the gap between them is smaller than the margin of error for the survey. A virtual tie or dead heat describes the current state of the race.
The critical point is that the margin of error is applied to the percentage of votes cast for both candidates. A 2-point edge is comfortably within the margin of error, according to the New York Times/Siena survey, which had a range of plus or minus 3.7 percentage points.
The leading candidate can be considered to have a “slight” advantage if the difference between them and the other contenders is one to two times the margin of error. Given that the margin of error is applicable to both statistics, it is plausible that candidate is not certainly ahead. Their lead could be smaller than what the survey shows, though.
A candidate is considered to be well ahead of the pack if their lead over the other is at least twice the margin of error.
An example from this week’s actual world: With a margin of error of plus or minus three percentage points, our AWN|Morning Consult poll of likely voters in California’s March 5 primary found that Democratic Rep. Adam Schiff led his closest opponent in the state’s Senate race by nine points, with 28 percent to 19 percent.
Schiff was so in a commanding position. However, in California, the top two finishers in each party’s primary go on to the general election. And while Garvey was officially in second place, Democratic Representative Katie Porter was only two points behind, so it’s fair to say that they’re virtually neck and neck.
The additional information that is necessary
While the polling scene may be evolving, that doesn’t imply the old norms are irrelevant.
Take note of the poll’s sponsors: Is someone from the outside trying to use the results to further their agenda—a political candidate, party committee, partisan media outlet, etc.? You shouldn’t automatically dispose of the poll; the polling company still cares about its reputation. However, take the findings with caution.
At what time was the survey taken? When did the big news event that could have an impact on the results happen? Before or right after? Was it just for a single day, which would have meant that only the most accessible voters could have cast their ballots?
How does this poll’s outcome compare to others? Outliers do occur; in fact, it is a statistical fact that 5% of polls will be off by more than the margin of error.
Just because one poll found a different outcome doesn’t mean the other is incorrect; the election landscape may have changed in the time between the surveys. Still, additional proof, positive or negative, is usually preferable.
Remember, patience is the most crucial guideline. Along with the more concrete suggestions made up there, it’s an admirable quality that will help us all when we cast our ballots in 2024. It’s tempting to draw wide conclusions from a single poll, but doing so is rarely wise.
