A Guide to Understanding Polls

Photo by Trending Twitter Topics from 10.10.2019, Creative Commons

I’ve had a difficult relationship with polls for most of my journalistic career. This reached its apex (or nadir if you look at it that way) years ago when I wrote a column encouraging people to lie to pollsters.

Lying, I argued, would skew the polls and force politicians to make decisions based upon their instincts and their actual principles. I was deluged with angry emails from pollsters at various companies who denounced my attempts to convince people to lie as being “anti-democratic.”

Well, I’m not sure it was anti-democratic, but over the years I have come around to understanding the reason why politicians and others use polls. We live in a big, complex society. If you’re running for national office (or even for a local office that covers a large area) it’s almost impossible for you to speak to every one of your constituents about their concerns.

In this case, polling can provide you with a guide to how the public views particular issues or how they perceive a politician’s efforts to solve important problems or champion important initiatives.

There’s one problem.

People don’t understand polls. People don’t know how they work, how they are taken, who’s doing the polling, how you determine if you can trust a poll or not and how much a particular poll is worth versus another poll.

Over the years I’ve had the opportunity to speak or listen to some of the best minds on polling in Canada and United States. Based on the things that they told me and what I have heard them say, I offer this guide to helping people better understand polls and the roles they play in our democracies.

1) One poll is a snapshot, not a movie

The idea for writing this guide came to me when a friend messaged me on Facebook about a new CNN poll that showed Biden in trouble in battleground states. A devoted Democrat, he was worried that this signaled trouble ahead for the Biden campaign.

Maybe. Maybe not. One poll is a snapshot. Never totally believe what one poll alone tells you. There are so many different conditions that can affect the results of one poll that it’s only a snapshot of a particular moment in a political campaign. This is true whether in Canada or the United States.

What you want to do is look at polling averages – the aggregate of several polls over a period. For instance, I pointed out to my friend that if you look at the average of a number of polls taken in battleground states over the past few months, Biden actually had a lead of 3 to 5 points in many of them. If future polls continue to show trouble for Biden in battleground states, then he should worry.

But one single poll basically means little in the long run.

2) National polls in the United States are useless

Well, maybe not useless, but they don’t mean much. The brains behind Democratic and Republican campaigns pay scant attention to national polls. They are much more interested in state polls. Since America basically elects a president based on the electoral college, a politician can be doing very well in national polls and still lose the election.

Take 2016. Final polls showed Hillary Clinton winning the national popular vote by 2% to 3%, which she did. But since the president of the United States is not elected on a popular vote but by the electoral college, she lost because she did not carry enough states.

It’s a bit different in Canada and Britain where a national poll can give you a better idea of how political parties are doing with public. In Canada you still need to pay attention to the vote in the Prairies versus the vote in Ontario, for instance, but you can trust a national poll more than you can a similar poll in the United States.

Again, that’s only one poll. If a lot of polls show that advantage, then you can take it to the bank.

There are also some important questions that you need to ask about each poll that you consider:

A) Who did the poll?

There are good polling organizations and there are bad polling organizations. It is unfortunate that too many media outlets when referring to recent polls don’t bother to tell you much about the organization that did a poll.

For instance, Ron Elway, the chief political correspondent at NPR, taught me that Quinnipiac was one of the best pollsters in the business especially in its coverage of states like Florida, Ohio and Pennsylvania. Why? They’ve been doing it for a long time and so over the years have ironed out errors in their methods.

You can also put your trust in a Marist poll or Survey USA poll, organizations that have received consistently high marks for their lack of bias in polling and the way they conduct each poll. On the other hand, any poll that you see from TCJ Research or Strategic Vision treat with a huge grain of salt. These polls almost always skew as many as 2-4 points towards Republicans.

B) How many people did they poll?

Randomness is the key to good polling. Normally, to achieve a statistically good sample of the public you need to interview at least 1000 to 1200 participants. That gets harder and harder these days because fewer and fewer people are willing to take part in polls.

When pollsters interview fewer people, the results are harder to trust. That’s why it’s important to pay attention to the statistical error of the poll. For instance if a pollster only interviews 400 to 500 people about a candidate or an important question and the result has a high error rate of 5% or more that means if the poll was something like 49% in favor and 51% against the question, the real result could be anywhere from 54% in favor and 46% against to 44% in favor and 56% against. Those are dramatic differences.

(Note: A friend read this and added something I had forgotten about the margin of error. To quote Pew “A margin of error of plus or minus 3 percentage points at the 95% confidence level means that if we fielded the same survey 100 times, we would expect the result to be within 3 percentage points of the true population value 95 of those times.”)

A poll with a relatively small number of participants can give you a hint about how people feel about a politician or a particular issue, you just shouldn’t bet your house on the result.

C) Who did they ask?

In a political poll, you want to know if the poll was done of registered voters or likely voters. You get a much truer picture of a campaign when you interview likely voters. These are people who are, as the phrase says, likely to vote in election. Since the United States has such a lousy turn out in elections (be they federal, state, or municipal) somebody can be registered to vote but that doesn’t mean they’re going to go to the polls.

Good pollsters have ways to determine the difference between a likely voter and a registered voter by asking them questions like: did they vote in the last election, have they ever written a letter or an email to their local media, have they ever participated in a campaign, do they know the location of their polling station, etc.

So a poll that says 54% of registered voters support Donald Trump versus a poll that says 54% of likely voters support Joe Biden would be good for Trump but great for Biden.

D) How was the question asked?

The way pollster asks a question will often determine the way the participant will answer it. Good pollsters ask the same question in several different ways to get at the participant’s real views.

Bad pollsters do what are called “push polls” where they ask questions in such a way that they’re pushing you to answer in a fashion that suits the politician or the organization behind the poll.

Politicians and big corporations are infamous for doing push polls in order to produce a result that they like and then spread it among the media, who too often don’t bother to ask how the poll was conducted or how the questions were asked.

E) How was the poll weighed?

Pollsters weigh polls to try to produce a truer result. For instance women and seniors tend to answer the phone more often, which can skew the sex and age ratio of the poll. So they weigh (or adjust) the poll to better match the actual sex and age demographics of the public. Or they might oversample a group to get a more representative result. Good pollsters will include this information in their methodology explanations included at the bottom of each poll.

F) What is the pollsters’ history of success or failure?

I refer you to Nate Silver’s Pollster Ratings where he and his team at fivethirtyeight.com regularly look at the history of each polling organization in terms of successful polling of particular issues, how many polls the pollster has conducted and how the polls are conducted (live, land-line phone, internet, cellphone).

This is a great help to determine the difference between a well-known pollster who has conducted many polls and a fly-by-night organization that pops up to produce polls that favor a particular candidate or issue. It helps to know the success rate of polling predictions because that gives you a better picture of how you should view the poll.

An organization that has a 90% success rate (like Survey USA) should be given greater credence then a poll from an organization that only has a 70% success rate (like Survey Monkey).

G) The shame factor

People do indeed lie to pollsters. This is particularly true in a poll that concerns an unpopular politician or an issue such as racism or misogyny. Or in an exit poll taken the day of an election. If the person being surveyed actually holds racist or misogynist views, they may lie to the pollster about their positions creating a false result.

Pres. Trump benefited from this factor in 2016 particularly after his “grab them by the pussy” remarks. At that time voting for Trump was not seen as a particularly popular thing to do, even though a lot of people wanted to do it. It was also a factor in exit polls. So when a pollster asked if they supported Trump, many people who did said no.

So what does all this tell us about polling? I would recommend you read Nate Silver’s book “The Signal and The Noise” which looks at how polling is done and how predictions made by pundits and politicians often bear no semblance to reality.

Polling may not be a great way to determine how we feel about politicians or controversial issues in, but it may be the best way that we have going forward.

Science and the Battle Against Dogma, Superstition and Conspiracy Theories

Scientists at work. (Photo by Polina Tankilevitch from Pexels)

“Every great advance in natural knowledge has involved the absolute rejection of authority”
– Aldous Huxley

Science upsets things.

Science told us that man was not the center of the universe. Science then told us that man was not even the center of the galaxy. Science told us that the earth was not flat and that man evolved from a simple creature that crawled out of the mud billions of years ago. Science gave women control over their reproductive cycles. Science took us to the moon.

Science has of course done so much more, but these particular scientific achievements deserve mention because of the havoc they caused. And continue to cause. Many of these achievements are still rejected in the 21st century by people who cling desperately to dogma, superstition and conspiracy theories like a thread-bare teddy bear.

Whether the religious deniers who insist the world was only created about 6000 years ago and that dinosaur skeletons were left by a god as a kind of cosmic joke or those insist the earth is flat because that is what their eyes tell them and anything that undermines that belief is a “conspiracy” to hide the truth or those people who insist we never went to the moon because, well, we didn’t or we were just trying to fool the Soviets, they all deny science because if they accepted what science had achieved their imaginary worlds would be turned upside down.

Science asks questions. Constantly. Even about accepted science. Good science is constantly upsetting the apple cart, even among scientists.

Authorities hate questions. They prefer the known order, the ‘right’ way of thinking. Dogma. If authority tell the public this is the truth, then that is the truth, even if it isn’t.

Authorities of any political ideology hate science. The Soviets suppressed knowledge that did not conform to their communist ideology, just as China does today. Conservatives have suppressed science through the ages.

Which is why conservatives and the far-right in America have worked so hard to undermine science and knowledge almost since the beginning of the country. While founding fathers like Franklin and Jefferson were certainly men of science, the people who came after them largely were not. The men (and it was men back then and even mostly now) who controlled power and authority …and money… hated science because it so often acted as an antidote to their formulas to achieve more power and money.

Denials don’t matter

As we head further into the 21st-century science has never been more important than it is now. It has also never been under such a sustained attack from the right. Whether the issue is climate change, vaccines that have saved billions of lives, less polluting forms of new energy to replace fossil fuels, or how to combat a world-wide pandemic, science constantly fends off attacks, most of them ridiculous.

But here’s the thing about science. You can deny it all you want but that doesn’t change the data or what will happen. GOP governors and senators can hide data that blows up their lies or try to prevent scientists from speaking out. The president can lie about infection rates or try to peddle snake oil about “potential cures.”

It doesn’t matter.

The oceans will still rise. The infection rate will still grow. More businesses and people will choose new forms of energy. Discoveries will continue that show the earth’s age is about 5 billion years in a universe about 15 billion years old. Women will still continue to use contraception. Vaccines will still save billions of lives.

Conservatives and authorities can change. The Catholic Church that once jailed Galileo now not only champions his theories but accepts a “divinely inspired” form of evolution. A Catholic priest first formulated the idea of a “Big Bang” start to the universe. Things that many people once viewed as magic are now seen as everyday natural events – thanks to science.

Dogma. Superstition. Conspiracy theories. These are the enemies of humanity, which often try to masquerade as “the truth.” But they will always lose in the end. It may take a while, but science will not be denied forever.

Science is still the best bet humanity has for prolonged survival. Which is why every time authority seeks to undermine science it must be challenged and confronted. Not just by scientists but by all people who understand its value and importance.

Another Case of Constitutional Stupidity

Costco store. (Photo by ccPixs.com)

Late last week, the national retailer Costco announced that starting today, May 4, you could not go into a one of their stores without wearing a mask. This made Costco the first national chain to enact this kind of measure. In announcing the mask requirement, Costco said they were doing it to protect both shoppers and their employees. (Costco also announced limits on the amount of meat customers could buy.)

As I write this, Costco stores have yet to open, but you can count on one thing happening for sure. A potential customer without a mask will try to get into the store, loudly complaining about their Constitutional right to enter the store mask-less. This being America, one or more of these people may pull on a gun on anyone standing near the front of the store telling people they can’t enter unless masked. (It’s already happened in Ohio involving other stores.)

I’m sorry to inform these “good and patriotic” Americans but they DO NOT have a Constitutional right to go into Costco without a mask if Costco says they must. No right. Nada, Zippo, Zilch. Rein.

Costco is a private business, and as such can decide to do whatever the hell it wants to do. It could make a rule it will only allow customers to enter if they are wearing a clown nose. (My guess is that this would hurt business and so it’s unlikely.)

This is yet another case of Constitutional Stupidity, a condition that affects too many Americans particularly those on the right. It is the belief that you can do or say anything you want to in America and there will be no consequences. While it is true that you can do and say anything in America (if it’s legal), consequences are a WHOLE different issue.

I‘m a naturalized US citizen. In order to achieve citizenship, I had to study the heck out of the Constitution, US history and the way governments work at all levels. They really do quiz you on this stuff at your interview and they don’t only ask softball questions.

Like many other naturalized citizens, I ended up knowing a lot more about this stuff than the average American who can barely remember studying it in high school. What they do remember has been twisted in weird ways by watching too much Fox News or listening too frequently to Rush Limbaugh.

Here’s how it works (for those people who believe they have a Constitutional right to not wear a mask in Costco).

You have the right to unfettered political speech, to petition the government or to peacefully assemble (as the protesters in Michigan or elsewhere did) without the fear that (like in Russia or Iran) the secret police will show up at your door in the middle of the night and take you away, never to be seen by your family again.

You have the right to protest a government action. People who complained that Ohio Gov. Mike DeWine’s order for everyone to wear a mask out in public violated their rights may have had a point, which is why he probably rescinded it. Sometimes these protests don’t achieve the desired result as in Michigan (Gov. Whitmer basically ignored them), but people still have the right to protest.

However, if your boss decides that this speech or action is a deterrent to his or her business, you could find yourself out on your ear. You can sue them for wrongful dismissal, but in a suit of this kind your boss has a chance to show what they did was lawful, depending on your terms of employment or state/federal laws. But there are very few protections for workers.

While the government can’t throw you in jail for saying President Trump is a lying, misogynist, racist, phony creep, or stop you from going to the Michigan legislature with your AR-15 to protest, if your employer depended on federal government for work, for instance, they tell could you to stifle it.

At that point, the choice is yours. You can continue to protest, even if it costs you your job or your company business. Or even risk arrest if you push the point too far on private property, like a Costco store. Or you can shop at a store other than Costco that does not require a facemask and take your chances with COVID-19. Many people will.

But private business are not subject to many of the same restrictions that any level of government is. So yes, Costco can tell you that you cannot enter their store without a mask, and they are NOT violating your Constitutional rights.