Photo credit: Michael Vadon (Donald Trump); Gage Skidmore (Hillary Clinton)
Here’s more bad news for online surveys: they’ve now been roped into the mudslinging of the presidential campaign, which can only mean more mud is being splattered indiscriminately on their reputation.
A recent headline in the New York Times reads “Why You Shouldn’t Trust `Polls’ Conducted Online.” The article details how data from respondents who answer invitations to take polls posted on news organizations’ websites is more or less worthless. These survey-takers simply aren’t representative of the voting public.
Granted, online market research has a much firmer basis in reality than the stunt-like ad-hoc online political polls the Times wrote about. But that basis is crumbling fast, and it’s dawning on many market researchers and clients that the data online research surveys produce has tumbled into a sinkhole of unreliability.
It’s a rare online research supplier that will publish its panel’s characteristics in detail. But researchers deserve to know who’s going to answer their questions, and anything less than open disclosure can breed doubt and distrust that's unhealthy for any industry.
For full disclosure of the demographic characteristics of the million-plus active members who make up MFour's Surveys on the Go® panel, just scroll to the bottom of this post. Please let us know if you can find comparable transparency elsewhere.
We did find one example in which a supplier of online sample published a detailed statistical profile of its panel composition in the U.S. Kudos to that company for doing the right thing.
But the facts it's sharing only add hard evidence that explains the rising note of discontent among end-users of market research, as the realization sinks in that the online surveys they're buying no longer are delivering the reliability and efficiency they need.
The supplier in question states that it has well over one million panelists in the U.S., of whom 71% are women. That's way out of sync with the reality of human biology, let alone the reality of the marketplace. It also lists the ethnicities represented on its panel – but glaringly omits Hispanics/Latinos. If research can’t connect with a representative cohort of U.S. Hispanics, how can it be in touch with the reality of the American marketplace?
Lastly, it acknowledges that only about 20% of its respondents complete surveys on a mobile device.
As many studies have made clear – not least MFour’s ongoing Millennial Insights Project -- mobile devices are contemporary society’s crucial, dominant, definitively pervasive (supply your own adjective) means of communication and information exchange. How can any industry, especially MR, whose raison d’etre is keeping a finger on the public pulse, credibly claim to be engaging a representative cross-section of the public if it’s communicating in ways most of the public regards as all but archaic?
We grant that, unlike the instant online presidential polls that will continue to proliferate from now to Election Day, online market research has integrity and does all it can to serve clients honorably and effectively. Its methods were verifiably effective for many years. But mobile technology has changed the world, and online’s day has passed. Soon it will be like the phone booth or the electric typewriter – a one-time necessity that fell into near-total disuse because technology transformed how the public communicates and transmits information.
“Follow the money” was the signature line from “All the President’s Men,” a classic film about the earthshaking political fallout from the 1972 presidential election. In market research, the new imperative is “follow the consumers.” The consumers are not online. They are to be found on mobile – in all their glorious, fully-representative diversity.
Research needs to be there, too.
As promised, here's the panel you'll get when you go with MFour. Please feel free to pass our chart around.
