Mobile News Mix

The GRIT Report’s Shocking Findings about the Research Panel Mess

Posted by admin on Jun 8, 2017 9:31:26 AM


GreenBook GRIT Cover Q1-Q2


There’s hardly a word for it other than “shocking” – although “travesty” does come to mind: only 74% of consumer sample providers think they owe it to their clients to specify whether the research panels they’re selling are representative of the consumer populations their clients need to target.


This and other eye-popping findings about panel quality and provider transparency leap from the downloadable pages of  the newly-released GreenBook Research Industry Trends Report (GRIT) for the first half of 2017.


If you were told that only 74% of surgeons thought they owed you an explanation of what they’d be doing when they cut you open, wouldn’t you make darn sure to avoid the scalpels of the other 26%?


If only 74% of home insurance agents were willing to give you a clear and complete rundown of the coverage they were selling, wouldn’t you walk out if you found yourself sitting across a desk from one of the other 26%?


GreenBook’s latest survey of market research professionals also found that 88% of research buyers think panel providers should “advise on whether their samples are representative of a target population.” What’s puzzling is that 12% of buyers don’t think that information is mandatory.


Fully 45% of the research clients who responded to the GRIT report’s survey of 2,637 market research professionals believed that panel quality is getting worse. Only 20% said it’s getting better. It’s no surprise that panel providers surveyed for GRIT painted a far rosier picture: 55% said their product is getting better, and 20% admitted it’s getting worse. This is truly shocking. If this happened in the construction industry – 20% of concrete or steel providers copping to declining quality – there’d be a Congressional investigation.


With that in mind, it’s also hardly surprising that, as the GRIT authors note, “only about one in ten insights buyers say their projects compare 'very well’ to their ideal project.” About 60% of buyers said their projects performed “somewhat well” when measured against a standard of ideal quality. That means nearly a third of market research clients come away disappointed that they didn’t get anything approaching the quality they should expect. Chronic disappointment without acting decisively to change things leads to not caring. And not caring leads to not surviving. 


What a mess – and please note that “mess,” “shocking,” and “travesty” are our descriptors for the state of sample provision, not GRIT's. GreenBook put it a little more delicately: “Some participants seem to understand that there is a disconnect between sample providers and buyers, both in collaboration and transparency.”


One unnamed MR professional quoted in the GRIT report just wished that panel providers would “[be] forthcoming on all the sources a sample comes from. Without this, the industry will constantly sustain black eyes.”


Another GRIT respondent had this to say about the terrible consequences of slipshod panel recruitment and cultivation: “The way the industry treats respondents is not improving so I am not sure why we would expect the quality of response/engagement to improve.”


To their great credit, GreenBook’s editors have been sounding a clear alarm about panel quality and panelists’ dissatisfaction with their survey experiences for more than a year. The GRIT report concludes with a detailed addendum on consumers’ satisfaction with the survey-taking experience, drawn from a separate study GreenBook recently published. Taking care of panelists by providing engaging, well-designed survey experiences is at “the heart of product development and marketing, but yet are hardly even a consideration in research,” write the GRIT authors. “Participants are the lifeblood of market research, and disregarding the respondent experience in the research process is counter-productive to say the least.” [The italics are ours].


Maybe the reason that more than a quarter of panel providers won’t tell you whether they can fill your survey-targeting needs is that the less said about their product on all fronts, the better.


The GRIT report proposes “a priority action list” for addressing the panel crisis. Topping the list is a suggestion to “Go `mobile first’ in designing studies.’” That’s a start, but in our view it won’t get you across the finish line. To get the insights you need from mobile research, you have to meet the smartphone-obsessed public where it lives. So you should be looking very closely at aligning your projects to consumers’ preferred information and content platform, the mobile app. MFour’s in-app research fields questions to a large and engaged active panel of more than one million U.S. consumers who take surveys on their smartphones. It’s not merely “mobile first.” It’s mobile definitive.


The takeaway we want to add to the GRIT report is that the research technology and panel you need to overcome all the obstacles GRIT documents already exist. Seize them and you and your research chops will gain a great deal of credibility among your clients or top brand executives, because you'll be giving them the data and insights they need to make the right business moves. We won’t go into particulars here, but to learn more about how to get mobile right, solve your panel problems and impress your research stakeholders, click on any or all of the links below. Or get in touch at


MFour Answers GreenBook’s Call to Solve the Panel Crisis


How Do Mobile Best Practices Drive Panel Quality? Take Six Minutes To Get Tuned In


MFour Panel Demographics


Mobile Panel Experience – and Why it Matters

Topics: MFour Blog

Post a comment

Subscribe to Email Updates

Recent Posts

Posts by Topic

see all