Let's Debate Panel Fraud and the Future of Market Research

Posted by admin on Aug 22, 2017 10:26:05 AM




There’s nothing like a courteous, substantive argument between two market research veterans who are passionate about the industry’s future and how it can move forward and prosper during a time of dizzying change. From these kinds of conversations, only good things can arise.


In that spirit, here’s a summary of one such conversation that took place recently on LinkedIn between Todd Costello, a Senior Solutions Executive for MFour who has long held leadership positions in the Insights Association (the industry’s leading nonprofit professional group), and a consumer insights professional with whom Todd has had a longstanding professional relationship.

You can click here for the full exchange, but here's the quick summary.


As he regularly does, Todd shared a post from the MFour blog with his many LinkedIn connections, as a way of keeping them up to speed on developments in market research overall and in-app mobile approaches in particular.


The blog post that touched off Todd’s exchange was entitled “Why Online Research Is Like a Quarterback Who’s Over the Hill. It drew an analogy between online research and a great athlete (in this case NFL quarterback Brett Favre) who flourishes for many years but inevitably can’t keep up the same level of performance, forcing his team to make a change. 


Todd’s colleague began by complaining that he’s tired of "'this methodology bad’ vs. ‘our methodology good’ article[s].” He asserted that “every methodology has bias and sampling error. There is nothing fundamentally broken about online research. In fact, it's thriving.”


Todd countered with polite disagreement, touching on some of the key points about consumers’ rapid shift to smartphones and mobile apps – and how that obligates researchers to rethink how to reach them. “Change is always hard but necessary for growth,” he wrote.


Additionally, Todd pointed out some of the specific differences between in-app mobile and online surveys:

  • Response rates: 55% for in-app mobile vs. 1-2% for online
  • Representation: in-app mobile's essential for reaching Millennials
  • Engagement: satisfying in-app mobile performance allows for LOI of 20+ minutes
  • Panel fraud: Todd noted that he belongs to five online panels, each under a different identity, and that he receives 200 survey invitations a day, affording multiple opportunities to duplicate his responses (with in-app mobile, each smartphone's unique ID validates and makes duplication impossible)

For an objective point of view, Todd invited his LinkedIn connections to read recent editions of the GreenBook Research Industry Trends Report (GRIT), in which the authors have been emphatic about the industry’s need to confront serious problems with online panels – and have urged getting mobile right as one of the first and most important steps the industry needs to take.


Todd’s skeptical colleague came away from the exchange satisfied that they’d had a meaningful talk: “Good stuff. Thanks for putting some more meat around the bones.”


This is how it should always go. To have a similar productive conversation about how in-app mobile differs from online and “mobile optimized” online, and how the in-app approach can meet your specific research needs, just get in touch at solutions@mfour.com.


And here’s a bonus for any classic rock fans out there – a wonderfully tuneful and idealistic song, whose refrain couldn’t be more appropriate: “Successful conversation will take you very far.”

Topics: MFour Blog

Subscribe to Email Updates

Recent Posts