RE: Study Finds Trouble for Opt-in Internet Surveys

In response to an article on ABC News: The Numbers¹

It’s not a problem that is going away anytime soon. Panel companies have an incentive to continue using less than precise and less than accurate samples, defined by two-words: gross margin.

The shear volume of online research done on 3p panels, relegates any ability to go back to the old methods of random sampling. As the costs associated are ridiculous and the availability of truly random sample scarce.

While I do not agree that all research done on 3p Panels is totally invalidated by this fact, we do think that it carries inherent flaws. The proverb “better, cheaper, faster – choose two” fits quite well here, with “cheaper and faster” being the only options available.

If I were a PnG or a Unilever, I might be willing to accept the value-proposition: it’s less accurate and precise but it’s also significantly less expensive. When you have 1000’s of products to test and track, I imagine that value proposition remains agreeable. Sometimes, being close is enough (don’t quote me on that)

RE: The Stanford Study, 5 years ago, they ran 9 of the same survey, using two separate methodologies (CATI and Online), different incentives, at different times of the year, with different types of sampling methodology, quotas and no quotas, and they received significantly different responses, and response rates.

Amazing, they changed almost everything and almost everything changed.