Ensuring the Right Online Sample Mix with Panel Blending
Online sample for survey research can be sourced from a variety of providers, including:
- Consumer access panels with members who were recruited to take surveys
- Online communities that have members who earn points for engaging in a variety of activities, including taking surveys, but were never expressly recruited to take surveys
- Providers who have access to members who joined social communities and social network sites
- “On the river” content site visitors who are willing to take surveys when an invitation is available
- Lists, databases, website intercepts, blogs and other resources.
In the June 2009 edition of Quirk's Marketing Research Review, GfK's Michael Fallig and Derek Allen explore how many sample providers, service bureaus and full service research firms are engaging in procedures to enhance the quality of their online respondent pools and/or study samples. They go on to offer an up-close look at an alternative approach to these methods that has been proven to deliver enhanced data quality -- panel blending.
According to Dr. Fallig and Dr. Allen, the current efforts to improve data quality in online research appear focused on eliminating “frauds,” duplicates, “satisficers” and the like. An underlining assumption is that with these issues eliminated, differences across online samples will be reduced and response patterns will be more believable.
Dr. Fallig and Dr. Allen argue that this current focus diminishes the myriad of individual differences found among members of a respondent pool and across different pools of respondents. Instead, they contend that these individual differences need to either be carefully controlled or randomly distributed within each sample that is drawn.
If done properly, they contend, blending sample from multiple pools can:
- Broaden coverage of the characteristics of the population at large
- Meet the needs of studies with unusually demanding characteristics (particularly continuous tracking studies and market level studies):
- Extremely low incidence of eligibility
- Large sample size requirements
- Very localized geographic requirements
- Lengthy past participation or study lock out requirements
- Reduce the risk of reporting or using an out of range response measure estimate for important decision making
But if online research programs can benefit from blending sampling, what are the risks involved? And, how can it be done without sacrificing data quality? In other words, how do you ensure you're doing it right?
