Telephone Surveys Versus Web Panel Surveys: The Practical and Scientific Perspectives

Telephone Surveys have long been the mainstay of marketing research methodologies, but Web Surveys are now overtaking Telephone Surveys globally in frequency of use. An important subset of Web Surveys is Web Panel Surveys for which panel participants have opted to participate in the panel. There are a large number of Web Panels available nationally and internationally. The number of panelists in many of these panels is now so large that they represent total populations of interest to researchers. While Telephone Surveys and Web Panel Surveys provide reliable and accurate data, each has its pros and cons.

The Practical Perspective

The standout advantages of Web Panels Surveys over Telephone Surveys are that they are typically less expensive and much faster to execute. Another important advantage of Web Panel Surveys is that the researcher can present a wide variety of stimulus materials such as websites, creative executions, and photos. Presentation of visuals is not typically an option for Telephone Surveys. The interviewer bias sometimes introduced during Telephone Surveys (e.g., if they misinterpret what a respondent is saying and enter it into the database incorrectly) is absent from Web Panel Surveys as respondents type in their own responses. Another important fact about Web Panel Surveys is that they significantly reduce interviewer or respondent reticence to discuss sensitive topics (e.g., sexual habits, use of control substances or chronic diseases).

Both Telephone Surveys and Web Panel Surveys allow access to a wide geographic range of respondents; however, due to the availability of listed telephone directories and companies specializing in random digit dial sample, telephone sample is readily available even for small regions such as a county. As there are literally hundreds of panels available and not one main source of panel samples, it is sometimes difficult to obtain panel samples for small areas. This said, the larger the panel with which a researcher is working the more likely the availability of samples. The required makeup of the sample for a panel is also critical. For example Q2 Insights was recently called upon conduct a Web Panel Survey in four counties in Chicago only to find that not only was insufficient sample available but also that the sample was predominantly female.

The Scientific Perspective

From a purely sampling point-of-view, both Web Panel Surveys and Telephone Surveys are somewhat flawed; however, there is evidence to suggest that Web Panel Surveys are equal or superior to Telephone Surveys in terms of data reliability.

One of the key objections to Web Panel Surveys from a statistical sampling perspective is that they do not allow for a random probability design. In the case of a true probability sample, the sample drawn from the population under study allows every person in the population an equal chance (greater than zero) of being selected in the sample. By definition Web Panel Surveys allow for convenience samples or non-probability samples, where some people in the population under study have no chance of selection because they are not a participant in the panel.

Many argue that Telephone Surveys allow for true probability samples as they are typically recruited from lists of people that represent a target population, and each person has an equal probability of being selected. However, it is easy to shoot holes in this argument for a variety of reasons, including, but not limited to the fact that not everyone has a listed telephone number (particularly with more and more people moving to cell phones rather than landlines), many people are included on the national “Do Not Call” list, and many households employ gatekeeper technologies (e.g. Caller ID, answering machines). These consumer strategies not only increase the cost of doing Telephone Surveys but they also make some researchers question the extent to which the findings can actually be projected to the total population under study.

The reality is that the closest thing we have to true probability sampling is Door to Door sampling and even this approach has flaws. (But this is a topic for another blog post.)

The verdict is not yet out on which methodology is superior to the other in terms of sampling. A study conducted by the University of South Florida comparing a Web Panel Survey (in which respondents were randomly selected from a panel) versus a Telephone Survey that employed a “cold-calling” method to randomly select respondents found that Web Panels can produce more reliable data estimates than Telephone Surveys.

The Final Analysis

While neither Telephone Survey nor Web Panel Survey methodology is perfect, both provide a means of gathering high quality, reliable data. As the industry data suggests, Web Panel Surveys provide opportunities for interacting with various populations in ways that were not previously available with Telephone Surveys, and they do so quickly and relatively inexpensively. When the specifics of the project dictate, Web Panel Surveys are definitely a viable option when deciding which methodology is appropriate for your study.

This article was written by Kirsty Nunez.  Kirsty Nunez is President and Chief Research Strategist at Q2 Insights. Q2 Insights is a market research consulting firm with offices in San Diego and New Orleans. Kirsty can be reached at (760) 230-2950 and kirsty.nunez@q2insights.com. 

This entry was posted in Tools and Techniques and tagged Tags: Survey Research, Telephone Surveys, Telephone Surveys Versus Web Panel Surveys, Web Panel Surveys on February 24, 2012 by Kirsty Nunez