How It's Done: Survey Methods

November 1, 2005

Door to Door
Mailed Questionnaire
Consumer Panels
Mail Intercepts
Telephone Surveys
Online Surveys
Multiple Contact Design

1. Door to Door

This is a methodology rarely used these days except:

Among segments where phone penetration is low
Among some segments of newly arrived immigrants

These are especially important to remember when doing international consumer research with groups that have a distrust of authorities and surveillance.

2. Mailed Questionnaire

Mailed questionnaires generally involve sending out a huge number of questionnaires and hoping that a reasonable sample will mail them back.

Advantage:

Longer and more thoughtful answers are given because respondents fill out questionnaires at their own pace, it is possible to ask a relatively large number of questions - including some questions that require time for reflection.

Mailed questionnaires are inexpensive, as you pay only for printing, postage and information processing - not professional interviewers.

Disadvantages:

Only a 1 - 2% response rate. As such, the degree to which the returned sample is representative of the targeted segment is always in question.

3. Consumer Panels

Panel sponsors enlist participants by offering incentives - such as lotteries, cash, coupons and consumer items. These panels have been constructed to include as broad a representation of the American consumer public as possible

Traditionally, researchers accessed panels of respondents through written and phone interviews, although recently, panels based on internet communications have proliferated.

Advantages

The response rate is greater than would be possible with other study methodologies because respondents have previously signed up to receive questionnaires.

Respondents are willing to fill out more detailed questionnaires - more detailed than would be possible using a telephone survey - because they are voluntarily taking part in your study.

You can cross-reference responses with demographic data that the panel sponsors have already collected (e.g., HHI, hobbies). Further, in some panel situations, you can follow future purchases - to help you see if your communications strategy has influenced their purchase behavior. This could be useful because it provides you the "within subjects" analysis instead of the "across subjects" readings that one usually obtains by other interview methods.

Disadvantages

It is not clear that the a panel is representative of the American public, insofar as the participants are self-selected.

Panels rarely including very upscale or very downscale consumer segments. Panels are a better bet for more mainstream consumer segments.

4. Mall Intercepts

Mall Intercepts are interviews conducted with individual mall shoppers who are recruited as they shop. This is a widely-used quantitative design option.

Advantages

Simultaneous comparison of alternatives (e.g., positioning statements, logos, layouts). If you want respondents to rank the importance of variables or their approval of assorted options, it is best to allow them to compare alternatives simultaneously.

Rich diagnostic information can be collected from use of a visual presentation of commercials, print ads, and direct marketing pieces. In no other kind of quantitative design can you ask respondents to pay such careful attention to creative executions.

Disadvantages

The sample is limited to people are shopping and who can afford the time necessary to take part in a survey. As such, in conducting a Mall Intercept study, you need to be aware that your sample will be rather self-selected - and thus less reliably representative of the American public.

Every shopping mall has a restricted demographic profile. While all of the major mall research suppliers have an array of malls with various demographics, it may be that your target audience is of an age/income/geographic description that will make relying on Mall Intercepts a rather expensive affair.

Note: Your results are only as good as the people who conduct your interviews. Make sure that your research supplier has confidence in the organization that is conducting the interviews.

5. Telephone Surveys

In telephone surveys, interviewers contact respondents by telephone and ask them a series of questions. The sample for a telephone survey can be assembled either at random, through random-digit dialing (which can be targeted at particularly promising area codes and/or exchanges), or by a client-provided list.

Advantages

The closest method to a truly random (and thus projectable) sample, due to the ubiquitous nature of telephones.

Relatively easy to conduct in a short period of time, even though the increased popularity of telemarketing has suppressed response rates in recent years. (People feel bombarded by telephone calls and may be suspicious of callers, even if they make it clear that theirs is not a sales call).

Raw data collected is available in a short time thanks to the proliferation of computer assisted interviewing systems.

Disadvantages

Effective questionnaires must be short in length. People summoned to a telephone interview are rarely interested in staying on the phone for more than 8 to 10 minutes. If you push an interview much longer than that, most respondents will lose interest and will not put much effort into answering your questions.

Items cannon be put "side-by-side" for comparison. As a result, you are relying on short-term memory and must restrict in the number of attributes, features or options that you can expect respondents to meaningfully compare.

6. Online Surveys

As the proliferation of the internet continues, more and more clients are conducting surveys over the internet. There are a variety of ways of assembling a sample for an online survey:

Ask people who frequent your website to become registered "members" to gather information on your customers' addresses, demographics and sometimes brand-related attitudes and usage. These data can also serve as a sample for a survey that you may want to conduct.
Purchase a pop-up ad or a banner ad on a site which tends to attract category consumers to gather information on potential customers for your brand. Respondents to the ad would be qualified and interviewed at a website that you set up for the purpose.
Enlist the help of a consumer or executive panel.

Advantages

Respondents can evaluate visuals (packaging elements, logos, even commercials) in addition to the usual kinds of questionnaire items.

The net is still perceived as fun, so that respondents may enjoy taking part in the survey - a factor that will drive completion ratios.

Ideal to target the high-level business segment who are less willing to take the time for a phone survey.

Longer, more complicated questionnaires than would be advisable with a phone interview.

Disadvantages

Not everyone is on the Internet.

A random sample of Internet users is not a random sample of the public. An online approach may make sense only if the populations you wish to sample are educated or technologically sophisticated.

Respondents are very self-selected - they are already visitors to your website.

7. Multiple Contact Design

A Multiple Contact design is often used when to explore the perceptions of a difficult to interview segment, but need a (relatively) large sample quantitative data.

Multiple contacts are particularly useful when you:

Have only a small sample to choose from; you need to make every interview opportunity count.
Are interviewing relatively high-level professionals who are very pressed for time - so that you need to make the interview as convenient as possible in order to enlist their participation.

An example of a multiple contact design situations:

You are working with a client company for whom it is of utmost importance to maintain a salient position in the rather restricted consideration sets of its clients. As such, your client wants to understand category brand perceptions held by corporate executives who form a key segment of the client's customer base. Your client has a list of relevant executives, but it only contains 170 names. Normally, due to the small potential sample, you would propose a qualitative project - in particular, a series of executive interviews. However, your immediate client is being asked by the Senior VP of Marketing to come up with a numerical measures of their brand's category standing.

Procedure

Instead of calling potential respondents in the hope that they will be available to take part in the survey or mailing out questionnaires and hoping that respondent mail them back, Phone/Fax/Phone projects, as the name implies, consist of three stages:

Stage 1 - Elicit participation

Interviewers call prospective interviewees, present the project, explain why they are important to the success of the study and recruit them to take part in the study. In addition, interviewers will obtain a fax number or address to where they can send a blank questionnaire and, if possible, they will schedule a session when interviewers can call back to conduct a complete interview.

Stage 2 - Collecting responses

In the second stage, the interviewers send each respondent a fax copy of the project questionnaire. (If respondents prefer, questionnaires can be mailed). The respondent is given the option of simply filling out the faxed survey at her or his leisure and then faxing (or mailing) it back, or setting up a time when they can respond to the survey items on the phone with an interviewer.

Stage 3 - Follow-up

If respondents have not elected to mail or fax back the survey, the third stage consists of a series of telephone interviews.

Advantages

The best way to sample harried executives or devotees of a brand or product category. By politely working around potential respondent's schedules, you greatly increase the probability that they will take part in the study. The implied flattery goes a long way to insure participation.

Disadvantages

A Phone/Fax/Phone design is expensive. You are paying for up to three phone interviews per respondents. As such, this design should be used only when you must have quantitative data and only a restricted sample to work with.

Source

"How It's Done: A Research and Planning Handbook For Marketing Communications Professionals." Seth Ginsburg. New York: ANA, 2005.