The proliferation of cheap, high-quality online survey tools has revolutionized our ability to conduct surveys to obtain quick snapshots of what our members are thinking. For all but the most complex projects, it is possible to begin and conclude a well-defined study of a subject of interest with the participation of a representative group of members within a 10-day timeframe. Perhaps more than any other development, this technology has made it possible for us to become "data driven" associations, as was put forth in 7 Measures of Success: What Remarkable Associations Do That Others Don't.

At the same time, we need to balance this easy source of quantitative data with a similar easy source of qualitative data. Why? Survey tools do a great job of providing us with definitive numbers and visuals to help tell our story, to ensure that our colleagues or committees understand and buy into our key findings. Databases also provide much better snapshots in the form of statistical reports and charts to help us document usage and sales baselines and trends. But to obtain greater insight into what actually determines these levels and drives changes, we often need to go deeper and directly engage representative members and customers in two-way dialogue. 

Structure and Method for Member Interviews

There are entire books regarding methodology for qualitative interviews, but as associations we often benefit by simplifying them considerably. Here are some suggestions:

Conduct only telephone interviews (rather than face-to-face), prescheduling from a small random sample of the members or other targeted constituency. This allows them to speak at their convenience, although speaking to them at home or on a cellphone undermines the quality of conversation.

Prepare a discussion guide in advance. Rather than treat this as an agenda or survey form, keep it broad and flexible. After all, an interview is two-way communication. The majority of each interview will probably consist of follow-up questions to probe initial responses more deeply.

Use (or be) good interviewers. Effective interviewers (and facilitators) are friendly and open, and they know how to probe effectively. Through active listening, surface level discussions rapidly give way to deeper motivations, and if the interviewer can demonstrate objectivity and candor, he or she can quickly establish a trusting relationship in the interview. You should welcome digressions, and don't worry if every interview is unique. The end product of aggregating all the interviews will be far more robust as a result.

Guarantee confidentiality. Ensure participants that no individual information or attribution will be released to others in transcripts or written reports.

Allow interviews to run long. Even with the shortest guides and most focused of objectives, we often find that interviews run 30 minutes or more. Members rarely get a chance to speak directly with their association. A member who begins an interview emphasizing their time constraints inevitably is the one who will speak the longest.

Don't do too many interviews. Since time is money, structure the interviews as a discrete project, with a limited number of conversations. For any specific topic, we find most issues converge within 10 to 15 interviews—that is, we begin to hear repeated comments and similar thinking so we are not learning much new information from each new conversation. As with all qualitative research, we are generally not trying to establish or force a consensus; instead we want to hear the widest range of perspectives possible and understand why members feel that way.

Strengths of Interviews

Often we think of focus groups when considering qualitative research. Group dynamics are sometimes important to measure, and focus groups have also migrated online to a certain extent, but there are several reasons why in-depth interviews are superior.

Relative absence of bias. Interviews generally have less observer or participant bias. Even a trained moderator will encounter subtle bias in membership focus groups. In associations people often know each other, which can lead to conscious or subconscious posturing or suppression of some comments. Groups may seem to have homogeneous participants, yet some factor differentiates them once they are in the room. For example, we may find while discussing a service with which two people have had negative experiences that they are overly eager to share. If they speak first, it can undermine the perceptions of those who speak later and have had no experiences, or only positive ones. In interviews, the member is rarely trying to impress the interviewer except by trying to be as articulate and well understood as possible.

Built-in flexibility. Although you lose some rapport and communications through phone contact, it is far more cost-effective, allowing you to efficiently conduct interviews back-to-back and to give members who miss an appointment to call back at their convenience. Too often we are constrained in focus groups by having members gathered at a conference or in their local area, which yields a sampling of only our most motivated “super-users” and cognoscenti, or group dynamics reflecting participants who are very familiar with one another.

An interview is two-way communication. The majority of each interview will probably consist of follow-up questions to probe initial responses more deeply.

"Feed" your survey. Often we design surveys based on our assumptions regarding what matters, drawn from internal management perspectives, questions from the last survey, or good ideas a consultant brought in. However, it is harder to get a candid take on current issues that are of greatest concern to members. Conducting interviews as part of the process of designing the survey helps provide timely, titillating observations, unproven hypotheses, and possible hidden connections between attitudes and behavior that you will want to quantify in the survey work.

Enough talk time for members. A 90-minute focus group allows each participant to speak perhaps eight to 10 minutes. Online surveys generally take between five and 15 minutes. Many members may only have only a few minutes of thought to share, but for subjects that do warrant more in-depth discussion and a clear understanding of their background, a 20- to 30-minute period for one person's feedback is more appropriate. Interviews lose a group dynamic, but they also spare interviewees from spending time listening to others—helpful particularly if your members have type A personalities and tend to equate "listening" with "waiting to speak again." (Yes, we all have many of them in our databases!)

Candor and intimacy. Even if you have never spoken to members regarding their inner feelings, don't worry—they will make it easy for you. Often members are flattered to be asked. They make the time to speak with you and they reward you with candor. Sometimes you may not like what you hear, but the more the interviewer plays the role of objective outsider, the better the process will be. As a market research director, I often introduced myself as "acting as an independent researcher today" and that's often all you need in order to pull yourself out of the equation and to put the focus of conversation where it belongs—on the member or customer you're interviewing.

Low-cost and easy interpretation. Even surveys that are easy to administer online require some statistical knowledge to properly interpret. To conduct and analyze, interviews require a finger to dial, an ear to listen, a telephone, and a keyboard or notepad. Like surveys today, interviews can launch in real time, and it is easy to share top-line reports in a day for time-sensitive projects.

Weaknesses of Interviews

Of course, interviews also have inherent weaknesses. These are a few of their  limitations:

Missing objectivity. There is a potential for observer bias in just about all qualitative research. If the people conducting the interviews are staff or service providers who can't maintain a strong sense of objectivity inside and out, the interviewee will pull their punches and not tell the whole truth, or the interpretation of the end results starts to resemble a process of hearing what you want to hear. Be on the lookout for what can be an almost subliminal bias.

Negative reactions. I often found that associations need to be prepared to accept what they hear. Not all of it is pleasant. The kneejerk reaction to negative feedback often can be outright rejection—a belief that the method just wasn't reliable enough. This may be true, but it is important to balance a sudden keen interest in valid methodology with an urgent need to cover one's backside. We are often politically sensitive and very PC, and when interviewees take advantage of glasnost to say exactly what they feel, it can be jarring. Sometimes you will need to smooth off the rough edges and edit the unadulterated stream of feedback, unless you are a big fan of Impromptu Job Loss or like being perceived as a traitor when you're only the messenger.

Open-endedness. Digressions and lack of standardization across interviews can be a good or a bad thing. When you try to make interviews "sum up" to a consensus or quantify them, you'll be disappointed. To push for consensus is to force interviews to do something they don't do well. It is best to accept this limitation, even to the point of managing your interviewee's expectations upfront. Sometimes an interviewee will refer to our "phone survey" and we gently correct them, since survey implies a rigid format. Interviews often yield digressions into arcane specialties, heretical opinions, conspiracy theories, and wildly inventive suggestions that each represent a unique viewpoint.

Subject to these caveats, qualitative interviews can be a valuable tool to help inform most association problems. Like some people we know, the feedback we receive may be amorphous, messy, and sometimes contradictory. However, regular use of the method can improve member and customer relations and provide a critical additional source of intelligence that we rarely obtain otherwise.

Editor’s Note: This article, originally published in 2009, has been updated.

Kevin Whorton is principal of Whorton Marketing & Research in Silver Spring, Maryland.

Email: info@kwhorton.com