Market research is expensive—not only for the time and money required to execute it, but also for the opportunity cost it represents to the organization sponsoring it. Each dollar or hour given to a project draws on a company's budget, chipping away at the financial and human resources available to direct to other initiatives. For this reason, organizations must be deliberate in choosing both the projects they pursue and the methods they leverage to solicit customer insight. Data gained is valuable only if it's used to inform an organization's development and marketing initiatives.
A quick note about language first. In the title and throughout this article, I've used the phrase "market research" versus "design research" for two reasons: First, "Design research," a term invented by the design community, is not recognized or known outside of this group; the term recognized by other individuals in business (and as an established profession) is "market research." I'm using this term then, as I believe it has more universal appeal and understanding. Second, when the design community refers to "design research," traditional methods such as focus groups and surveys are often dismissed, where more emerging methods like ethnographic research and listening labs get all the ink (or pixels). I'd like to help balance that out.
Ultimately, I'd like this article to act as a resource to help product managers determine how best to stretch dollars available for market research in these tough economic times. And while I can't possibly profile all research methodologies available for soliciting customer insight, I will take on the four—ethnographic research, listening labs, focus groups, and surveys. In each, I will discuss the method's strength and value to the product development process, cost, and timeframes.
Use: Offers direction for strategy and product development
Average time: 3.5 months
Ethnographic research is an emerging, qualitative research method and, quite possibly, the least understood. In its purest form, ethnographic research is about entering the natural habitats of your users, seeking to understand through participation, listening, and observation the behaviors, values and motivations of a culture. When used appropriately in a business context, this methodology can be very powerful in providing direction for strategic planning and product development initiatives. Yet, the continued bastardization of the method in applied settings has qualified practitioners of the approach continually backtracking—trying to validate its strength and value—as the price tag for ethnographic research projects tends to be higher than that for your average focus group.
Contrary to popular belief, ethnography is not just about doing an interview in someone's home. Nor is it about taking a video camera into a person's work and, voilà, you now have all the contextual data required to successfully develop your new product.
In order to reap the maximum rewards possible from ethnographic research, you first need to exploit being in the user's natural environment (i.e. home, work place, or place of leisure). You need to immerse yourself in the details, from the step-by-step tasks performed to accomplish the goals of an activity, (like "preparing dinner") to noticing areas of people's homes where they've channeled their earnings. Attention to detail is what will give insight into what your users value and aspire to. This, coupled with what they tell you, helps create a holistic understanding of who your customers are, and consequently, how your offering might need to align with their needs, behavior patterns and values.
Additionally, sufficient time for analysis needs to be given to ethnographic-based initiatives. Fieldwork is a bit messy, as the protocols for ethnographic research are largely non-directed and unscripted. So it's only in the analysis of that data that you uncover the real nuggets of information that offer the AHA! and new insight into who your customers are. Expecting recommendations to be delivered immediately after fieldwork is complete is not only unrealistic, but if acted on, seriously compromises the depth of insight yielded.
To recap, ethnographic research is best leveraged in product development, when needing to:
- Define both articulated and unarticulated needs of your user
- Get an up close and personal look at your user's context—the environment in which they'll use your product and the meaning that it might offer to their lives
- Uncover micro-trends in customer behavior that could impact the market's acceptance of your product
- Learn the differences between what people say they do versus what they actually do
User insight gained from ethnographic research can be very powerful in:
- Identifying new growth opportunities for your organization
- Defining target markets for new offerings
- Developing "design criteria" to scope and prototype the development of a new product or service
Use: Assess the customer experience provided by an offering
Average time: 2 months
Listening labs (or product walkthroughs) are another emerging, qualitative method that has proven to be useful in product and service development. This method works best when leveraged to understand the customer experience that one has with an offering.
Oftentimes, this approach is confused with usability research. Yet, there is stark difference in how a usability test is conducted versus a listening lab. In a listening lab, specific tasks are not assigned to users, nor are the tasks time-based as they are in usability tests. Rather, the overarching goal of the service is stated, such as "book a hotel room" (in a reservation system). The learning is gained through seeing the non-directed paths that users take to accomplish that goal and hearing how the customer experience at each stage of the process aligns or is in dissonance with their expectations.
Listening labs are an excellent methodology to leverage once functional prototypes of your product or service exist. To be of most value within the product development process, it's important to leverage this method early and often. When used only as a "last check" technique—as is often the case—the true rewards that can yield from this approach are short-changed. The focused feedback gained in this methodology has the power to transform an experience created around an offering, often making the difference between an experience that is perceived as just being "OK" to one that people talk about positively amongst their peers.
To recap, listening labs are best leveraged in the product development process to understand the experience that your customer has interacting with your product or service. User insight gained through this method can inform the:
- Definition of a product or service's information architecture, features, form and/or function
- Development of "design criteria" to inform next stages of a product or service's development
Use: Solicit targeted feedback on narrowly scoped questions
Average time: 2 months
Focus groups are the most traditional form of qualitative research leveraged by marketing professionals. Generally 3-10 people participate in a group, and a moderator uses a semi-structured protocol to solicit customer opinions on a topic.
If you know nothing about market research, you likely know what a focus group is and how it works. Because the terminology is so ubiquitous (and because it's a relatively cheap form of market research), this is one methodology that is overused and abused often.
Focus groups do not deliver a contextual, detailed understanding of reality. They aren't the methodology that you use to determine the next market for your organization to enter. Likewise, if you seek knowledge on what a "day in the life" of your target user is like, a focus group will fail you. Focus groups are best suited for addressing very specific and narrow questions, like "Which ad campaign is more attractive and why?"
When deciding if a focus group is the best method for you to use, you have to give some thought to the history of your offering. If it's a new product or service, then you need to question if you've done the due diligence to ensure that your product is fulfilling a customer need. You can't count on getting the reality check that you require regarding how this offering is going to integrate into a customer's life via a focus group alone. Per the name, a focus group is "focused," so you'll only capture insights on the specific questions you ask.
Yet, despite the disclaimers and cautions I've set forth, focus groups have a purpose and can be extremely useful and cost effective when used appropriately. The strengths of using focus groups in an offering's development are in:
- Answering non-hypothetical and narrowly scoped questions, such as "Which toothpaste packaging catches your attention most and why?"
- Gaining initial reactions and soliciting targeted feedback on a product, service, concept, advertisement or packaging idea
User insight gained through focus groups can inform the selection of one development path over another.
Use: Quantify the opinions of your target market
Average time: 1.5-2.5 months
Surveys offer the quantitative data (the "numbers") that so many people yearn for. They're great at getting to the specifics—like assessing the true size of a market opportunity—but are often misappropriated and leveraged in exploratory phases of development.
Like focus groups, surveys are best when leveraged to answer narrowly-focused questions. If you need to determine a pricing strategy for your offering, "surveys" are your method. The same holds true if you want customer satisfaction data or need "black and white" feedback on which package ("A" or "B") is most attractive.
From surveys, you can't expect to get an understanding of why; why someone answered this way or that. Surveys offer only a picture of "what is." So, when deciding if this is a suitable method for your initiative, you must assess your development requirements. Do you need to quantify how large an opportunity is or how many people (representative of your customer base) prefer one option to another? If your answer is "yes," then this is the method for you.
Surveys can be conducted online or in-person. Naturally, online surveys are less time consuming and more cost effective, as you can collect responses from multiple sources simultaneously and enjoy benefits of having responses automatically tabulated for you. But online surveys are only effective if you have a known set of answer variables, such as a Likert scale, where customers specify their level of agreement with a statement. In situations in which the answer choices might not be quite as concrete, or in cases where reaching your target audience via the Internet might not be possible, human-moderated surveys might be more appropriate. These are generally conducted in-person or via the telephone. An advantage of human-moderated surveys over online ones are that you get to clarify questions if they aren't sufficiently understood when first asked. Additionally, capturing "write in" data for open-ended questions is easier when facilitated. Still, the costs for conducting human-moderated surveys are much more expensive than online versions, so in most cases, this methodology is not used unless perceived to require one of the two advantages noted above.
To reiterate, surveys are best at getting to the specifics, and therefore their strengths in the product development process are the following:
- Soliciting customer feedback on narrowly focused topics for which an understanding of why is not needed
- Quantifying the prevalence of a need or attractiveness of a concept
User insight gained through surveys can inform the assessment of a market need, response or opportunity.
When to use each type of research in an offering's development?
To truly reap the benefits of market research, you must know when each method will be most productive—most informative—in the product's/ service's development process. Generally speaking, qualitative data, such as that garnered from ethnographic research, listening labs or focus groups, is best for giving direction on an offering's development, while quantitative data, gathered through surveys, is best for calculating the prevalence of a need or issue across a representative sample of your customer base.
The table below offers some guidance on when it's appropriate to use each of the methods shared in this article across the early stages of the new product development (NPD) process.
At any one stage of a product or service's development one or more methods can be leveraged. The selection of which methods are chosen depends on the decisions that the research is intended to inform.
As a final note, market research is best when designed from the perspective of informing the next stages of an offerings development. If used appropriately, it can help a company mitigate risk and reduce development costs, as decisions are based on the facts of what your intended users need, as opposed to what you perceive might be in their best interests. All of which makes you wiser and better positioned to compete in the marketplace.
Brianna Sylver is the founder of Sylver Consulting, an international research and innovation firm. She's also an adjunct faculty professor at the Institute of Design, IIT in Chicago teaching courses in human-centered communication design, early prototyping, and persona-driven system development.