DUTCH DESIGN WEEK 2009
PRAGUE DESIGN DAYS 2009
LONDON DESIGN FESTIVAL 2009
GIZMODO GALLERY 2009
EUROBIKE 2009
MAKER FAIRE AFRICA
NEW YORK INTERNATIONAL GIFT FAIR
ISTANBUL DESIGN WEEKEND 2009
BAUHAUS SUMMER SCHOOL 2009
Introduction
Market research is expensive—not only for the time and money required to execute it, but also for the opportunity cost it represents to the organization sponsoring it. Each dollar or hour given to a project draws on a company's budget, chipping away at the financial and human resources available to direct to other initiatives. For this reason, organizations must be deliberate in choosing both the projects they pursue and the methods they leverage to solicit customer insight. Data gained is valuable only if it's used to inform an organization's development and marketing initiatives.
A quick note about language first. In the title and throughout this article, I've used the phrase "market research" versus "design research" for two reasons: First, "Design research," a term invented by the design community, is not recognized or known outside of this group; the term recognized by other individuals in business (and as an established profession) is "market research." I'm using this term then, as I believe it has more universal appeal and understanding. Second, when the design community refers to "design research," traditional methods such as focus groups and surveys are often dismissed, where more emerging methods like ethnographic research and listening labs get all the ink (or pixels). I'd like to help balance that out.
Ultimately, I'd like this article to act as a resource to help product managers determine how best to stretch dollars available for market research in these tough economic times. And while I can't possibly profile all research methodologies available for soliciting customer insight, I will take on the four—ethnographic research, listening labs, focus groups, and surveys. In each, I will discuss the method's strength and value to the product development process, cost, and timeframes.
Ethnographic Research
Use: Offers direction for strategy and product development
Cost: $$$
Average time: 3.5 months
Ethnographic research is an emerging, qualitative research method and, quite possibly, the least understood. In its purest form, ethnographic research is about entering the natural habitats of your users, seeking to understand through participation, listening, and observation the behaviors, values and motivations of a culture. When used appropriately in a business context, this methodology can be very powerful in providing direction for strategic planning and product development initiatives. Yet, the continued bastardization of the method in applied settings has qualified practitioners of the approach continually backtracking—trying to validate its strength and value—as the price tag for ethnographic research projects tends to be higher than that for your average focus group.
Contrary to popular belief, ethnography is not just about doing an interview in someone's home. Nor is it about taking a video camera into a person's work and, voilà, you now have all the contextual data required to successfully develop your new product.
In order to reap the maximum rewards possible from ethnographic research, you first need to exploit being in the user's natural environment (i.e. home, work place, or place of leisure). You need to immerse yourself in the details, from the step-by-step tasks performed to accomplish the goals of an activity, (like "preparing dinner") to noticing areas of people's homes where they've channeled their earnings. Attention to detail is what will give insight into what your users value and aspire to. This, coupled with what they tell you, helps create a holistic understanding of who your customers are, and consequently, how your offering might need to align with their needs, behavior patterns and values.
Additionally, sufficient time for analysis needs to be given to ethnographic-based initiatives. Fieldwork is a bit messy, as the protocols for ethnographic research are largely non-directed and unscripted. So it's only in the analysis of that data that you uncover the real nuggets of information that offer the AHA! and new insight into who your customers are. Expecting recommendations to be delivered immediately after fieldwork is complete is not only unrealistic, but if acted on, seriously compromises the depth of insight yielded.
To recap, ethnographic research is best leveraged in product development, when needing to:
- Define both articulated and unarticulated needs of your user
- Get an up close and personal look at your user's context—the environment in which they'll use your product and the meaning that it might offer to their lives
- Uncover micro-trends in customer behavior that could impact the market's acceptance of your product
- Learn the differences between what people say they do versus what they actually do
User insight gained from ethnographic research can be very powerful in:
- Identifying new growth opportunities for your organization
- Defining target markets for new offerings
- Developing "design criteria" to scope and prototype the development of a new product or service
Listening Labs
Use: Assess the customer experience provided by an offering
Cost: $$
Average time: 2 months
Listening labs (or product walkthroughs) are another emerging, qualitative method that has proven to be useful in product and service development. This method works best when leveraged to understand the customer experience that one has with an offering.
Oftentimes, this approach is confused with usability research. Yet, there is stark difference in how a usability test is conducted versus a listening lab. In a listening lab, specific tasks are not assigned to users, nor are the tasks time-based as they are in usability tests. Rather, the overarching goal of the service is stated, such as "book a hotel room" (in a reservation system). The learning is gained through seeing the non-directed paths that users take to accomplish that goal and hearing how the customer experience at each stage of the process aligns or is in dissonance with their expectations.
Listening labs are an excellent methodology to leverage once functional prototypes of your product or service exist. To be of most value within the product development process, it's important to leverage this method early and often. When used only as a "last check" technique—as is often the case—the true rewards that can yield from this approach are short-changed. The focused feedback gained in this methodology has the power to transform an experience created around an offering, often making the difference between an experience that is perceived as just being "OK" to one that people talk about positively amongst their peers.
To recap, listening labs are best leveraged in the product development process to understand the experience that your customer has interacting with your product or service. User insight gained through this method can inform the:
- Definition of a product or service's information architecture, features, form and/or function
- Development of "design criteria" to inform next stages of a product or service's development
Focus Groups
Use: Solicit targeted feedback on narrowly scoped questions
Cost: $
Average time: 2 months
Focus groups are the most traditional form of qualitative research leveraged by marketing professionals. Generally 3-10 people participate in a group, and a moderator uses a semi-structured protocol to solicit customer opinions on a topic.
If you know nothing about market research, you likely know what a focus group is and how it works. Because the terminology is so ubiquitous (and because it's a relatively cheap form of market research), this is one methodology that is overused and abused often.
Focus groups do not deliver a contextual, detailed understanding of reality. They aren't the methodology that you use to determine the next market for your organization to enter. Likewise, if you seek knowledge on what a "day in the life" of your target user is like, a focus group will fail you. Focus groups are best suited for addressing very specific and narrow questions, like "Which ad campaign is more attractive and why?"
When deciding if a focus group is the best method for you to use, you have to give some thought to the history of your offering. If it's a new product or service, then you need to question if you've done the due diligence to ensure that your product is fulfilling a customer need. You can't count on getting the reality check that you require regarding how this offering is going to integrate into a customer's life via a focus group alone. Per the name, a focus group is "focused," so you'll only capture insights on the specific questions you ask.
Yet, despite the disclaimers and cautions I've set forth, focus groups have a purpose and can be extremely useful and cost effective when used appropriately. The strengths of using focus groups in an offering's development are in:
- Answering non-hypothetical and narrowly scoped questions, such as "Which toothpaste packaging catches your attention most and why?"
- Gaining initial reactions and soliciting targeted feedback on a product, service, concept, advertisement or packaging idea
User insight gained through focus groups can inform the selection of one development path over another.
Surveys
Use: Quantify the opinions of your target market
Cost: $-$$
Average time: 1.5-2.5 months
Surveys offer the quantitative data (the "numbers") that so many people yearn for. They're great at getting to the specifics—like assessing the true size of a market opportunity—but are often misappropriated and leveraged in exploratory phases of development.
Like focus groups, surveys are best when leveraged to answer narrowly-focused questions. If you need to determine a pricing strategy for your offering, "surveys" are your method. The same holds true if you want customer satisfaction data or need "black and white" feedback on which package ("A" or "B") is most attractive.
From surveys, you can't expect to get an understanding of why; why someone answered this way or that. Surveys offer only a picture of "what is." So, when deciding if this is a suitable method for your initiative, you must assess your development requirements. Do you need to quantify how large an opportunity is or how many people (representative of your customer base) prefer one option to another? If your answer is "yes," then this is the method for you.
Surveys can be conducted online or in-person. Naturally, online surveys are less time consuming and more cost effective, as you can collect responses from multiple sources simultaneously and enjoy benefits of having responses automatically tabulated for you. But online surveys are only effective if you have a known set of answer variables, such as a Likert scale, where customers specify their level of agreement with a statement. In situations in which the answer choices might not be quite as concrete, or in cases where reaching your target audience via the Internet might not be possible, human-moderated surveys might be more appropriate. These are generally conducted in-person or via the telephone. An advantage of human-moderated surveys over online ones are that you get to clarify questions if they aren't sufficiently understood when first asked. Additionally, capturing "write in" data for open-ended questions is easier when facilitated. Still, the costs for conducting human-moderated surveys are much more expensive than online versions, so in most cases, this methodology is not used unless perceived to require one of the two advantages noted above.
To reiterate, surveys are best at getting to the specifics, and therefore their strengths in the product development process are the following:
- Soliciting customer feedback on narrowly focused topics for which an understanding of why is not needed
- Quantifying the prevalence of a need or attractiveness of a concept
User insight gained through surveys can inform the assessment of a market need, response or opportunity.
When to use each type of research in an offering's development?
To truly reap the benefits of market research, you must know when each method will be most productive—most informative—in the product's/ service's development process. Generally speaking, qualitative data, such as that garnered from ethnographic research, listening labs or focus groups, is best for giving direction on an offering's development, while quantitative data, gathered through surveys, is best for calculating the prevalence of a need or issue across a representative sample of your customer base.
The table below offers some guidance on when it's appropriate to use each of the methods shared in this article across the early stages of the new product development (NPD) process.
At any one stage of a product or service's development one or more methods can be leveraged. The selection of which methods are chosen depends on the decisions that the research is intended to inform.
As a final note, market research is best when designed from the perspective of informing the next stages of an offerings development. If used appropriately, it can help a company mitigate risk and reduce development costs, as decisions are based on the facts of what your intended users need, as opposed to what you perceive might be in their best interests. All of which makes you wiser and better positioned to compete in the marketplace.
Brianna Sylver is the founder of Sylver Consulting, an international research and innovation firm. She's also an adjunct faculty professor at the Institute of Design, IIT in Chicago teaching courses in human-centered communication design, early prototyping, and persona-driven system development.
Dutch Design Week
Prague Design Days
1 Hour Design Challenge Winners!
Coroflot Salary Survey Results
Comments
How useful, in your opinion, is an application of "listening labs" methodology to monitoring customers feedback on the Internet? There are growing volumes of such raw content available for some qualitative analysis, and very interesting technologies to produce meaningful and actionable reports.
Hi Gregory,
I'd agree with you that it's useful and beneficial to analyze the volumes of data available on the Internet. Yet, I don't believe listening labs are the appropriate methodology to leverage in this case. You'd be better off exploring methodologies of virtual ethnography or web mining for an exercise like the one you're speaking about above.
Hope this helps!
Brianna
I struggle with the authors understanding of what design research is and the context in which it is carried out. I understand that some design research is geared towards developing competitive products to fuel consumption. However academic design research conducted mainly in universities helps define methods for dealing with design problems. It can provide business with new insights and opportunities not revealed by other research methods. I would like to critique the claim that �simple methods are dismissed� in design research. In my PhD research I�m using some of these simple methods. However, my approach perhaps differs from that of market research in that it needs to be methodologically and ethically sound. Also I don�t think Ethnography�s that new an approach. Modern ethnographic approaches date back about 100 years.
Ms. Sylver,
Your ideas about the term design research are misinformed. It is true that this term is more widely used within the design community than outside, but it is more widely recognized than you realize. It is common among universities across many fields and among government research and funding agencies. I can understand that this might not be visible to you: you are a consultant selling services to industry, and North American industry is often ill-informed about many aspects of design, including design research. To measure the degree to which a term addressing research issues is understood, you've got to ask in a research environment.
Market research is not synonymous with design research. There are many aspects of design research -- including ergonomics, materials, systems analysis, user studies, logistics, and a couple of hundred more -- that people trained in market research cannot do because they involve issues that they don't deal with in market research.
Your description of these aspects of what you call market research is also inadequate. For example, ethnography is not a form of market research a branch of anthropology. It has a rich array of research methods and tools suitable for many purposes. Some of these tools can be applied to market research, just as these tools can be used in medicine, geography, political studies, and more. While your firm specializes in what you label "design enthnography," most enthnographers study anthropology and the social and behavioral sciences to master the skills of enthnographic research. A fully trained enthnographer normally completes her education with a PhD, along with whatever skills she or he may master in design. No enthnographer would describe ethnography as a branch of market research. This confuses a scientific and professional discipline with one specific application for which specific ethnographic methods are suited.
Ken Friedman, PhD, DSc (hc), FDRS
Professor
Dean
Swinburne Design
Swinburne University of Technology
Melbourne, Australia
The term, 'Design Research' was never intended to replace 'Marketing Research'. In academic circles it refers to the study of design history but in practice it includes anything that would be helpful for the designer.
Designers usually source such "market research" directly from your "established profession". But we are increasingly forced to find alternatives when this data becomes compromised.
These days marketing firms claim to be doing a survey on something like radio station preference when they are actually just creating a database of incomes and addresses, which they sell to predative lenders.
It's unfair to expect designers to trust any data collected by people with no scientific or professional integrity when there are people like Hans Rosling who can actually show us the writing on the wall.
Brianna,
Nice overview on the pros & cons of research methodologies. Our office has reaped a great deal of actionable, consumer insights via ethno research, so I was confused at your statement regarding "the continued bastardization of the method in applied settings." Can you further explain, what you mean here?
Thanks, David
Thanks for the article. Excellent, concise summaries of some of the methods we have at our disposal. However, I disagree with your premise that Design Research (DR) is just a rebranding by the design community of Market Research (MR). In my opinion they are two different, albeit overlapping, disciplines. At the most basic level, DR is focused on the product design while MR is focused on quantifying market potential. I believe DR should always provide answers to the questions: what should we design, what features/benefits/experiences should it provide, and how should people interact with it. On the other hand MR should answer: how big is the pool of potential customers, who are they, where are they, and what product will they buy?
Using your table, I believe that stages 1,2, and 3 are within the scope of DR, while 3, 4, and 5 fade into MR (overlap is intentional). Another differentiating factor is the training of practitioners. As a Design Researcher, my background is ID, Psychology, and Human Factors. My job is to make sure our clients design the "right" product, and that it fits the needs, expectations, and abilities of the user. I would expect a Market Researcher to have training more focused in Business, Marketing, and Statistics. Their job is to make sure there is a market that will buy the product, and to craft a marketing strategy to communicate with that group of people.
Both are important and we share a lot of methods, but personally I believe there are important differences between the two. DR is considered an "emerging discipline". People have been doing it naturally for a long time, but only recently have we gained ground in formalizing our role and methodologies.
Again, thanks for the interesting article!
I won't reiterate Ken Friedman and other comments here that are spot on.
In practice, most market research investment is used to help disparate groups within large corporations on the same page about the customer. The data is analyzed to find consistencies across the target subjects. The exercise is about looking for norms. This is how we can witness large budgets dedicated to uncovering what is often obvious to anyone, whether that is a designer or the man on the street. I have heard of designers dumbfounded by how their companies will pay millions to find out facts like "teenage girls like to stay connected and will use technology to do this" or "people like to be able to find their size when they go into a retail store," but the same companies pinch pennies by cutting out human factors research, concept refinement phases, or even trips for manufacturing oversight and builds offshore. The companies want facts that can be scientifically "proven," and often -- not always -- end up finding out about stuff that is provable but unremarkable. They cut corners elsewhere.
Design research tends to be more affordable, even at its most ridiculous. Designers tend to be people who understand and empathize with their audience. They do not aspire to be stand off and sterile-ly scientific. Often design research is about immersing the design team in the life of the people who they are designing for. They are often more interested in understanding peculiarities than finding norms because therein lies insight into creating something differentiated for their firms. At times, it could even be equated with how an actor understands a role. Once the quirks and foibles, motivations and meanings, are ascertained, the designer understands how the target customer would respond to a new situation (product, experience, Web site).
Except for evolutionary products (the "next flavor of oatmeal," by friend Eric would call them), market research rarely informs innovative design. Design research spews it.
Market researchers will moan and argue against this, but there are very few approaches in their world that preserve and study the anomalies. They do tend to win the attention and budget of management because they are fact based, and the bigger the norm they find, the easier it is for management to buy in. They don't tend to be the types of people who will come up with a genuinely breakthrough concept (there are exceptions, surely).
Market research should inform direction on the front end. Segmentation work, when done well, is a case in point. Business models need to be based on facts and norms.
Design research should inform new product development. It's the quirks of a target customer, maybe a first adopter, not the norms of the masses, that inform new, differentiated products. Design research can be as simple as hiring great designers who are also good at immersing themselves.
Business Week's number 1 most innovative company is Apple this year and last. Interesting that my research indicates Apple does not do any (zero) market research to inform new product development. They do seem to have some pretty awesome designers.
Cheers-
Lori H.
I knew that my statement that "design research" was a term invented by the design community (and one not easily recognized by others outside this community) would provoke debate, especially on the Core77 site. But, honestly, some of the comments went in a direction I never expected.
First things first, in my understanding, two types of "design research" currently exist. The first is academic-based "design research," which is about developing new methodologies or answering a research question. The second is applied "design research," which has been adopted by the design community and refers to activities such as user research, user experience design, observational research, and ethnography as it's used in the product development process.
I can share from previous conversations that I've had with academics on this topic that they're a bit irked that the design community has picked up the name "design research" and ran with it. From these conversations I've understood that academic "design research" is about planning and executing a course of study that results in something that advances the knowledge of that profession, like a new method. The academics I've spoken with don't believe that going into peoples' homes to understand their food storage habits, for instance, reflects the rigor that they generally attach to the academic definition of "design research."
So, when I wrote the "Spend it Wisely" article and spoke of "design research" I wasn't referring to academic "design research," but to the form of "design research" that has been adopted by professionals, such as designers, human factor specialists, etc. to design new products and services. Should this have been stated in the article more clearly? Probably, but it would have taken more space and further delayed getting to the point of that article, which is about how to choose the right method for the right level of customer insight required at the current stage of a product's development.
Now, onto my point that "design research" (as practiced by designers developing new products) is really a subset of "market research." At a high level, I think the intent of "design research" and "market research" is the same; to help a product be successful in the marketplace. By nature of this alone, I think "design research" and "market research" are more related than the design community would like to believe.
I also believe that designers have a history of separating themselves from the group, which more times than not works against them. I believe this adamant stance that "design research" is not a type of "market research" is just another example of how designers have left opportunity on the table.
There is an arm of "market research" out there that addresses more of the business questions associated with how to bring a product to market. In this research the focus is on scoping the opportunity, segmenting the target market and determining in which markets and at what price point to introduce a new product or service into the marketplace. But there is another, equally as prominent, arm of "market research" that's more customer focused and is about helping companies to determine the features and functions that should be embedded in their new product or to identify the needs of their customer base. And it's common that it's the "market research" companies, not the "design research" companies, which are called upon to do that work, leveraging focus groups to obtain that customer understanding.
Is a focus group the best method to use to define the new functions and features of a product? In most cases, the answer would be "no" (as my article clearly states). But market research companies are leveraging focus groups every day to answer that exact question because the designers who are using the methodologies, such as "ethnography" or "listening labs," that really can answer those questions are making themselves so hard to be found because they insist on calling their work "design research."
I'm unable to comment about design versus market research. But as someone who helps rather well-respected companies with product, design & marketing innovation I have to disagree with the use of consumer research in idea generation. At PSFK we believe that consumers can only help you optimize existing products and services but not ideate for the future.
I also have a concern about the length of time you suggest each each type of research should take. At PSFK we research and provide concepts based on that research within 4 weeks. If you're sitting around for 2 to 3 months talking to people, you'll be left behind by the rapid change that is occurring every single day.
Interesting article, although I do agree with the majority of commenters that design research is ultimately a different field than market research.
For me, I think the major difference between design and market research is that design research is there to serve the user, their needs, wants and experiences. Ultimately, this will obviously benefit companies too. In contrast, market research is there to inform business opportunities, position and direction. It takes much less consideration into user needs.
This is a really nice overview. I think the difference between design-driven research and other kinds of market research is that you can break up certain kinds of products with features into their parts and weight each factor's relative level of importance. This can feed a quantitative conjoint analysis that allows you to price goods and services with a relatively high level of precision. That is where pure market research fits. Design and aesthetic components however are not easily disaggregated.
Design-driven products are usually purchased with the impression they make on someone else in mind. Since that impression is not known at the time of purchase, it's very tough to assign any kind of empirical value to the product since the product is in itself a 'bet' on the part of the consumer. Designed products allow consumers to maintain and grow their influence. in the eyes of others. The more of a sure thing the bet is, the more the product is worth. That is very different from feature driven products where the consumer knows exactly what he or she needs and wants out of the product they purchase.
Our company, hertaste, polls the people the end user is trying to impress and we avoid any kind of feature driven product categories. That's why we assembled the worlds largest panel of attractive women to comment on luxury products. We would address the car exterior and styling, but not how much horsepower is enough. What's been monumentally difficult is figuring out which groups of people the end user is trying to impress and which people's opinions are ignored. We've used something similar to a supercomputer to measure taste and influence and are after two years just starting to be able to quantify these metrics.
Strangely, the people who claim to have the most influence or are closest to the trends often have the least influence on the most active buyers. Figuring out how to screen for the right influencers is tough and most of them have better things to do with their time than participate in focus groups.
In summary, none of that complexity is necessary to forecast demand, price, or segment the market for a feature driven product so I'd say the two types of research truly are very different.
m zabel is spot on. Design Research = what features people like, and Market Research = what features people will pay for. Using only the term MR is too vague and broad and is unclear about which problem you are trying to solve. Usually both. If a designer runs the study it would generally be misleading to call it MR. If Marketing specs features from an MR study you generally miss the expertise of a designer's creativity in interpreting the user needs. Of course DR and MR overlap, and the vast majority of designers have no problem working side by side with Marketing, in which case the name of the research generally defaults to MR. Whatever. Sounds like you have a personal grudge against designers. By dismissing DR I believe you are the one leaving a valuable opportunity, and colleague, at the table.