Steve Portigal opens up a can of whoop-ass on a recent design school survey, and surveys in general as lazy substitutes for design research. We could not agree more; most of these things are ridiculously unscientific, flawed, and don't reveal anything that the designer didn't already know.
A plea to all design educators out there (and to students as well): please stop using crappy surveys as a substitute for actual research.
Survey design is a craft. If you haven't studied it, you don't know how to write a survey well, and the data you get is garbage. Surveys are quantitative tools. They require math to plan (what does your sample size need to be to ensure that your results are valid?) and to analyze (regression analysis (or any other buzzword) anyone?). They are very tough to write. Questions have to be worded correctly, and sequenced correctly.
Yet design instructors constantly send their students onto the Internet to "do research." Students spend about 30 seconds writing open-ended questions about their issues, and then blast the "survey" off to email lists populated by other designers. And so in the spirit of helping a good cause, people might respond. But the questions are vague, hard to answer, and not at all controlled.
Garbage in, garbage out.
Do read the whole post here, and enjoy.
Create a Core77 Account
Already have an account? Sign In
By creating a Core77 account you confirm that you accept the Terms of Use
Please enter your email and we will send an email to reset your password.
Comments
AGREE - Surveys need to be well written in a clear and concise manner.
AGREE - Good surveys take more than a day to write
AGREE - They shouldn't be used as a substitute for actual research.
DISAGREE - "Survey design is a craft. If you haven�t studied it, you don�t know how to write a survey well, and the data you get is garbage." Sure to extent writing a survey is a craft, but I don't think that you should discourage others from doing so just because they haven't studied the craft. I feel that to write a good survey you need to:
A: Understand the problem
B: Target the right people
C: Word the questions & format of the survey so that you extract the most useful information in the shortest amount of time.
I feel that if you can do the things listed above, write well and know how to use a spell checker there is no reason why someone can't put together a really good survey.
DISAGREE - "They require math to plan (what does your sample size need to be to ensure that your results are valid?) and to analyze (regression analysis (or any other buzzword) anyone?)"
What happens if you're researching a product that is used by relatively few people of whom you know, you don't have the time to put together a large list of people to survey, but know that essential information can be extracted from those few people? And what if you suck at math?
I surveyed 8 people for my major project last year and the information I got was invaluable to my project. I know this will be the case for many other up and coming ID students this year and in years to follow.
I feel this post is rather negative towards those who aren't trained in the art of survey writing and may discourage some from using a highly effective means of research.
Have a great day.
He's actually criticizing the lack of instruction in creating the surveys in design education. They're basically endorsing a research method without instructing the students on proper use of the method.