DUTCH DESIGN WEEK 2009
PRAGUE DESIGN DAYS 2009
LONDON DESIGN FESTIVAL 2009
GIZMODO GALLERY 2009
EUROBIKE 2009
MAKER FAIRE AFRICA
NEW YORK INTERNATIONAL GIFT FAIR
ISTANBUL DESIGN WEEKEND 2009
BAUHAUS SUMMER SCHOOL 2009
Steve Portigal opens up a can of whoop-ass on a recent design school survey, and surveys in general as lazy substitutes for design research. We could not agree more; most of these things are ridiculously unscientific, flawed, and don't reveal anything that the designer didn't already know.
A plea to all design educators out there (and to students as well): please stop using crappy surveys as a substitute for actual research.Survey design is a craft. If you haven't studied it, you don't know how to write a survey well, and the data you get is garbage. Surveys are quantitative tools. They require math to plan (what does your sample size need to be to ensure that your results are valid?) and to analyze (regression analysis (or any other buzzword) anyone?). They are very tough to write. Questions have to be worded correctly, and sequenced correctly.
Yet design instructors constantly send their students onto the Internet to "do research." Students spend about 30 seconds writing open-ended questions about their issues, and then blast the "survey" off to email lists populated by other designers. And so in the spirit of helping a good cause, people might respond. But the questions are vague, hard to answer, and not at all controlled.
Garbage in, garbage out.
Do read the whole post here, and enjoy.
Dutch Design Week
Prague Design Days
1 Hour Design Challenge Winners!
Coroflot Salary Survey Results
Comments
It's all very well for an established design professional to "open up a can of whoop-ass" on student surveys. We need to start somewhere you must remember, this is like a surgeon complaining that a medecine student asked his patients the wrong questions.. but how do you learn otherwise?
Bob - in the case with the surgeon/student, there are mechanisms to help that student learn from their mistakes. But what's going on here is a status quo - educators who tolerate or even approves of ineffective and wrong work. My first target - and this absolutely clear - is the instructors. That's how the students will learn.
oh please, i am in grade 9 and i have learned that stuff. The correct sample size is the square root of the population. Admittedly, it can be hard to keep sampling, response or non-response bias out of the data. Writing a survey should take hours plus research. You have to research before you actually start researching so to speak.
@ Bob
He's actually criticizing the lack of instruction in creating the surveys in design education. They're basically endorsing a research method without instructing the students on proper use of the method.
After reviewing his blog it seems more personal than professional.
I think its ridiculous to pay for water, when its the most abbundant source of anything on the planet!
Hi there, I'd just like to say that I agree and disagree with a couple points you made.
AGREE - Surveys need to be well written in a clear and concise manner.
AGREE - Good surveys take more than a day to write
AGREE - They shouldn't be used as a substitute for actual research.
DISAGREE - "Survey design is a craft. If you haven�t studied it, you don�t know how to write a survey well, and the data you get is garbage." Sure to extent writing a survey is a craft, but I don't think that you should discourage others from doing so just because they haven't studied the craft. I feel that to write a good survey you need to:
A: Understand the problem
B: Target the right people
C: Word the questions & format of the survey so that you extract the most useful information in the shortest amount of time.
I feel that if you can do the things listed above, write well and know how to use a spell checker there is no reason why someone can't put together a really good survey.
DISAGREE - "They require math to plan (what does your sample size need to be to ensure that your results are valid?) and to analyze (regression analysis (or any other buzzword) anyone?)"
What happens if you're researching a product that is used by relatively few people of whom you know, you don't have the time to put together a large list of people to survey, but know that essential information can be extracted from those few people? And what if you suck at math?
I surveyed 8 people for my major project last year and the information I got was invaluable to my project. I know this will be the case for many other up and coming ID students this year and in years to follow.
I feel this post is rather negative towards those who aren't trained in the art of survey writing and may discourage some from using a highly effective means of research.
Have a great day.