Don't listen to your customers.

27 Mar 2008 - 12:05pm
6 years ago
27 replies
2199 reads
gibbardj at goo...
2008

I'm in a quandary.

I like Dell Ideastorm [1], I like myStarbucksIdea [2] and I like the
approach listening to customers espouse what they like and don't like about
stuff I, and my clients, do.

But, I keep digging up these quotes with monotonous regularity:

a) "If I had asked my customers what they wanted, they would have
said, 'a faster horse" - Henry Ford

b) "We don't ask consumers what they want. They don't know. Instead we
apply our brain power to what they need, and will want, and make sure we're
there, ready" - Akio Morita, founder of Sony

c) "It sounds logical to ask customers what they want and then give it
to them. But they rarely wind up getting what they really want that way" -
Steve Jobs

d) "It's really hard to design products by focus groups. A lot of
times, people don't know what they want until you show it to them" - Steve
Jobs (again)

So should I stop talking about focus groups? Is the old method of ask and
listen not applicable - particularly when designing stuff that's 'future
proof' and therefore impossible to assess with the users of the future - or
should we seek out new methods? Some have suggested trawling user
communities, eavesdropping on online dialogue to perform a gap analysis .
but is the next iPod or Flickr going to come out of a conversation on a
Facebook wall. It just seems so vague. Of course, myStarbucksidea (flawed as
it is from an Ix point of view) is an attempt to localise the dialogue but
will the ultimate output of this just be a 'faster horse'?

For us in the IxD arena when we're trying to create something unique and
something innovative we press ahead with the development of prototypes and
visuals that may reflect an interface and design that doesn't reflect where
our users are today and, because they've not seen the insight we might have
done, simply don't get why they'd need it. A case in point: a piece of work
I've been involved with presented the idea that banking customers could tag
transactions in their account - customers didn't get it: "why would I do
that" . but we know from Mint [3], Wesabe [4] and others that people do use
this feature. The problem being that the client has heard too many users in
testing being dismissive about the idea and therefore increasingly thinks
it's a waste of time. Granted, we could have fleshed out the prototype with
'why would I do this' type content and is this the failing here or simply
that users don't always know best?

Your learned opinions are sought.

John.

[1] http://www.dellideastorm.com/

[2] http://www.mystarbucksidea.com <http://www.mystarbucksidea.com/>

[3] http://www.mint.com <http://www.mint.com/>

[4] http://www.wesabe.com <http://www.wesabe.com/>

Comments

27 Mar 2008 - 6:16pm
Andrei Herasimchuk
2004

On Mar 27, 2008, at 11:05 AM, <gibbardj at googlemail.com> wrote:

> Your learned opinions are sought.

Don't ask people what they want. Simply ask them what they *think*
they want.

Pause.

Then ask them why.

After that, you're on your own.

--
Andrei Herasimchuk

Principal, Involution Studios
innovating the digital world

e. andrei at involutionstudios.com
c. +1 408 306 6422

27 Mar 2008 - 6:54pm
Katie Albers
2005

First of all, I have never seen anything useful come out of a focus
group. Marketing, design, whatever...useless. The only reason to do a
focus group is that your management/client likes them and wants you
to do one.

But on to the actual topic: There's a difference of scale here. Great
new ideas virtually never come out of listening to your users because
the user focus is on making their own day-to-day life a little
easier. That's why so much bad design (in the broadest sense) is
perpetuated. Users are accustomed to it and they want incremental
change...slight betterment, something that will make their work a bit
simpler, something they recognize and have to think *less* about from
day one.

This is not how you get Visicalc, or cars, or refrigerators, or ipods
or TiVOs. Each of these changes fundamentally the paradigm of the
work to be done and the only way to get to those is by looking
waaaaay down-level at what you're doing and how that figures in your
life. You have to entirely redefine the problem at a very low level.
It means not saying "how do we add all those numbers fast and keep
track of them" but "what do we need to do with the numbers", forget
about getting the ice to market faster, take the ice out of the
equation. Look past programming a VCR to tape your favorite show at a
particular time and channel and make a machine do the work of
tracking the show, taping whenever it's on and wherever, and taping
everything that has the person you watch it for whenever *she* is on
a talk show rerun at 3:45 AM, the computer world is going to the
network and going wireless, why spend time, energy and money putting
connection buses on the computer?

This kind of thinking means that instead of figuring out how many
cars would be using the freeway exits per minute (incidentally, they
came up with 1) so that you can avoid accidents, you decide not to
have the entry ramps and exit ramps cross, so that the first problem
disappears.

In many ways, Interaction Design can be said to be an ongoing
decision regarding what problems to look at so that we solve the
right problems. This is never going to be something you'll find out
in a focus group, because even if a participant were to say "Why
solve that problem in the first place? Why not solve this underlying
problem, instead?" You've already decided what your product is, and
it's very unlikely that you'll do more than dismiss that questioner
as a crank.

So the question is: what are you trying to do? Build a better
mousetrap or do away with house mice?

Katie

At 6:05 PM +0000 3/27/08, <gibbardj at googlemail.com> wrote:
>I'm in a quandary.
>But, I keep digging up these quotes with monotonous regularity:
>
>a) "If I had asked my customers what they wanted, they would have
>said, 'a faster horse" - Henry Ford
>
>b) "We don't ask consumers what they want. They don't know. Instead we
>apply our brain power to what they need, and will want, and make sure we're
>there, ready" - Akio Morita, founder of Sony
>
>c) "It sounds logical to ask customers what they want and then give it
>to them. But they rarely wind up getting what they really want that way" -
>Steve Jobs
>
>d) "It's really hard to design products by focus groups. A lot of
>times, people don't know what they want until you show it to them" - Steve
>Jobs (again)
>
>
>
>So should I stop talking about focus groups? Is the old method of ask and
>listen not applicable - particularly when designing stuff that's 'future
>proof' and therefore impossible to assess with the users of the future - or
>should we seek out new methods? Some have suggested trawling user
>communities, eavesdropping on online dialogue to perform a gap analysis .
>but is the next iPod or Flickr going to come out of a conversation on a
>Facebook wall. It just seems so vague. Of course, myStarbucksidea (flawed as
>it is from an Ix point of view) is an attempt to localise the dialogue but
>will the ultimate output of this just be a 'faster horse'?
>
>
>
>For us in the IxD arena when we're trying to create something unique and
>something innovative we press ahead with the development of prototypes and
>visuals that may reflect an interface and design that doesn't reflect where
>our users are today and, because they've not seen the insight we might have
>done, simply don't get why they'd need it. A case in point: a piece of work
>I've been involved with presented the idea that banking customers could tag
>transactions in their account - customers didn't get it: "why would I do
>that" . but we know from Mint [3], Wesabe [4] and others that people do use
>this feature. The problem being that the client has heard too many users in
>testing being dismissive about the idea and therefore increasingly thinks
>it's a waste of time. Granted, we could have fleshed out the prototype with
>'why would I do this' type content and is this the failing here or simply
>that users don't always know best?
>
>
>
>Your learned opinions are sought.
>
>John.
>
>
>
>[1] http://www.dellideastorm.com/
>
>[2] http://www.mystarbucksidea.com <http://www.mystarbucksidea.com/>
>
>
>[3] http://www.mint.com <http://www.mint.com/>
>
>[4] http://www.wesabe.com <http://www.wesabe.com/>
>
>________________________________________________________________
>Welcome to the Interaction Design Association (IxDA)!
>To post to this list ....... discuss at ixda.org
>Unsubscribe ................ http://www.ixda.org/unsubscribe
>List Guidelines ............ http://www.ixda.org/guidelines
>List Help .................. http://www.ixda.org/help

--

----------------
Katie Albers
katie at firstthought.com

27 Mar 2008 - 7:05pm
Marijke Rijsberman
2004

All right. I'll bite. But rant first:

Even though I understand exactly what people mean when they say "don't
listen to your customers" or "don't pay attention to what users say, pay
attention to what they do," these things irk the hell out of me. Of course
you should listen to your customers--as you should listen to your kids, and
your pets, and your friends, and your enemies, and everyone and everything
that has the ability to express itself. Duh!

But listening to your customers is not the same thing as listening to your
parents when you're 5 years old. You don't have to do what they say, just
because they say so. So to begin with, it would make sense to ask people
only for information you actually are going to want to listen to. (I don't
ask my cat whether he wants his flea medicine; I do ask, in one way or
another, whether he prefers the beef-goop or the salmon-goop that comes out
of the can of cat food.)

Okay, now that I've got that off my chest ... your post actually brings up a
great question about what kind of research will get you what kind of
results.

For instance, testing prototypes is not a good way to suss out what (small?)
percentage of people is going to do something like write reviews, tag their
expenses, or do some other "power user" type of thing which demands a lot
more dedication than the average user would bring to it. That requires a
different (and likely more quantitative) type of research.

To run a useful study on such prototypes, you'd want to make sure you have
figured out what kinds of people are the most likely to do power-X and then
recruit _them_. This might mean recruiting for people who study and analyze
and dissect their quarterly credit card expense breakdowns. Or, frankly, the
people who tag their expenses on Wesabe and Mint. If _they_ don't like the
functionality you're building, then you should wonder whether you're doing
the right thing.

In short, absolutely listen to your customers, but only the right ones
responding to the right questions.

marijke

John said
>For us in the IxD arena when we're trying to create something unique and
>something innovative we press ahead with the development of prototypes and
>visuals that may reflect an interface and design that doesn't reflect where
>our users are today and, because they've not seen the insight we might have
>done, simply don't get why they'd need it. A case in point: a piece of work
>I've been involved with presented the idea that banking customers could tag
>transactions in their account - customers didn't get it: "why would I do
>that" . but we know from Mint [3], Wesabe [4] and others that people do use
>this feature. The problem being that the client has heard too many users in
>testing being dismissive about the idea and therefore increasingly thinks
>it's a waste of time. Granted, we could have fleshed out the prototype with
>'why would I do this' type content and is this the failing here or simply
>that users don't always know best?

27 Mar 2008 - 5:56pm
christine chastain
2008

Depending on the situation, I do or don't listen to what people say.
For really big innovations that have absolutely nothing to do with
anything out there now, asking the average consumer about them isn't
useful in most cases. One is better off researching trends, coming up
with multiple scenarios of the future, forming hypotheses and then
going into the field to substantiate or refute those based on what is
happening today. Even that will not provide answers but simply point
toward a more likely future scenario provided there are no great
disruptions, new technologies out of left field, etc.

For near term improvements on a product/service or innovations that
are near-term and resemble something which already exists, I still
don't listen to what people say but rather observe what they do and
try to understand their base, emotional unmet needs. Here, archetype,
semiotics, ethnography is very useful.

For immediate improvements to existing products/services/interfaces, I
listen to what lead or extreme users have to say and I watch them to
uncover new opportunity. I look for outliers - people who do things
differently from everyone else and either accomplish the same thing or
figure out a better way. I also listen for what people don't say and
observe any workarounds. If great detail is involved, I'll suck it up
and do a time-motion study...;)

So I really do think your objectives or those of your client should
lead one's approach. But then, there are always exceptions to every
rule!

27 Mar 2008 - 9:08pm
Troy Gardner
2008

I say skip it all, provide paths for everything, collect heatmaps,
normalize UI ruthlessly.

27 Mar 2008 - 10:06pm
Dwayne King
2005

Talk to the customer to understand the needs and issues, not to have
them help design. Designers design, consumers consume - that doesn't
mean the consumer can't tell you what is bad about their current
experiences to feed the design fire. We hold a lot of user meetings
and have to craft them to keep the users from trying to design
solutions. That said, the insight is invaluable.

My opinion.

On Mar 27, 2008, at 11:05 AM, gibbardj at googlemail.com wrote:

> I'm in a quandary.
>
>
>
> I like Dell Ideastorm [1], I like myStarbucksIdea [2] and I like the
> approach listening to customers espouse what they like and don't
> like about
> stuff I, and my clients, do.
>
>
>
> But, I keep digging up these quotes with monotonous regularity:
>
>
>
> a) "If I had asked my customers what they wanted, they would
> have
> said, 'a faster horse" - Henry Ford
>
> b) "We don't ask consumers what they want. They don't know.
> Instead we
> apply our brain power to what they need, and will want, and make
> sure we're
> there, ready" - Akio Morita, founder of Sony
>
> c) "It sounds logical to ask customers what they want and then
> give it
> to them. But they rarely wind up getting what they really want that
> way" -
> Steve Jobs
>
> d) "It's really hard to design products by focus groups. A lot
> of
> times, people don't know what they want until you show it to them" -
> Steve
> Jobs (again)
>
>
>
> So should I stop talking about focus groups? Is the old method of
> ask and
> listen not applicable - particularly when designing stuff that's
> 'future
> proof' and therefore impossible to assess with the users of the
> future - or
> should we seek out new methods? Some have suggested trawling user
> communities, eavesdropping on online dialogue to perform a gap
> analysis .
> but is the next iPod or Flickr going to come out of a conversation
> on a
> Facebook wall. It just seems so vague. Of course, myStarbucksidea
> (flawed as
> it is from an Ix point of view) is an attempt to localise the
> dialogue but
> will the ultimate output of this just be a 'faster horse'?
>
>
>
> For us in the IxD arena when we're trying to create something unique
> and
> something innovative we press ahead with the development of
> prototypes and
> visuals that may reflect an interface and design that doesn't
> reflect where
> our users are today and, because they've not seen the insight we
> might have
> done, simply don't get why they'd need it. A case in point: a piece
> of work
> I've been involved with presented the idea that banking customers
> could tag
> transactions in their account - customers didn't get it: "why would
> I do
> that" . but we know from Mint [3], Wesabe [4] and others that people
> do use
> this feature. The problem being that the client has heard too many
> users in
> testing being dismissive about the idea and therefore increasingly
> thinks
> it's a waste of time. Granted, we could have fleshed out the
> prototype with
> 'why would I do this' type content and is this the failing here or
> simply
> that users don't always know best?
>
>
>
> Your learned opinions are sought.
>
> John.
>
>
>
> [1] http://www.dellideastorm.com/
>
> [2] http://www.mystarbucksidea.com <http://www.mystarbucksidea.com/>
>
>
> [3] http://www.mint.com <http://www.mint.com/>
>
> [4] http://www.wesabe.com <http://www.wesabe.com/>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

28 Mar 2008 - 1:34am
Bill Fernandez
2007

At 6:05 PM +0000 3/27/08, <gibbardj at googlemail.com> wrote:
... snip ...
>So should I stop talking about focus groups? Is the old method of ask and
>listen not applicable - particularly when designing stuff that's 'future
>proof' and therefore impossible to assess with the users of the future - or
>should we seek out new methods?...

If you're designing something completely new you have the opportunity
to approach it in an entirely new way (cf the iPhone), but if you are
improving or extending an existing product you can't break completely
out of the existing mold (cf the 2nd to 5th generations of the iPod).
And each project will have it's own limits imposed by time, budget,
the visions/imaginations of stakeholders, the political structure,
etc.

Given all that, I have learned that you can almost never take a
user's words verbatim. Listen to them, gather all the raw data that
seems reasonable, but then try to dig down to the root causes, the
core motivations that leads people to say what they do. then try to
solve the "real" problems rather than the "stated" problems.

For two interesting and useful perspectives on how far what users
think and say depart from what the fundamental reality truly is, you
might read "Freakonomics" and "The Culture Code". These don't
directly apply to UX design, but from two very different viewpoints
they make it clear that you shouldn't take what most people say at
face value.

FWIW,
Bill

--

======================================================================
Bill Fernandez * User Interface Architect * Bill Fernandez Design

(505) 346-3080 * bf_list1 AT billfernandez DOT com *
http://billfernandez.com
======================================================================

28 Mar 2008 - 6:40am
Todd Warfel
2003

On Mar 27, 2008, at 9:05 PM, Marijke Rijsberman wrote:

> For instance, testing prototypes is not a good way to suss out what
> (small?) percentage of people is going to do something like write
> reviews, tag their expenses, or do some other "power user" type of
> thing which demands a lot more dedication than the average user
> would bring to it. That requires a different (and likely more
> quantitative) type of research.

Completely disagree. Last year we did several rounds of usability
testing for LA Times w/prototypes looking at tagging, reviews, and
other social idioms. In fact, the usability testing highlighted
something we never would have seen in quantitative research—that while
people aren't sure what tags are, the interaction of what a tag does
meets their expectation.

If we had done a quantitative approach, we would have seen near 0%
interaction and based on that would have scrapped tagging, ratings,
and reviews from the new Calender Live site. However, with in-person
testing, we were able to get feedback from users that showed:
1. Only power-users are likely to migrate to tagging, ratings, and
reviews.
2. Power-users are not age-defined.
3. 3-5% of users will rate, tag, or review.
4. Non-power-users were willing and often interested to explore
tagging, ratings, and reviews, but sometimes needed some type of
prompting. Understanding what kind of prompt they needed helped us
engage them in future rounds of testing. Gaining this understanding is
only something we could have obtained by in-person discussions, not
through a web-survey.
5. Through in-person studies were able to perform some collaborative
design with the participants and determine the priority levels of the
information on the screen. This lead to design concepts that enabled
us to put tags clouds (something that less than 2% of our participants
knew what it was) in the appropriate place on the screen so that they
were out of the way of those who wouldn't use them, but reachable for
those who would.
6. When encouraged to explore tags, every participant who did found
them extremely useful and immediately saw the benefit. We didn't
explain the benefit and ask them to try them, we simply asked what
they expected to happen if they clicked on those "things" and then had
them try it out and followed up with "how does that compare to what
you expected?" Very vague, but it does the trick w/o leading.

Numbers 1-3 could be accomplished w/a quantitative study, but 4-6 took
a qualitative study to perform. And frankly, 4-6 were insights that
were new, while 1-3 are things we could have learned by googling.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

28 Mar 2008 - 7:34am
Dan Saffer
2003

The most common misconception about design research is that you are
asking users what the design should be. You aren't (or shouldn't be).
Instead, the best design research I've been involved in is about
finding data on three things:

1. Unmet needs. Usually unspoken and unrealized. Yes, people would
have asked for a faster horse, but what is the need there? To travel
longer distances quicker. The automobile was the solution to that need.

2. Pain points. Where is what is being done now difficult?

3. Opportunities. Where is there a space for a product or service that
would meet those unmet needs or fix the pain points?

Then it is our job to design the solution. This is what we are paid to
do. :) Now, obviously, if a research subject comes up with a good
solution, by all means steal it!

Dan

28 Mar 2008 - 1:48am
Kristof Versluys*
2008

What people tell they do & What people actually do = completely different

Listen to your customer. Get him involved.
But even better, see him use a product/website/...
Give him simple tasks. Ask him to describe what he's doing.

Pay attention to the underlying issues;
if he/she wants a faster horse, you don't have to build or find a faster horse.
Extraction: you now know they want to go faster

Your customer won't bring the solution, but he can point out some key-issues.
That is, if you're a "good" listener ;

K.

28 Mar 2008 - 6:45am
Todd Warfel
2003

On Mar 27, 2008, at 2:05 PM, <gibbardj at googlemail.com> <gibbardj at googlemail.com
> wrote:

> Granted, we could have fleshed out the prototype with 'why would I
> do this' type content and is this the failing here or simply that
> users don't always know best?

There is a difference between doing what your customers say and
actually finding out/interpreting their needs based on a conversation
with them and observing their behaviors.

Context is key. They guys at Cheskin will tell you about some
ethnographic-based research they were doing for a luxury goods
company. During one interview the participant claimed she didn't
perform any luxury services. Cheskin looked down and noticed she had a
French manicure. When the facilitator/interviewer asked about her
obvious luxury item of a French manicure, the woman replied "Oh,
honey, that's not a luxury, that's a necessity!"

As an outsider, we can watch for these types of things and run them
through our interpretation filter to determine want vs. need. And
really, its about keeping the proper balance between listening,
understanding, and interpretation.

The customer isn't always right, but they can be a good source for
inspiration of what to do and what not to do.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

28 Mar 2008 - 12:11pm
Jack L. Moffett
2005

Well put, Dan. I'd like to add one item to your list.

On Mar 28, 2008, at 9:34 AM, Dan Saffer wrote:

> ...the best design research I've been involved in is about
> finding data on three things:
>
> 1. Unmet needs
> 2. Pain points
> 3. Opportunities

4. Conventions - What practices, knowledge, or experiences do your
users have that can be leveraged to make a solution "fit"—make it
comfortable and understandable (i.e. intuitive).

Jack

Jack L. Moffett
Interaction Designer
inmedius
412.459.0310 x219
http://www.inmedius.com

When I am working on a problem,
I never think about beauty.
I think only of how to solve the problem.

But when I have finished,
if the solution is not beautiful,
I know it is wrong.

- R. Buckminster Fuller

28 Mar 2008 - 10:13am
Chris Bernard
2007

Microsoft uses a lot of focus groups. Take that for what's it worth. From an ideation and concepting perspective I think they have minimal value and can in fact be disruptive, in that they can force you down a prescribed path far too soon. Far better to follow Andrei's advice or even better augment it by watching people. Even one person with a camera and notebook making quite observations can be a great augmentation to structured interviews.

The canonical example of focus groups is New Coke. They focus grouped the heck out of that before they launched.

Chris Bernard
Microsoft
User Experience Evangelist
chris.bernard at microsoft.com
630.530.4208 Office
312.925.4095 Mobile

Blog: www.designthinkingdigest.com
Design: www.microsoft.com/design
Tools: www.microsoft.com/expression
Community: http://www.visitmix.com

"The future is already here. It's just not evenly distributed." William Gibson

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com [mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of Andrei Herasimchuk
Sent: Thursday, March 27, 2008 7:17 PM
To: IXDA list
Subject: Re: [IxDA Discuss] Don't listen to your customers.

On Mar 27, 2008, at 11:05 AM, <gibbardj at googlemail.com> wrote:

> Your learned opinions are sought.

Don't ask people what they want. Simply ask them what they *think*
they want.

Pause.

Then ask them why.

After that, you're on your own.

--
Andrei Herasimchuk

Principal, Involution Studios
innovating the digital world

e. andrei at involutionstudios.com
c. +1 408 306 6422

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

28 Mar 2008 - 10:24am
Uday Gajendar
2007

> it's a waste of time. Granted, we could have fleshed out the
> prototype with
> 'why would I do this' type content and is this the failing here or
> simply that users don't always know best?

I always take user research findings (quant or qual) with many grains
of salt. It's supplementary data to help a designer understand,
empathize, interpret, and then make a "good" decision. Designers are
informed visionaries, not "short-order cooks" doing simply what the
"user asks" b/c often users often don't know what they want, nor how
to express exactly what they want. If they did, we wouldn't have
jobs :-) Designers must exercise their judgment (comes with years of
experience, i know) to use or dismiss that data.

Also, what's the goal?

1) breakthrough innovation: no one asked for a Wii, iPod, Prius, Dyson
or flickr, but once manifested, then users wanted them. Discovery
activities, asking user motives/reasons, social/tech trends, scenarios
might help...but again, take salt with what you find!

2) incremental clean-ups: if it's minor tweak for the next point
release, listening to those 500 complaints on your forum about the
wrong button label would be good :-) But again, if there's a chance to
introduce an innovative UI or behavior, then try it, get a pulse on
users reaction, and decide for yourself how to proceed.

For other resources, google the following folks: Larry Keeley (Chicago
ID), Vijay Kumar (Chicago ID), Roger Martin (Rotman), and Craig Vogel
(Cincinatti)--all academics but with practical understanding how
breakthrough innovation happens in large corps like Motorola, Ford,
Whirlpool, etc.

But I'd say: Gather some user info, question what's said, reflect on
it, create a solution per your design abilities, defend and iterate,
etc.

and good luck :-) (with lots of salt!!)

Uday Gajendar
Sr. Interaction Designer
Voice Technology Group
Cisco | San Jose
------------------------------
ugajenda at cisco.com
+1 408 902 2137

28 Mar 2008 - 2:28pm
Gene Moy
2008

There are tools for the appropriate places and times. Focus groups are
attitudinal and small group dynamics being what they are can skew
consensus. Perhaps not the best use for some things we do.
Ethnography and other social science approaches towards observing
the user in the environment may be more fruitful for yielding gap
analyses in terms of need generation.

Then too, does anyone remember the often-recounted study about yellow
boomboxes vs black ones, the users said they would all buy yellow
ones, but then when asked to pick up free ones on the way out, they
all picked up black ones? I love that. Don't get me started on who
says they wash their hands coming out of the bathroom vs those who
actually do.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=27702

28 Mar 2008 - 2:44pm
SemanticWill
2007

Wasn't it mentioned here or somewhere else that the first use of "Focus
Groups" was for the Edsel?? If that doesn't about say it all -

There is the story about the 12 people (?) brought in to focus group on a
new personal stereo (boombox they were called at the time), and people were
asked what colours they would like - and a large majority responded very
favorably to the canary yellow boombox.
At the end - as they were walking out the door - they were offered boomboxes
as thank you's for doing the focus group. Yellow was offered. Everyone took
black.

Users lie. Ouch! What did Will just say?

They lie. Sometimes they don't even know it. In user testing - they could
have completed a task 10 minutes ago - and they will lie about what they did
- well - they will not "remember" correctly what they did -- which is why
you observe what they do - not what they said they did.

Let the flames begin - I am pulling out my umbrella now - just incase anyone
throws veggies.

On Fri, Mar 28, 2008 at 12:13 PM, Chris Bernard <Chris.Bernard at microsoft.com>
wrote:

> Microsoft uses a lot of focus groups. Take that for what's it worth. From
> an ideation and concepting perspective I think they have minimal value and
> can in fact be disruptive, in that they can force you down a prescribed path
> far too soon. Far better to follow Andrei's advice or even better augment it
> by watching people. Even one person with a camera and notebook making quite
> observations can be a great augmentation to structured interviews.
>
> The canonical example of focus groups is New Coke. They focus grouped the
> heck out of that before they launched.
>
> Chris Bernard
> Microsoft
> User Experience Evangelist
> chris.bernard at microsoft.com
> 630.530.4208 Office
> 312.925.4095 Mobile
>
>

28 Mar 2008 - 2:57pm
Erik van de Wiel
2008

Hi John,

Instead of using focus groups, I use personas to get a clear view on the
user goals. Constructing personas isn't about asking what users want. It
is about trying to figure out their daily goals (anything from being
happy to finish my todo list by the end of the day). I believe that as a
designer I should always try to design in a way that enables users to
accomplish these goals.

Personally I prefer the goal directed design approach from Cooper.
"Qualitative research helps us understand the domain, context, and
constraints of a product in different, more useful ways than
quantitative research does. It also helps us identify patterns of
behavior among users and potential users of a product much more quickly
and easily than would be possible with quantitative approaches."[1]

The people from Cooper posted some articles on their journal[2], these
might be helpful. I think The Persona Lifecycle[3] is a great first
introduction with personas.

Gr.
Erik van de Wiel

1: Book: About Face 3: The Essentials of Interaction Design -- Alan
Cooper (quote: page 50)
http://www.amazon.com/About-Face-Essentials-Interaction-Design/dp/0470084111/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1206699216&sr=8-1
<http://www.amazon.com/About-Face-Essentials-Interaction-Design/dp/0470084111/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1206699216&sr=8-1>

2: http://www.cooper.com/insights/journal_of_design/articles/personas/

3: Book: The Persona Lifecycle
http://www.amazon.com/Persona-Lifecycle-Throughout-Interactive-Technologies/dp/0125662513/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1206699232&sr=1-1
<http://www.amazon.com/Persona-Lifecycle-Throughout-Interactive-Technologies/dp/0125662513/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1206699232&sr=1-1>

28 Mar 2008 - 2:29pm
Gene Moy
2008

There are tools for the appropriate places and times. Focus groups are
attitudinal and small group dynamics being what they are can skew
consensus. Perhaps not the best use for some things we do.
Ethnography and other social science approaches towards observing
the user in the environment may be more fruitful for yielding gap
analyses in terms of need generation.

Then too, does anyone remember the often-recounted study about yellow
boomboxes vs black ones, the users said they would all buy yellow
ones, but then when asked to pick up free ones on the way out, they
all picked up black ones? I love that. Don't get me started on who
says they wash their hands coming out of the bathroom vs those who
actually do.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=27702

28 Mar 2008 - 2:04pm
Stefan Walger
2008

The company I currently work for provides online services for
restaurants. One of our greatest accomplishments for this company as
a design team is a seating management program that is about to go
into several test restaurants.

When we began the research for this product, we went into all the
restaurants here in the town of Durango to interview hosts and
hostesses. We did not really introduce ourselves as designers looking
to develop a better way to seat guests but more so as researchers for
a start-up company. We began by asking them about the work they do
and had them walk us through a typical workday. We then proceeded to
examine their actual behavior. We documented the different types of
mediums they used in taking reservations, handling walk-ins and
dealings with restaurant staff. It was in observing their behaviors
that allowed us to clearly see both the problems and solutions.

Once we compiled this information, we were able to build solid
personas, true-to-life scenarios, story boards, workflows and then
wireframes that we then could take back to the host and hostesses for
usability testing. From there, we did visual comp specs that we then
tested again before turning over to engineering.

Ultimately, it is the quality of the research and experience of the
researcher that will determine the true value it can bring to the
design process. We were fortunate to have an outstanding individual
who truly understood how to properly perform research to give us such
powerful results. I apologize if I seem condescending, but if you are
not getting value out of your research, then you need to ask yourself
if you are doing it right. We also have employees here who are not
adept to doing research correctly and the results have been less than
useless.

Even with my experience doing this research, I do not consider myself
an expert in this field.

My only bit of advice (at this moment) is:

Do not go straight into telling people what you are doing and then
asking them what they think they would want you to do. Even by
telling them what you are doing, you begin influencing their
answers.Your interviewees will always want to impress you with their
intelligence and try to say what they think you want to hear. So
don't give them any clues.

Research their behaviors, maybe even secretly (I know, easier said
than done) They will give you more answers if they think they are not
being tested under the microscope.

Maybe save the hardcore "interviewing" as a means of usability
testing with some low fidelity mock ups. We just made color copies
and had people pretend they were computer screens.

Have the right people do the interviewing. Many of us think we can do
it right, but if others observed us, they may think differently.

As for myself, I look forward to continually learning from all of you
here - Thank you for your time!
Stefan

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=27702

28 Mar 2008 - 3:20pm
Nancy Broden
2005

I second Todd's observation. There is nothing better than being in
your customer's environment to help you as a designer puts some
context around what they are saying.

I'll share an IDEO story that they like to show to clients to help
them understand why contextual sessions with customers are necessary.
IDEO was working with a client in the health and beauty industry. As a
part of the project, the team interviewed "extreme users" - those
people who said they never, ever used beauty products or services, as
well as those for whom pampering was a regular habit. The clip that
IDEO plays is of a forklift operator - a big burly guy who falls into
the former category. During the session, one of the observers noted
that there was a home foot spa next to the sofa where the interview
was taking place. When asked about it, the guy admitted that it wasn't
just for his fiancee, that he used it as well, explaining that the
boots he had to wear for work every day did a number on his feet and
the spa helped relieve his aches and pain. He simply didn't (or didn't
want to) interpret that to be a 'beauty product' or his daily foot spa
to be 'pampering'...

On Mar 28, 2008, at 5:45 AM, Todd Zaki Warfel wrote:

> There is a difference between doing what your customers say and
> actually finding out/interpreting their needs based on a conversation
> with them and observing their behaviors.

--------------------------------
Nancy Broden
nancy.broden at gmail.com

28 Mar 2008 - 7:04pm
Rob Tannen
2006

At the end of the day, customers are an essential source of data for
informing a design, but they are one of many. The key to effective
design research is triangulation - finding support and patterns
across multiple data sources including user observations and
interviews, competitive market research, and designer expertise.
Multiple, independent data sources will support or cancel out ideas,
resulting in a viable set of results.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=27702

29 Mar 2008 - 9:28am
dszuc
2005

Yup!

Question assumptions and be lucky enough to work in an environment/
culture where you are a) allowed to fail b) not look stupid for doing
it and c) rewarded with the aim of coming up with something better.

rgds,

Daniel Szuc
Principal Usability Consultant
www.apogeehk.com
T: +852 2581 2166
F: +852 2833 2961
"Usability in Asia"

The Usability Kit - www.theusabilitykit.com

On 28/03/2008, at 6:04 PM, Rob Tannen wrote:

> At the end of the day, customers are an essential source of data for
> informing a design, but they are one of many. The key to effective
> design research is triangulation - finding support and patterns
> across multiple data sources including user observations and
> interviews, competitive market research, and designer expertise.
> Multiple, independent data sources will support or cancel out ideas,
> resulting in a viable set of results.
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=27702
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

31 Mar 2008 - 7:31am
AJ Kock
2007

I am going to get slightly philosophical and maybe it will illustrate
the topic better/differently.

People have functions
Products serve functions
People rarely think about these functions, they just do them.
Therefore asking people about these functions rarely gives you the
right answer, because people don't think about them.
But we ask people for other reasons: For example, you need to make
sure that your definition of their environment corresponds with their
definition of their environment, so that you are both on the same
wavelength and your deductions are not based on assumed facts.

So, whichever test you use, understand what you are learning from that
test and the relevance. Now put all these famous quotes in context and
you will understand that they all listen to their target market, they
just do it different. When something like "You have to listen to your
customers" becomes fashionable, people tend to use it at inappropriate
times. There is a correct place and time for everything and there is
more than one way of "listening".

Addition:
The Evolution of Functions & Products
What are functions? Functions are things you do everyday: You go to
work, you work on a PC, you write with a pen, etc. It is everything
that you do.
What are products? Products are everything that you use to complete
those functions: A car to drive to work, a Dual Core PC to do work on,
a Parker to write with, etc.
In general, we do functions and we have to get them done. Any product
that improves the time it takes to get things done is a plus and if it
does it better, it is another plus. Now the problem with getting
things done is that we suddenly have more time available, but that
leads to us adding more functions and the cycle continues.

Example: Compare your original PC with your current one. How fast was
the original and how many things did you do with it? How fast is the
new one and how much are you doing with it now? Yet, we still complain
that our current PC's are slow when we want to get things done, when
it is probably 20 times faster than our original PC. Why is that?

31 Mar 2008 - 9:00am
zack Frazier
2007

This article about Ketchup, of all things, is a good reminder that
market forces and embedded tastes cannot be underestimated ...
* http://www.gladwell.com/2004/2004_09_06_a_ketchup.html

2 Apr 2008 - 2:47pm
Oleg Krupnov
2008

Did you run across the situation when after researching the users you find
that people stick to a wrong or inefficient way of doing the target
function, and you find yourself choosing between insisting on the better
practices and thereby perhaps losing a big part of the conservative
audience, or try to please them by indulging and optimizing what they are
used to?

Oleg.
--
View this message in context: http://www.nabble.com/Don%27t-listen-to-your-customers.-tp16342555p16451198.html
Sent from the ixda.org - discussion list mailing list archive at Nabble.com.

4 Apr 2008 - 7:45am
Stew Dean
2007

On 28/03/2008, Kristof Versluys <kristofversluys at gmail.com> wrote:
> What people tell they do & What people actually do = completely different
>
> Listen to your customer. Get him involved.
> But even better, see him use a product/website/...
> Give him simple tasks. Ask him to describe what he's doing.

I say take it up a level. You're right what they do is the key here
NOT what they want.

My focus has been on finding out what they do not even mentioning the
product or website. This sets the 'universe' that the product/site
exists in. To me this is the basis of any final solution, even
redesigns. The users are going to use it to do something - how does
that fit with their world. It's very contextual and I find it scares
many folks as they expect us to start by testing the website and I
totally ignore 90% of the time when talking to users.

So the 'faster horse' thing is spot on. I try never to ask the user
'what do you want on the website' but instead 'what would make what
you do easier'. There's then a series of steps to get from that to
actual interaction design which can be squeezed into suprisingly short
amout of time and radically improve the quality of the end project
(and make it much simpler).

> Pay attention to the underlying issues;
> if he/she wants a faster horse, you don't have to build or find a faster horse.
> Extraction: you now know they want to go faster

Get the questioning level right, I've found, and you can get them to
tell you that they just want to go faster. In short let the users set
the scope and what functionality is important to them and what
information they need and when - we can do the rest :)

--
Stewart Dean

11 Apr 2008 - 8:56am
DrWex
2006

Sorry I'm a bit late to this thread. Ignore if you want.

I think people have forgotten where the idea of "listen to your
customers" (or users) came from, though I think Dan Saffer's response
in this thread came close to how I feel.

The idea source, as I remember it, was the radical notion that (a)
your customers/users know things you don't, and finding out those
things can be valuable; and (b) the interaction between a company and
a customer should more resemble a conversation or ongoing stream than
one-off product sale(s). I strongly agree with these premises

Under these basic assumptions, it seems trivially true that one ought
to listen. One listens to gain knowledge one cannot get in other
ways. One listens because a conversation where only one side speaks
isn't really a conversation and the non-speaking side is likely to get
bored or frustrated and walk away.

Where this idea goes off the rails is when one narrows down the notion
of "listening" to "focus groups." Or when one interprets the idea "I
must listen" to mean "I must get design ideas from my customers." I
strongly disagree with these consequents that some people seem to draw
from the original premises. But just because we don't like the
consequents doesn't mean the original premises are flawed.

In the original post to this thread John Gibbard gave some quotes that
indicate the notion of listening to customers has limits. It's not a
cure-all. Granted. But we could just as easily say the same things
about any IxD process or artifact (scenarios, personas, user tests,
task hierarchies, etc). All of these have limitations and also have
value.

Just because listening to customers has its limits doesn't mean it has
no value. Baby, bathwater, sploosh.

Best,
--Alan

Syndicate content Get the feed