User Research takes Kodak higher

27 Dec 2004 - 9:23am
9 years ago
16 replies
731 reads
Dave Malouf
2005

NY Times today had a great article about how user research was used to shore
up Kodak's marketshare from 5% to near 20% in 4 years. they called it
"anthropology" but we know what it really is. ;-)

Here is the URL (you'll need to register to see it):

<http://www.nytimes.com/2004/12/27/technology/27kodak.html?ex=1105156872&ei=
1&en=85312961a76d178b>
http://www.nytimes.com/2004/12/27/technology/27kodak.html?ex=1105156872&ei=1
&en=85312961a76d178b

-- dave

Comments

27 Dec 2004 - 3:31pm
Alain D. M. G. ...
2003

--- David Heller <dave at ixdg.org> a écrit :
> NY Times today had a great article about how user research was used
> to shore
> up Kodak's marketshare from 5% to near 20% in 4 years. they called it
> "anthropology" but we know what it really is. ;-)
>

Given that most usability practitioners are not usually given the time
to do the kind of cultural anthropology study that Kodak made, I think
that the omission of the term "usability" or "interaction design" or
like is perfectly understandable.

In fact I often have the impression that most interaction design and/or
usability specialists are not interested in doing or do not have the
training or experience to do such studies.

Alain V.

__________________________________________________________
Lèche-vitrine ou lèche-écran ?
magasinage.yahoo.ca

27 Dec 2004 - 3:37pm
Alain D. M. G. ...
2003

oops, sorry, I forgot that they did mention that Paul Porter, head of
Design and usability at Kodak was behind this. They even have his
picture with the words in the caption, but my comment still stands on
the dearth of cultural anthropology studies in this field.

Alain V.

>
> Given that most usability practitioners are not usually given the
> time
> to do the kind of cultural anthropology study that Kodak made, I
> think
> that the omission of the term "usability" or "interaction design" or
> like is perfectly understandable.
>
> In fact I often have the impression that most interaction design
> and/or
> usability specialists are not interested in doing or do not have the
> training or experience to do such studies.
>
> Alain V.
>
> __________________________________________________________
> Lèche-vitrine ou lèche-écran ?
> magasinage.yahoo.ca
> _______________________________________________
> Interaction Design Discussion List
> discuss at ixdg.org
> --
> to change your options (unsubscribe or set digest):
> http://discuss.ixdg.org/
> --
> Questions: lists at ixdg.org
> --
> Announcement Online List (discussion list members get announcements
> already)
> http://subscribe-announce.ixdg.org/
> --
> http://ixdg.org/
>

__________________________________________________________
Lèche-vitrine ou lèche-écran ?
magasinage.yahoo.ca

27 Dec 2004 - 3:57pm
Dave Malouf
2005

> oops, sorry, I forgot that they did mention that Paul Porter,
> head of Design and usability at Kodak was behind this. They
> even have his picture with the words in the caption, but my
> comment still stands on the dearth of cultural anthropology
> studies in this field.
>
> Alain V.

Alain, there is very little difference between what they are calling
"anthropology" and what we would call "goal-directed studies" or "contextual
inquiry".

Applied anthropology has firmly entered the IxD realm. Many consultancies
use it pretty much as a standard practice and a growing number of large
in-house organizations use it. I did contextual inquiry studies as part of
my work at Documentum and did ethnographic work as part of a discovery study
for work on a supply-chain-management tool I designed a couple of years
before that.

A standard part of UCD is "context" ... One way or another the strongest
addition to a designer's arsenal is the data and alaysis of a human being's
environment, social relationships, and goals and motivations.

-- dave

28 Dec 2004 - 2:35pm
Steven Streight
2004

Context sensitivity and ethnomethodology are indeed
important to factor into all UCD, but it rarely occurs
in any product, IMHO.

I constantly encounter web sites that disguise or bury
the primary function I want to use.

The contents and function are organized and presented
in a manner that makes sense to the designer,
management, etc., but not to a typical user.

This is why I believe that only User Observation
Testing is a valid means for determining true
usability.

But even UOT usually occurs outside of the real
context of use. Testing itself sets up an artificial
environment, but at least the user is not a
hypothetical character only vaguely understood.

While heuristic evaluation can suggest improvements
based on general principles, watching actual users
attempt to find information and operate functions is
the source of true illumination.

Context means users are in a big hurry--and surrounded
by distractions at home or at work. While some
designers think users are stupid, often it's the other
way around. USD specialists have to know users first,
then the product. Users should come first in all
design decisions.

Users almost never have the luxury of focusing
intensely on your precious web site. And if they can't
quickly find what they're looking for--they'll leave
your site and go somewhere else.

=====
Steven Streight
Web Usability Analyst/Content Writer

http://www.vaspersthegrate.blogspot.com
http://www.streightsite.blogspot.com
http://www.arttestexplosion.blogspot.com
EMAIL: vaspersthegrate at yahoo.com

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

28 Dec 2004 - 3:20pm
Andrei Herasimchuk
2004

Steven Streight wrote:

> The contents and function are organized and presented
> in a manner that makes sense to the designer,
> management, etc., but not to a typical user.

A little bit of hyperbole to end the year?

> This is why I believe that only User Observation
> Testing is a valid means for determining true
> usability.

Is that a little like the only way to get "good" movies is through test
screenings?

Users are not the end all be all in our profession. I really dislike
seeing such sweeping statements for our work, which requires attention
to details in so many aspects of building the product to get right.

> But even UOT usually occurs outside of the real
> context of use. Testing itself sets up an artificial
> environment, but at least the user is not a
> hypothetical character only vaguely understood.

Indeed. I find user testing to be good for gut checks only. The best
information I've found has always occured from use of products over a
one to three month time frame. Behind the glass testing rarely gives me
any meaningful data to work with as designer.

> While heuristic evaluation can suggest improvements
> based on general principles, watching actual users
> attempt to find information and operate functions is
> the source of true illumination.

Along with being a user yourself as much as possible. I find that those
who watch only have half the story. Until you actually become a user
yourself and feel the problems first hand, you tend to only see half the
solution possible.

If you designed and built a chair but never sat in it, how can possibly
know how to fix its flaws? By only watching other people sit in it? Hardly.

> Context means users are in a big hurry--and surrounded
> by distractions at home or at work. While some
> designers think users are stupid, often it's the other
> way around. USD specialists have to know users first,
> then the product. Users should come first in all
> design decisions.

More hyperbole. Great design is when something works on many different
levels, and users at best are only 1/3 of that equation.

> Users almost never have the luxury of focusing
> intensely on your precious web site. And if they can't
> quickly find what they're looking for--they'll leave
> your site and go somewhere else.

A tired turn of phrase used by too many people in our business. My new
repsonse to people when they say to me is: Yeah? And? What if I give
them what they want and they still leave?

Andrei

28 Dec 2004 - 5:48pm
Listera
2004

Andrei Herasimchuk:

> What if I give them what they want and they still leave?

In one corporate boardroom once, I was holding a high-level design strategy
meeting where various VPs were vetting their expectations and fears.
Everyone wanted tests and focus groups done, not so much to understand what
users may want/need but to be reassured/guaranteed of success. One of them
asked the same question you posed almost in verbatim. His boss told him:
"They may, after all. [pointing at me] But isn't that why we pay him to
design." I knew then that the project was going to be a success.

Ziya
Nullius in Verba

28 Dec 2004 - 9:18pm
Alain D. M. G. ...
2003

--- Andrei Herasimchuk <andrei at designbyfire.com> a écrit :

> > This is why I believe that only User Observation
> > Testing is a valid means for determining true
> > usability.
>
> Is that a little like the only way to get "good" movies is through
> test
> screenings?
>

Even a grand master like Stankey Kubrick depended on test screenings
for his works of Art. During the first pre-gala showing to a general
public of 2001 A Space Odyssey, he watched the audience carefully and
noted the places where they seemed to get impatient, and for how long.
He then cut mercilessly all the sections which had been too long. He
called it taking care of the "fidget factor".

This guy was a genius, and he also was a terribly hard worker,
constantly pushing himself and others. But he did not hesitate to junk
all those precious minutes of film which had taken so much time to do,
to take care of that medium's version of usability.

And he did it after watching his users.

Alain V.

__________________________________________________________
Lèche-vitrine ou lèche-écran ?
magasinage.yahoo.ca

28 Dec 2004 - 10:47pm
Listera
2004

Alain D. M. G. Vaillancourt:

> And he did it after watching his users.

Yep, you gotta watch for those usage patterns:

"Struck mid-size tower with car going 25mph, propelling it 15-20 feet
forward. This causes damage to car but troublesome DVD drive finally ejected
jammed disc upon contact with pavement. Still worked but HDD reported
errors, and case wasn't attractive. Sold on eBay (with new case and HDD).
Beware of this computer if you find it on eBay."

<http://www.technologyreview.com/articles/04/12/wo_delio122404.asp?p=1>

:-)

Ziya
Nullius in Verba

28 Dec 2004 - 11:40pm
Steven Streight
2004

Web sites are made for users.

Except for purely artisitic or personal sites, web sites do not exist to please or win the approval of other designers.

Web sites exist to enable users to find information, perform a task, post a comment, or otherwise interact with the content or site operator/owner.

Web sites are not meant to be static works of art that seek passive admiration.

All that matters to users is: how do I accomplish what I need to accomplish here at this site?

Of course, if the online content or product offerings are not good, the raw usability of the site can't save the day.

Or if the design is ugly or inappropriate for the target audience, users won't even try to use the site, they'll think it lacks credibility and they'll bail out fast. (See ConsumerWebWatch credibility study by B.J. Fogg et al for more details).

We've had to ask users what they wanted to do at a web site.

We've found that we can't rely on what the client says they think users will want to do, nor what the designer hopes users would want to do at the site. What actual users have said they wanted to do is often unexpected.

Yes, the client also wants to accomplish certain things with a web site. But in most cases, if the user needs are not satisfied, if products are not ordered for example, or if users are not finding and using the information the client wants users to have, the site is a failure.

If I become a user and test the site, my research is very limited and biased. I'm generally more skilled at computer operation and navigation, and more knowledgable of the site, than a typical user will be.

Again, most sites are not made for usability analysts, but for a certain type of customer/user.

IMHO, users are not "at best 1/3 of the equation."

Even if the designer is happy and management is proud, if users can't do what they want to do, quickly and easily, the site is a failure. How much of the equation is that?

Yes, the design has to be more than usable. It also has to convey credibility, simplicity, and beauty.

"what if I give them what they want and they still leave?"

How do you know you gave them what they want? Without asking them? Without testing users? We know the limitations of surveys and other remote feedback.

If they still leave, it's probably due to a better product, or better content, or better usabilty, or more functionality, at another site.

There is only one "level" at which any web site "works"--and that's the level of its intended audience, the actual users.

Same as any other product: cars, televisions, computers, etc.

Only works of art are admired for their own non-functional sake.

Steven Streight
Web Usability Analyst/Content Writer

http://www.vaspersthegrate.blogspot.com
http://www.streightsite.blogspot.com
http://www.arttestexplosion.blogspot.com
EMAIL: vaspersthegrate at yahoo.com

---------------------------------
Do you Yahoo!?
Dress up your holiday email, Hollywood style. Learn more.

29 Dec 2004 - 12:14am
Andrei Herasimchuk
2004

On Dec 28, 2004, at 8:40 PM, Steven Streight wrote:

> Web sites are made for users.

Except in cases where they are made to make money for business as well
as being made for users. you might claim they are the same thing. In
truth, it's not quite that black and white.

> Except for purely artisitic or personal sites, web sites do not exist
> to please or win the approval of other designers.

Not exactly sure why that matters, or how you have come to believe that
designers believe this.

> Or if the design is ugly or inappropriate for the target audience,
> users won't even try to use the site, they'll think it lacks
> credibility and they'll bail out fast. (See ConsumerWebWatch
> credibility study by B.J. Fogg et al for more details).

Nielsen's site is one of the ugliest around and you've defended him in
the past. In fact, Nielsen's site is also one of the most inappropriate
for his audience. At least one would assume Nielsen would want to win
the hearts and minds of designers so they create better design, so he
would at least attempt some modicum of aesthetically pleasing visuals.

So which is it then?

> We've had to ask users what they wanted to do at a web site. We've
> found that we can't rely on what the client says they think users will
> want to do, nor what the designer hopes users would want to do at the
> site. What actual users have said they wanted to do is often
> unexpected.

Just because a user wants something, doesn't mean the design has to
make it happen just like they want it. The point you are missing, and
have missed often in the past, is how usability can inform design, not
how usability dictates design. That is where most usability folk lose
credibility or fail when they work in practice.

> If I become a user and test the site, my research is very limited and
> biased. I'm generally more skilled at computer operation and
> navigation, and more knowledgable of the site, than a typical user
> will be.

Again, I ask the question: If you make a chair, and never sit in it,
how on earth do you expect to design it well or make it better by
simply only watching others trying to sit in it or listening to what
they have to say about that experience?

> IMHO, users are not "at best 1/3 of the equation."

Like it or not, they are. The other 2/3? The needs of the business and
the viability of the technology. When any of those 1/3 are ignored, the
end result is mediocre design and mediocre products.

> Even if the designer is happy and management is proud, if users can't
> do what they want to do, quickly and easily, the site is a failure.
> How much of the equation is that?

Once you again, you are so consumed by your particular point of view,
you block out any hope of seeing the bigger picture with regard to what
is needed for good design to happen. Since I have been over this point
with far too many times in the past, I won't bore the list trying to
convince you once again.

> "what if I give them what they want and they still leave?" How do you
> know you gave them what they want? Without asking them? Without
> testing users? We know the limitations of surveys and other remote
> feedback.

Ok... I'll give you one example: The History feature in Photoshop. It
was the result of Mark Hamburg watching users and understanding how
multiple undo works in other programs. Users cried for multiple undo in
Photoshop. Mark instead gave them a variation that could somewhat
behave like multiple undo plus had an impact on a much larger scale for
those users who grokked what it did.

He didn't ask users what they wanted. He saw what you people did, and
knowing far more than they did abut the technology, gave them something
they could never possibly ask for. And now it's one of the best
features in the program for many users.

Again, this is when usability techniques inform design. Observation,
maybe asking them questions, or other usability or research techniques.
But in the end, it only informs. It doesn't dictate.

> If they still leave, it's probably due to a better product, or better
> content, or better usabilty, or more functionality, at another site.
>
> There is only one "level" at which any web site "works"--and that's
> the level of its intended audience, the actual users.

> Same as any other product: cars, televisions, computers, etc.
>
> Only works of art are admired for their own non-functional sake.

Do you believe the audience on this list are children? If not, then
please start addressing it beyond simple, almost borderline
pedantic-style commentary. Also, as a writer, I'm sure you'd understand
that single sentence paragraphs are both hard to read and come across
poorly in terms of tone. You might want to reconsider the usability of
your communication approach.

Andrei

29 Dec 2004 - 12:21am
Listera
2004

Steven Streight:

> Web sites are made for users.

That's decidedly NOT the case.

Users do not pay for web sites to be created. Business owners do. To part
users from their money. Designers work for and get paid by business owners.

> All that matters to users is: how do I accomplish what I need to accomplish
> here at this site?

And all that matters to business owners is: how do I get users to buy stuff
from me?

While those two goals can align, they don't always do. Designer's job is to
align those two interests.

> How do you know you gave them what they want? Without asking them?

While consulting users is often a good idea, one cannot rely on users to
design products. That's the designer's job.

Take the hottest-selling consumer digital product on the planet today: the
iPod. Do you really think users told Apple/Ive how exactly they wanted their
next MP3 player designed? Do you really think the *design* of the iPod was a
direct result of user 'wants'? Do you think users at focus groups/tests
jumped up and said they wanted the particular UI that made the iPod so
'usable'?

Great designs often lead users, not passively follow them. That's risk
management. Designers make bets and manage risks. User testing/validation is
but one part of risk management, which itself is not science.

> Same as any other product: cars, televisions, computers, etc.

If you consider highly-researched and tested design artifacts like shopping
malls, department stores, casinos, supermarkets, etc., you'll notice that
they are not designed for circulation efficiency. In fact, 'inefficient'
circulation is the design goal: pausing customers at certain locations,
enticing them to 'waste' more time on their premises, causing them to spend
less time where there are fewer opportunities for purchases, burying highly
sought spots (information, restrooms, wrapping, etc) in the back so that
customers have no choice but to circulate among the merchandise, and so on.
There are myriad studies and books on these.

One can't be a successful designer without fully understanding the
ramifications of who signs one's checks. To reach positions of influence and
decision making (so often discussed here), one needs to correctly parse that
client/user alignment proposition.

Ziya
Nullius in Verba

29 Dec 2004 - 12:21am
Andrei Herasimchuk
2004

On Dec 28, 2004, at 6:18 PM, Alain D. M. G. Vaillancourt wrote:

> Even a grand master like Stankey Kubrick depended on test screenings
> for his works of Art. During the first pre-gala showing to a general
> public of 2001 A Space Odyssey, he watched the audience carefully and
> noted the places where they seemed to get impatient, and for how long.
> He then cut mercilessly all the sections which had been too long. He
> called it taking care of the "fidget factor".

He didn't "depend" on them. He used them as one tool to inform his
movie-making process. (See my response to Steven on that point.) And if
you know anything about his movie making experiences, you'd also know
that things like A Clockwork Orange tested horribly, but is now
considered one of his masterpieces. There are other aspects to his
movies that he never altered, like his marketing approach, even though
it didn't pack them into the movies.

Considering many react to Kubrick as a "love him" or "hate him" kind of
director, to claim he listened to all his "users" is really quite
absurd. In the end result, he did his own thing, no matter what. He
preferred to try and please people with his movies, but he made the
movie he wanted to make, not the movie the "collective" wanted to see.

At least after Spartacus was released, which he personally despised due
to the studio system intent on making movies for the people.

> This guy was a genius, and he also was a terribly hard worker,
> constantly pushing himself and others. But he did not hesitate to junk
> all those precious minutes of film which had taken so much time to do,
> to take care of that medium's version of usability.

If you believe that Kubrick practiced "usability" in his methods in
making movies, at least as outlined in the way Steven is claiming in
his post, I'd have to say you're horribly over-stating the metaphor.
Kubrick did no such thing.

Andrei

29 Dec 2004 - 7:04am
Dave Malouf
2005

To me the question of where a designer falls in this debate is often related
to their work environment.

Whether purposefully or not, Andrei referenced Don Norman's 3 legged stool
in his reply earlier--you have the user, the business, and the technology.
Most companies that appear off-balance usually tend to think of the world as
one of these legs more than the other.

I know for myself, I understand what Andrei and Ziya are saying and totally
agree, I do understand that my mind goes in directions like Steven's and
Allain's because my current employer is so skewed at the moment towards
business followed by technology. Sometimes this causes my discourse with
designers to be a bit off, b/c I have the "arguments" ready for my work
environment. I wish I could get more practice with these arguments in this
or similar spaces, but it is difficult to preach to the choir. ;)

On that note, a few thoughts ...

The article for Kodak for me was a good one b/c it specifically spoke about
how changing their strategy was effected by user research and how that
change in strategy effected a key value statement--market share.

I have been struggling within my very business focused organization that is
also very successful in their market with what I will call bluntly a very
mediocre product. My struggle is to get the business owners to see value in
design instead of feature add-ons (or at least at the same time). Even this
article and Ziya's great example of the Apple stock ticker (Thanx Ziya) got
back replies of but that is product, what about service? Or those companies
don't have our customers.

Anyone have examples of this working in software like Enterprise Content
Management, Collaboration, or other Enterprise software where doing the user
research had a qunatifyable effect on some aspect of value? Reduced
support-call volume, increased win/loss ratios, increased marketshare,
increased stockvalue, increased analyst awareness, or higher analyst rating,
higher revenue, higher total profitability?

-- dave

-- dave

29 Dec 2004 - 1:59pm
Listera
2004

David Heller:

> that is product, what about service? Or those companies
> don't have our customers.

Aha. So their argument here is that your customers demand mediocrity?
They may be right...or wrong.

Do these same customers always eat at McDonald's, shop at Wal*Mart, drive
Pintos, etc? If they did, three years after its introduction, the iPod would
not be commanding a 90%+ market share, against all odds.

But there's the other side of the argument as well: in the fatter part of
the bell curve, people don't value design, quality, value, etc., as much as
other variables.

So this becomes a branding issue. For example, here are some two-word
branding propositions for companies we know about:

IBM -> complexity/solutions
DELL -> price/distribution
MSFT -> mediocrity/ubiquity
AAPL -> innovation/simplicity

When these companies venture outside their branding zone they usually lose
big time.

So is there money to be made by serving a commodity market? Sure. Is there
an opportunity cost being missed? Likely. Can you move your customers from
commodity to premium? Yes. Sony sold about 400 million Walkman units, always
commanding a premium price justified by design in a commodity market.

One needs to think through these market/competitive analyses to better gauge
the customer base, product reconfiguration possibilities, etc. As I said
before, design (just like marketing) is risk management. You consider
various data points and make bets. Users happen to be one part of this
complex equation. A designer can read all the Nielsen alert boxes as he can
and 'test' like there's no tomorrow, it won't amount to a hill of beans if
the product strategy was wrong.

It's strategy > architecture > design.

Ziya
Nullius in Verba

29 Dec 2004 - 3:24pm
John Vaughan - ...
2004

Listera said:

> Users do not pay for web sites to be created. Business owners do. To part
users from their money. Designers work for and get paid by business owners.

> And all that matters to business owners is: how do I get users to buy stuff
from me?

> While consulting users is often a good idea, one cannot rely on users to
design products. That's the designer's job.

> Great designs often lead users, not passively follow them. That's risk
management. Designers make bets and manage risks. User testing/validation is
but one part of risk management, which itself is not science

Cogent. Correct. And a necessary reminder/context for any discussions in this space.

Thanks

29 Dec 2004 - 4:40pm
Pradyot Rai
2004

David Heller <dave at ixdg.org>
> Anyone have examples of this working in software like Enterprise Content
> Management, Collaboration, or other Enterprise software where doing the user
> research had a qunatifyable effect on some aspect of value? Reduced
> support-call volume, increased win/loss ratios, increased marketshare,
> increased stockvalue, increased analyst awareness, or higher analyst rating,
> higher revenue, higher total profitability?

Yes.
I have personally involved in few enterprise applications leading
re-designed efforts via user-research/ethnographic field study for
slightly different reasons. The reasons are none of those quantifiable
as you mentioned, but are purely tactical.

In this example the plan to go out in the field and observe/talk to
users, understand processes, are purely geared to overcome the
partners obnoxious complaints about the usage/expectations of apps.
Also it was to understand the processes at partners end, so that we
can realign expectations. The idea of redesign is completely to bring
equilibrium between service partners, technological constraints and
usage of the app.

As Dave quoted, user is just one of the 3 pegs, others are business
partners expectations, technological constraints such as control and
flexibility on data, and last but not the least, user's satisfaction
should be maintained.

The expected goals are -- Increased business partner satisfaction,
managing product requirements efficiently, having more control on how
the application is used while allowing more control on the data to the
user/customer. Clearly the idea is not typically on of those
user-centered, however using every bit of UCD to undertake design and
analysis efforts made good sense.

The initial findings are that user loves the apps. Identified problem
areas are with Business Partners (their expectations, pricing,
processes/regulations, etc.) and with the issues with data
(technology). Read between the lines, this may seem unintuitive for --
why redesign when user has no complaints? But makes perfect sense from
the business tactical reasons (which I can't disclose for my client's
confidentiality reasons).

Now, somebody may ask, Am I paid to make the application difficult
(?), to which my answer will be -- Off course Not! …Go beyond it. I am
also paid to help business and technology perform as expected.

Prady

Syndicate content Get the feed