When those who know little about Usability offerUsability Services

31 Oct 2007 - 3:16am
6 years ago
3 replies
374 reads
Peter Boersma
2003

Alan asked:
> [..] However after talking to them for a while it becomes clear that
> they really don't know what Usability is [..]
> So I ask you: Is this the price we pay for growing popularity of
> Usability and User-Centered Design?

Yes, and we should fight it with (drumroll...) discount usability methods.
Jakob Nielsen wrote an article on Guerrilla HCI in 1994:

Guerrilla HCI: Using Discount Usability Engineering to Penetrate the Intimidation Barrier
"I used the term "guerrilla HCI" in the title [..] because I believe that simplified usability methods can be a way for a company to gradually build up its reliance on systematic usability methods, starting with the bare minimum and gradually progressing to a more refined lifecycle approach."
(http://www.useit.com/papers/guerrilla_hci.html)

If these simplified methods come to the people with less knowledge in a well-documented and illustrated way, there is hope that they won't mess it up and bring our practice down again.

And, since this is IxDA and not CHI-Web, I believe each field under the UX umbrella should have its own set of simplified methods, as I explained for IAs on my blog in "A piece of IA pie: little, micro, lite or guerrilla?" (http://www.peterboersma.com/blog/2005/02/piece-of-ia-pie-little-micro-lite-or.html or http://tinyurl.com/c7tw4).
IxDA.org's Resource Library (http://resources.ixda.org/) and the IA Institute's Learning IA section (http://www.iainstitute.org/en/learn/) should be examples of collections of these well-documented, simplified methods.

Peter
--
Peter Boersma | Senior Interaction Designer | Info.nl
http://www.peterboersma.com/blog | http://www.info.nl

Comments

31 Oct 2007 - 6:54am
Chauncey Wilson
2007

One of the issues that should be considered in discount usability
engineering is that these discount methods still need some rigor.
Take the discount method of "think-aloud usability testing" (TAUT),
one of the prominent discount methods. Doing a good TAUT requires
careful attention to the choice of tasks that you use (or if you let
the user define the task, a careful interview to understand what
he/she would like to do). Choosing tasks and deciding on the order of
tasks is a single line in a checklist, but it requires some thought
about biases of stakeholders, the appropriate granularity of the task,
the degree to which the task reflects the real world in terms of it
complexity, data inputs, etc., whether one tasks will affect the next
task, etc. What are the criteria for choosing tasks? You can choose
tasks by frequency of use, criticality (rare, but devastating), new
features that marketing will advertise about heavily, and many other
criteria. The choice of tasks is even an ethical issue in that you
could choose tasks that make the team look good (I've seen this when I
was a lab director and rented out lab out to colleagues who wanted to
please their managers or vice presidents).

Oh, and each task should be tied to both a business goal and user
experience goal which is often not done by those with limited
experience.

Thinking-aloud (which can be used for a variety of methods) in a
usability test can be affected greatly by the verbal and non-verbal
prompting and interview style of the facilitator. Facilitating a
think-aloud usability test is often portrayed as a simple discount
method, but the reality is that it is a complex interaction that
requires some rigor and constant self-reflection. Studies at the
University of Washington have shown that the variability in how
facilitators conduct sessions is extreme.

Analyzing the results of think-aloud usability testing seems easy,
just list the problems that you hear and see, but extracting problems
is difficult. If I show a 10-minute tape of a typical session of
something that is a bit complex the results will vary considerably.
There are few places where people are trained in the art and science
of taking good notes (or in the case of automated logging - marking
the places where you then have to take good notes). Note-taking is
one of the dirty secrets of our field. People often see the same
event in different ways and inter-rater reliability is often low. One
of the issues here is that it isn't always clear when a problem
starts. Theoretically, a person might do something that does not
prompt a "problem alert", but later something happens because of this
unnoticed thing and a visible problem occurs. Another subtle issue
that I've seen is that some people take notes on what they hear (the
thinking aloud) while others note what they see on the screen.
Sometimes a subtle hesitation or oscillation of the mouse between a
few links or menu items might be a problem, but in my experience, few
people will write that down. Since the 1980s, when nearly everyone
watched hours of videotape, there has been a general trashing of
videotape viewing because it takes too long. Well, yes, verbatim
videotaping is probably not cost effective, a quick scan of tapes,
especially with complex products will reveal things that people
missed, sometimes serious issues that are not obvious when a person is
focused intently - sometimes the issues might show up across task, but
there is often not a good way to connect that quickly in the
notetaking. A colleaue and I did a little experiment where he took
notes during a session that last an hour on a complex network
performance product. I watched a tape of the session and listed the
problems that I saw and the difference between our two lists was
significant. I saw about 50% more problems - including some serious
ones.

The issue of what constitutes a "usability problem" is hard because
things that might be a problem for one user group might be a positive
thing for another group so even defining a problem requires some
reflection and consideration of the participants.

Oh, and then there is the severity rating issue - ratings of severity
by different people of the same thing are often quite unreliable. One
reason for the unreliability is that there is often limited training
on rating severity with examples and discussions of context. Much of
the research on this topic shoves a 1-4 or 1-5 or 1-10 scale at
participants with no real training or background and those who use
scales in the real-world fail to work on a convergence of
understanding in some semi-structured way.

So, even a "discount" or "guerilla" requries some rigor and training
and reflection (like having usability teams review a tape together to
help each other understand why they listed something as a problem or
not), and practice (there are very few HCI or Usability programs where
people get extensive training on the basic types of interviews -
structured, semi-structured, and unstructured interviews). Moreover, a
good HCI/UUX/IxDA plan should consider a mix of discount and
full-featured methods (for example, conducting field interviews for
the next version while conducting simplified thinking-aloud and user
interface inspections on the current product).

Discount methods still need rigor, practive, review, and reflection.

Chauncey

On 10/31/07, Peter Boersma <peter at peterboersma.com> wrote:
> Alan asked:
> > [..] However after talking to them for a while it becomes clear that
> > they really don't know what Usability is [..]
> > So I ask you: Is this the price we pay for growing popularity of
> > Usability and User-Centered Design?
>
> Yes, and we should fight it with (drumroll...) discount usability methods.
> Jakob Nielsen wrote an article on Guerrilla HCI in 1994:
>
> Guerrilla HCI: Using Discount Usability Engineering to Penetrate the Intimidation Barrier
> "I used the term "guerrilla HCI" in the title [..] because I believe that simplified usability methods can be a way for a company to gradually build up its reliance on systematic usability methods, starting with the bare minimum and gradually progressing to a more refined lifecycle approach."
> (http://www.useit.com/papers/guerrilla_hci.html)
>
> If these simplified methods come to the people with less knowledge in a well-documented and illustrated way, there is hope that they won't mess it up and bring our practice down again.
>
> And, since this is IxDA and not CHI-Web, I believe each field under the UX umbrella should have its own set of simplified methods, as I explained for IAs on my blog in "A piece of IA pie: little, micro, lite or guerrilla?" (http://www.peterboersma.com/blog/2005/02/piece-of-ia-pie-little-micro-lite-or.html or http://tinyurl.com/c7tw4).
> IxDA.org's Resource Library (http://resources.ixda.org/) and the IA Institute's Learning IA section (http://www.iainstitute.org/en/learn/) should be examples of collections of these well-documented, simplified methods.
>
> Peter
> --
> Peter Boersma | Senior Interaction Designer | Info.nl
> http://www.peterboersma.com/blog | http://www.info.nl
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://gamma.ixda.org/unsubscribe
> List Guidelines ............ http://gamma.ixda.org/guidelines
> List Help .................. http://gamma.ixda.org/help
>

31 Oct 2007 - 8:54am
Mark Schraad
2006

On Wednesday, October 31, 2007, at 09:41AM, "Todd Zaki Warfel" <lists at toddwarfel.com> wrote:
>Ah yes, the side dish companies.
>
>On Oct 30, 2007, at 5:47 PM, Adam Connor wrote:
>
>> So I ask you: Is this the price we pay for growing popularity of
>> Usability and User-Centered Design?
>
>Yes. That's what happens. An executive or sales person reads about
>usability, ethnography, or personas in the latest issue of Business
>Week. They see they can monetize it and decide they should start
>offering this service they know nothing about. After all, how hard can
>it really be?
>
>They aren't a firm specializing in research, ethnography, usability,
>personas, etc. They are a firm that offers these services as a side
>dish.
>
>The upside is that it raises the visibility. The downside is that it
>decreases the value of the service by diluting the quality of work.

That depends. The expertise and experience is not intrinsic to the firm, but to the team members. I have seem many firms go out and hire the talent to do this right. Folding new skill sets into the company's offering is not always home grown. I think it can be an indicator of a firm in touch with what its clients need and with the profession. If new tools are making for better results amongst your competitors... is it wiser to ignore them or to learn and adapt?

Mark

2 Nov 2007 - 3:12pm
dszuc
2005

Hi Jay:

This is where running more events on Usability in a city and continually
educating the market can help.

UPA Singapore have recently re-ignited their Chapter --
http://www.upasingapore.org/

Rgds,

Daniel Szuc
Principal Usability Consultant
Apogee Usability Asia Ltd
www.apogeehk.com
'Usability in Asia'

The Usability Kit - http://www.theusabilitykit.com

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of Melvin
Jay Kumar
Sent: Thursday, 1 November 2007 10:45 AM
To: Adam Connor
Cc: discuss at ixdg.org
Subject: Re: [IxDA Discuss] When those who know little about Usability
offerUsability Services

This issue is ooohhh so common in Singapore and the region.

I know of so many web design firms who now have with a wave of a hand
suddenly have a UCD practice and offer all services from usability testing
to analysis to taxonomy development etc...etc....etc...

This is happening especially here because, select companies, Government
agenices have start to include "usability" as in their RFP's.

So all these web design firms cannot submit their proposal without a
"usability" component and thus they all have UCD practices in their firm.

I know of a company that recruited people who have no experience or
knowledge in these areas send them for a "usability analyst" 3 three seminar
done by one of the firms you'll definltely know, and then bham! they are
usability and UCD experts.

They know understand every compoment from interaction design to interface
design tc...etc...etc...

That is not the worst thing, the worst is that, to get the job, they reduce
their rates so low and promise their clients the world.....its just
ridiculous sometimes.

I could talk about it, but its just funny...ever heard of usability testing
for 49.99? ;P

Regards,

Jay Kumar

On 10/31/07, Adam Connor <adam at littlegreentoaster.com> wrote:
> In the past few months I've run into quite a few designers, studios,
> developers, etc. who have begun offering Usability services to their
> clients. However after talking to them for a while it becomes clear
> that they really don't know what Usability is (they've never conducted
> a usability test, are unfamiliar with many common best practices,
> don't have any training, and so on and so forth...). Instead it would
> seem that they have picked up on the importance of usability and have
> included it in their services to better promote themselves to clients.
>
> I've also noticed that most of these individuals I've spoken to
> mistake Usability for a set of logical conventions. Something like
> "button A goes in the top left corner because thats where it would
> logically go..." or "module B should look like this because most sites
> do it that way..." They forget that while many usability best
> practices may conform to logical thinking, the user's expectations are
> what matter and if that differs from what you believe is logical,
> well... you need to make adjustments.
>
> So I ask you: Is this the price we pay for growing popularity of
> Usability and User-Centered Design?
>
> Have you noticed other groups doing similar things?
>
> Does it bother you as much as it does me?
>
> -adam
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah* February 8-10, 2008 in
> Savannah, GA, USA Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org Unsubscribe
> ................ http://gamma.ixda.org/unsubscribe List Guidelines
> ............ http://gamma.ixda.org/guidelines List Help
> .................. http://gamma.ixda.org/help
>
________________________________________________________________
*Come to IxDA Interaction08 | Savannah*
February 8-10, 2008 in Savannah, GA, USA Register today:
http://interaction08.ixda.org/

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org Unsubscribe ................
http://gamma.ixda.org/unsubscribe List Guidelines ............
http://gamma.ixda.org/guidelines List Help ..................
http://gamma.ixda.org/help

Syndicate content Get the feed