Why not ask for *opinions*? (was "The oldPlay/Pause toggle")

11 Aug 2006 - 6:12pm
8 years ago
1 reply
352 reads
jbellis
2005

I'd like to cast 100 votes in support of Christopher's argument. For every
100 requests that begin..."Does anyone know of a study..." how many times
does the list offer a high-fidelity response? I'd estimate less than 5%, for
reasons Christopher explains. This is a highly inventive and expanding field
and will be so for a long time. Chances are that long before empirical
studies isolate many issues, they will be resolved by "patterns" found on
the top 50 sites and apps.

Mark, your implication that testing should somehow displace expertise (if I
understood it correctly), confuses me. If testing were the
best path to answers, then we should be able to test our way to
profitability. The whole essence of "patterns" is that the same problem is
frequently tackled twice, no?

A more literal answer to the "Why..." question is "because people believe
that numerical evidence constitutes fact." Certainly in many domains the
relationship is unassailable. But in our domain, I think it's often
diminishing returns. (I like
to say that numbers are "numb-ers.") Let me put it this way: someone who
needs Fitts' Law to know that small, distant targets are hard to hit might
make a great manager, but is unlikely to make a good designer.

-Jack
www.WorkAtHomeWednesday.com
----- Original Message -----
From: "Mark Schraad" <mschraad at mac.com>

> Because heuristic evaluations (that means "in my professional opinion
> based on all of my experience"), though they may well solve some
> problems, are short cuts.

The same is true with a published study, if
> the constraints and conditions are not precisely the same. That is
> why it pays to test nearly every stage with USERS even though in
> inevitably slows the dev process and pisses off the business or
> product managers. Defining the problem is usually 90% of the work.
> Rarely do designers solve the exact same problem twice.
>
> Mark

Comments

11 Aug 2006 - 7:23pm
Mark Schraad
2006

Obviously, in a perfect world, we would all have available subjects
to test models on real users at critical stages. From my own
experience there are a variety of reasons that this is not the norm.
And from reading the posts here... it is not for most of us.
Expertise, like previous studies is contextual to the "defined"
problem. But at least previous studies are more evidentiary than "in
my experience. I was not trying to invalidate experience, just put it
in proper perspective.

All that being said, evaluating two roughly similar problems or
situations as the same, requiring the "same" remedy is dangerous. I
have seen very few cases where a solution was applied to a second
situation with a perfect match and optimal results. The truth be
told, the reason we do not more frequently research and test is
typically because of egos, budgets and deadlines, not because we
already have the answers as a result of our years of experience. In
my opinion, most designers - design for themselves. It takes a great
deal of discipline not to.

As for the numbers argument... testing usually indicates, as opposed
to proving anything. The designer still has to interpret... make
adjustments, and test again - and of course correlation is not
causality. As designers, we contribute greatly to the fact that 80%
of all new products fail. We need to overreach sometimes in order to
achieve reasonable goals. So I tend to advocate for user testing more
often than I ever really expect to get it. We all too often fool
ourselves with our vast experience and knowledge.

Mark

On Aug 11, 2006, at 7:12 PM, jackbellis.com wrote:

> [Please voluntarily trim replies to include only relevant quoted
> material.]
>
> I'd like to cast 100 votes in support of Christopher's argument.
> For every
> 100 requests that begin..."Does anyone know of a study..." how many
> times
> does the list offer a high-fidelity response? I'd estimate less
> than 5%, for
> reasons Christopher explains. This is a highly inventive and
> expanding field
> and will be so for a long time. Chances are that long before empirical
> studies isolate many issues, they will be resolved by "patterns"
> found on
> the top 50 sites and apps.
>
> Mark, your implication that testing should somehow displace
> expertise (if I
> understood it correctly), confuses me. If testing were the
> best path to answers, then we should be able to test our way to
> profitability. The whole essence of "patterns" is that the same
> problem is
> frequently tackled twice, no?
>
> A more literal answer to the "Why..." question is "because people
> believe
> that numerical evidence constitutes fact." Certainly in many
> domains the
> relationship is unassailable. But in our domain, I think it's often
> diminishing returns. (I like
> to say that numbers are "numb-ers.") Let me put it this way:
> someone who
> needs Fitts' Law to know that small, distant targets are hard to
> hit might
> make a great manager, but is unlikely to make a good designer.
>
> -Jack
> www.WorkAtHomeWednesday.com
> ----- Original Message -----
> From: "Mark Schraad" <mschraad at mac.com>
>
>> Because heuristic evaluations (that means "in my professional opinion
>> based on all of my experience"), though they may well solve some
>> problems, are short cuts.
>
> The same is true with a published study, if
>> the constraints and conditions are not precisely the same. That is
>> why it pays to test nearly every stage with USERS even though in
>> inevitably slows the dev process and pisses off the business or
>> product managers. Defining the problem is usually 90% of the work.
>> Rarely do designers solve the exact same problem twice.
>>
>> Mark
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org

Syndicate content Get the feed