IxD Metrics

13 Nov 2006 - 4:57pm
7 years ago
24 replies
877 reads
LukeW
2004

Hi all,
Forgive me if this has been talked about on the list and I missed
it. But... I've had several discussions with folks recently about the
evaluation of "good" interaction design. One side of the debate seems
to side with efficiency metrics: completion times, error rates,
reduced interruptions, etc. The argument goes that good interaction
design can be measured by the user's ability to perform a task as
quickly and efficiently as possible.

The other side side of the discussion thinks there's more to
successful interaction design than transparency (getting out of the
way so people can perform tasks). Here engagement, utility, delight,
and narrative come into play. That's not to say task completion isn't
important, its just that it doesn't paint a complete picture of
successful IxD.

Any thoughts? What's the criteria by which you judge a "good"
interaction design? Or is it so context-specific that across the
board metrics don't apply?

thanks~

::
:: Luke Wroblewski -[ www.lukew.com ]
:: Principal, LukeW Interface Designs
:: luke at lukew.com | 408.879.9826

::
:: Blog: http://www.lukew.com/ff/
:: Book: http://www.lukew.com/resources/site_seeing.html
:: Bio: http://www.lukew.com/about/luke/
::

Comments

13 Nov 2006 - 6:03pm
Sergio
2006

Luke,

I believe the short answer to your questions is that the more value an
interaction design creates for the client, the better the design. In
other words, web sites have a number of different ways of generating
value for their owners (e.g. direct sales, ad revenue, brand building,
supporting sales, driving donations), and the ultimate litmus test for
any design is how it affects the revenue stream. For example, I have
seen a number of research papers that claim that link rich pages are
more usable. They have solid data to back the hypothesis including
reduced errors, task completion times, etc. Despite this, Marissa Mayer
of Google stated something completely contradictory and shocking at the
Web 2.0 conference in San Francisco last week--increasing the number of
search results on a page severely limited user interaction and ad
revenue according to a study Google performed over a long time span. So
what does it matter if a design improves task completion times, limits
errors, or makes users feel fuzzy inside, if that same design reduces
their revenue, both short and long term, by about 20 percent, as was
the case in Google study? A proper multivariate regression analysis
applied to a robust data set gathered over a sufficiently long period of
time will be able discern what will be the combined long term and short
term ramifications of changes to a given design.

(For a slightly better write-up of Mayer's talk see:
http://www.paradymesolutions.com/articles/marissa-mayer-of-google-speed-good-ajax-not-so-good/
)

Regards,
Sergiusz Paluch

P.S. Just for the record, I do support making users feel fuzzy inside
because I believe that it generally leads to improved customer loyalty.
And who doesn't like warm and fuzzy?

13 Nov 2006 - 9:11pm
Kim Goodwin
2004

LukeW wrote:
" What's the criteria by which you judge a "good" interaction design?"

A few of us at Cooper were kicking this question around with Hugh
Dubberly several years ago. We came up with 4 criteria we felt applied
to all sorts of design, not just interaction. Good design is:

Ethical
Ideally, a designer's first rule is the same as a physician's: do no
harm. In the case of surgery tools, car dashboards, and airplane
cockpits, this is obvious: don't kill people. However, even business
software can do harm by wasting a user's time, leading to errors,
contributing to repetitive stress injuries, or just making people feel
dumb. Of course, there are likely a few situations where this principle
is challenging, such as a missile guidance system---if you had to design
something of that sort, the principle might have to be interpreted as
"minimize harm by making darn sure you hit the intended target."

Purposeful
Good designs help users accomplish their goals. (Note that how someone
wants to *feel* can also be a goal---we call it an experience goal to
distinguish it from an end goal). In many situations, efficiency is an
important goal, so aspects of the design that reduce work (motor,
memory, cognitive, visual) are beneficial.

Pragmatic
Good designs can be built within the appropriate business parameters
(cost, time, etc.) and accomplish the appropriate business goals (more
online transactions, lower support costs, etc.) A design that's great in
every other respect won't see the light of day if it's too expensive to
build.

Elegant
A good design is the simplest complete solution.

14 Nov 2006 - 12:14am
Robert Reimann
2003

Luke,

I think to some extent you've answered your own question. While all
the elements you've mentioned would contribute to what I would call
good IxD, the relative importance is I believe a matter of context.
Desktop productivity software is in most cases going to weight task
efficiency over, say, "delightful" screen transitions: what might be
fun in a casusally used application may become overbearing in an
application used most of the day to accomplish mission critical tasks.

This is where the use of tools like personas become so important: the
most important element of "good" IxD IMHO is its appropriateness to
the context, and the extent to which it reflects and addresses the
goals, needs, and motivations of its users.

And, as Kim and Sergio mention, any design in the end needs to be
measured by how well it achieves the pragmatic goals of those who
commision it, and needs to do so in a manner that doesn't weigh too
heavily on the designer's conscience.

Robert.

--
Robert Reimann
President, IxDA

Manager, User Experience
Bose Corporation
Framingham, MA

On 11/13/06, LukeW <luke at lukew.com> wrote:
> Hi all,
> Forgive me if this has been talked about on the list and I missed
> it. But... I've had several discussions with folks recently about the
> evaluation of "good" interaction design. One side of the debate seems
> to side with efficiency metrics: completion times, error rates,
> reduced interruptions, etc. The argument goes that good interaction
> design can be measured by the user's ability to perform a task as
> quickly and efficiently as possible.
>
> The other side side of the discussion thinks there's more to
> successful interaction design than transparency (getting out of the
> way so people can perform tasks). Here engagement, utility, delight,
> and narrative come into play. That's not to say task completion isn't
> important, its just that it doesn't paint a complete picture of
> successful IxD.
>
> Any thoughts? What's the criteria by which you judge a "good"
> interaction design? Or is it so context-specific that across the
> board metrics don't apply?
>
> thanks~
>
>
> ::
> :: Luke Wroblewski -[ www.lukew.com ]
> :: Principal, LukeW Interface Designs
> :: luke at lukew.com | 408.879.9826
>
> ::
> :: Blog: http://www.lukew.com/ff/
> :: Book: http://www.lukew.com/resources/site_seeing.html
> :: Bio: http://www.lukew.com/about/luke/

14 Nov 2006 - 9:26am
Marc Resnick
2006

Luke,

I would have to agree with Robert and add to his points. It depends
largely on the business model of the organization creating the system. I
actually think the design process in your book (free plug for you - I
recommend it to everyone!!) is a good start. I expand on this in a
soon-to-be-released book called "How Ergonomics Works" (free plug for me
:-D) for which I am contributing a chapter on the user-centered design
process. It starts with understanding the business model. Some companies
focus on the high end, which means delight, engagement, customer loyalty.
Some focus on efficiency (i.e. Dell) which would suggest a more spartan,
time-focused system. I would disagree that it is best to try to do it all.
Focusing on the key competitive advantages of the company and the
requirements of the target market segments is key.

But you can't ignore any of them either. There are minimum barriers to
entry for every metric that you need to at least meet. You can get these
benchmarking your competition. And then you spend your resources
maximizing the ones you want to focus on and that differentiate you from
the rest.

Marc Resnick
Usability Solutions
Miami, FL
(305) 348-3537
resnickm at fiu.edu

>> I've had several discussions with folks recently about the evaluation of
"good" interaction design . . .

14 Nov 2006 - 10:00am
jbellis
2005

Based substantially on my sense of what I see in Luke's site and his sample
chapter on the "personality" of sites, I hear in Luke's question, something
different from the other replies. To me Luke's question boils down to the
utilitarian vs. all of the other things, which I suggest might be
categorized as "sensibilities."
========
Utilitarian:
I think that all of the things mentioned in Robert and Kim's letters, except
"delightful" transitions and "how someone wants to feel", were actually
utilitarian. Every software effort should deliver them.

Sensibilities:
Components that appeal to the five senses: visual beauty (beyond the basic
HTML defaults), entertaining sound (beyond the proper utilitarian sounds),
touch/smell/taste (which are only achieved through design slight-of-hand).
Components that appeal to other mental senses: -of humor, of curiousity
(mystery), -of social belonging, -of worth, etc.
========

That sensibilities are harder to measure is a key factor in the discussion.
(I've observed over the years that the more difficult something is to
measure, the more important it is at least to me, both as work and interest.
Conversely, once something is readily measurable, I move on.) But ultimately
I think the question boils down, not to good vs. bad (surely we accept that
aesthetics in many contexts are measurably valuable by now, right?), nor to
definition of terms (Ix certainly includes many sensibility components, even
if not in every context) but to...

DIVISION OF LABOR!

Luke happens to be at the tectonic-plate-edge of the sensibility components,
whereas others barely know that the ground is moving. Yet both can claim to
labor in Ix. The fact that folks don't measure, let alone value, what they
can barely see should be no surprise.

Consider two sites for the most mundane of applications, timesheets. One is
raw HTML (black, white, Times Roman, gray borders see
http://usabilityinstitute.com/beforeandafter/justbefore.htm). The other,
while being just as fast and fault tolerant because it was designed by
experts, is drop-dead gorgeous, has humorous messages, uses
massively-multiparticipant concepts (incentives for timely submission, an
increasingly decorated likeness of your character on the home page), and
plays random selections of your favorite music in the background. Is someone
really going to argue that the benefits of the second are not Ix??? Does it
help the discussion to relegate those aspects as Ux? Not to me... it's just
division of expertise.

www.jackbellis.com

----- Original Message -----
From: "LukeW" <luke at lukew.com>

> evaluation of "good" interaction design. One side of the debate seems
> to side with efficiency metrics: completion times, error rates,
> reduced interruptions, etc. The argument goes that good interaction

> The other side side of the discussion thinks there's more to
> successful interaction design than transparency (getting out of the
> way so people can perform tasks). Here engagement, utility, delight,
> and narrative come into play.

15 Nov 2006 - 11:12am
LukeW
2004

Hi Sergiusz,
Thanks for the thoughts. I guess I should clarify that in particular
I was referring to an interaction design in relative "isolation". The
most obvious example is an IDE or a design pattern library. Where
there are a number of multi-state widgets available for designers or
developers to utilize in their product designs. Clearly these need to
be mapped to business goals and user needs when applied to an actual
product solution, but when developing the components (the building
blocks, I guess), what's the most appropriate way to measure if the
interaction is "good".

Let me give a very concrete example. Bill Scott recently documented
his creation of a carousel component. Its a relatively simple
interaction with a few states. Not to cast any doubt on Bill's IxD
skills :), but how do we judge the "goodness" of this component? Is
it strictly effectiveness as I alluded to in my first post (being
able to accomplish a task, as few errors as possible, fast completion
rate) OR is engagement, for example, a more effective metric (how
compelling the interaction is, any kind of wow or pleasure associated
with using it, etc.)?

OR can we not judge this component's interaction design as a stand-
alone entity. Do we always have to consider it in context of a
product solution (where there are the types of value generation
metrics you outlined: sales, revenue, brand, etc.) in order to
determine how "good" it is?

thanks~

On Nov 13, 2006, at 3:03 PM, Sergio wrote:

> I believe the short answer to your questions is that the more value
> an interaction design creates for the client, the better the
> design. In other words, web sites have a number of different ways
> of generating value for their owners (e.g. direct sales, ad
> revenue, brand building, supporting sales, driving donations), and
> the ultimate litmus test for any design is how it affects the
> revenue stream.

::
:: Luke Wroblewski -[ www.lukew.com ]
:: Principal, LukeW Interface Designs
:: luke at lukew.com | 408.879.9826
::

15 Nov 2006 - 11:24am
LukeW
2004

Thanks for the thoughts Robert, I guess I should have been a bit
clearer in my original question. I was in particular focused on an
interaction "out of context". Like the carousel component I mentioned
in my last email. Seems there are a few meta-points to address to
this discussion:

1) Is a widget or component or design pattern an "interaction design"
in of itself? Or is it simply a "raw material" to be designed with?
2) Can interaction design be measured out of context? Can you
determine the "best" or "good" solutions without the context of use
being known? To go back to the carousel example: is this a good
interaction design despite what Web page or Desktop application it
gets used in?
3) What are the basic parameters of context? Is the environment in
which it gets used (a Web browser) enough? Or do we need the "goals,
needs, and motivations of users" to really understand context and
thereby evaluate any design?

I'm mostly throwing these questions out there for the sake of
discussion. I don't actually have an answer formed :)

On Nov 13, 2006, at 9:14 PM, Robert Reimann wrote:

> This is where the use of tools like personas become so important: the
> most important element of "good" IxD IMHO is its appropriateness to
> the context, and the extent to which it reflects and addresses the
> goals, needs, and motivations of its users.
>
> And, as Kim and Sergio mention, any design in the end needs to be
> measured by how well it achieves the pragmatic goals of those who
> commision it, and needs to do so in a manner that doesn't weigh too
> heavily on the designer's conscience.

::
:: Luke Wroblewski -[ www.lukew.com ]
:: Principal, LukeW Interface Designs
:: luke at lukew.com | 408.879.9826
::

15 Nov 2006 - 11:45am
Dave Malouf
2005

LukeW wrote:
> 1) Is a widget or component or design pattern an "interaction design"
> in of itself? Or is it simply a "raw material" to be designed with?
>
My quick answer is No. Designs of all types, interaction or otherwise,
are for specific contexts.
Now a context can be pretty darn generalized, like an enterprise
application, whose design needs to be flexible enough to work within
many specific contexts which is why these designs require so much
configurability.

A widget though, like the carousel example, in my mind is not an
"interaction design", until it is placed within the context of a user flow.

> 2) Can interaction design be measured out of context? Can you
> determine the "best" or "good" solutions without the context of use
> being known? To go back to the carousel example: is this a good
> interaction design despite what Web page or Desktop application it
> gets used in?
>
Even if it could be measured out of context, to what end? Are you trying
to evaluate various solutions against each other to determine which ones
should be pursued or thrown away?

while this might lead to expediency and a perceived efficiency, I'm
afraid that this will also lead to missing opportunities through the
design process itself, which should allow for all patterns to be in play
through exploration and then evaluation is added on top of those
exploration artifacts.

And is the Carousel a "widget" or a "pattern" and how do we tell the
difference? and can one be evaluated and measured and the other not?

> 3) What are the basic parameters of context? Is the environment in
> which it gets used (a Web browser) enough? Or do we need the "goals,
> needs, and motivations of users" to really understand context and
> thereby evaluate any design?
>
>
the short answer is "yes!" ... context is about gestalt of use, so it
requires a good understanding of "everything" in order to understand the
context. But I'm betting that for priority sake we can concentrate on
specific aspects of the context that are more relevant to use than
others, but I would bet these would change depending on the design
problem/solution.

--

David Malouf
Vice President
dave(at)ixda(dot)org
http://ixda.org/
http://synapticburn.com/

AIM: bolinhanyc // Y!: dave_ux //
MSN: hippiefunk(at)hotmail.com // Gtalk: dave.ixd(at)gmail.com

15 Nov 2006 - 11:42am
Austin Govella
2004

I've been trying to define a set of metrics along the same lines. So
far, I have a set of metrics to measure different aspects of the
implemented pattern. They're meant to be general, so we can get along
to measuring without worrying about the specifics:

1. Usable: combines completion time, error rate, learnability, etc.

2. Accessible: device portability and how well it works without css or
js dependencies

3. Users like it: a simple rating of how much they do or don't enjoy the widget

4. We like it: how much is it preferred by us (dev, design, eng,
editorial, bus, etc.)

We're still in the design stages and haven't implemented these
metrics, but determining how much users enjoy a given widget is
important. And I think you can measure as stand alone entities. It's
the designers job at the end to make sure they combine patterns in a
way that accentuates their strengths and doesn't diminish them. But
focus requires people to always use a single widget in isolation.
(Several elements may compete for attention, but we can only give one
element our attention at a time.)

(Also, as a side note: we're generally disinguishing between a design
pattern and a widget which is an implementation of the pattern. For
example, tabbed navigation is a design pattern, but you may have
several widgets that use tabs but that look or behave differently:
tabbed global nav, in-page tabs, tabs on the side, etc.)

--
Austin Govella
Thinking & Making: IA, UX, and IxD
http://thinkingandmaking.com
austin.govella at gmail.com

15 Nov 2006 - 11:54am
Dave Malouf
2005

I want to clarify/question something here:

Is design about "measurement"?
I know there are "design awards" and such, which implies there is a
differentiation of value.
But "measure" implies quantification as opposed to qualification and I
think of designs as having qualities that are 'critique-able'.

Now, I realize that there are "results" to designs and these results are
definitely measurable and are probably the best indication of overall
"success" of a design, but I'm not sure that "success" is really the
final arbiter of "quality" of a design.

-- dave

15 Nov 2006 - 12:21pm
Dan Saffer
2003

Just tossing this in to the mix:

In order to determine the "success" of a design (although one could
argue that only time is the final arbiter of that), you need to have
some sort of baseline metric to compare the new design against. This
can (ideally) be quantitative (drop-off rates, adoption rate,
customer satisfaction survey data, log analysis data, sales figures,
etc.) or qualitative (interviews with customers, say).

This assumes, however, you have something to test against. If there
isn't an existing product to establish a baseline, this is a trickier
problem. You might have to do some A/B testing ("Try this page with
this widget. Now try it with this one.") to determine the relative
"goodness" of a design.

Now, granted, you can break other features of a product with a fix of
something else, but that's another thread...

Dan

15 Nov 2006 - 12:21pm
Todd Roberts
2005

"OR can we not judge this component's interaction design as a stand-
alone entity. Do we always have to consider it in context of a
product solution (where there are the types of value generation
metrics you outlined: sales, revenue, brand, etc.) in order to
determine how "good" it is?"

It seems as though there is an isolation->contextual continuum against which
the quality of an interaction widget (defined as a discrete piece of
interaction) can be measured that roughly corresponds to the trio of
usability, usefulness and delight, and the metric used depends on your goals
for measurement.

On the isolated end, there is a technical question. Does it do what it's
supposed to do in a usable and efficient way? E.g. how well does the
carousel support scrolling linearly through a set of objects.

In the middle, there is a general context/design pattern question. What
types of problems does this widget address, and how well does it address
those problems? Is it useful in this situation?

At the specific context end, the desirability issue comes in. How much does
the interaction widget add to the desirability and experience of the
product? How well does it integrate with the other interactions in the
product? Is there a better widget that addresses the same problem but would
add more to the whole?

Each of these would be valid aspects to measure depending on what you are
trying to do. There's no point in sticking with an efficient, usable widget
when it just doesn't work in a specific context for whatever reason. It's
probably necessary in most cases to move up and down the measurement
continuum at different stages of the project.

When creating a widget, you could look at it in isolation. Does it
efficiently and usably do what it is technically supposed to do? When
trying to solve a design problem, look at the general context. In this class
of problems, does this type of interaction tend to work well? When
evaluating a product, look at the specific context. How does the widget
impact the experience?

Feedback may influence decisions made at different levels of specificity.
For example, a poor fit in the specific context could lead to a change in
the technical implementation that creates a new interaction widget.

15 Nov 2006 - 12:26pm
Nelson RodrĂ­gue...
2006

Hi Luke,

LukeW wrote:
> 1) Is a widget or component or design pattern an "interaction design"
> in of itself? Or is it simply a "raw material" to be designed with?

I think it isn't. It might be an interface element, a component, just a
widget. I believe that in order to call it interaction design it must be
part of a flow or a more complex context of interaction. It is at the
same level as a radio button or a text field.

> 2) Can interaction design be measured out of context? [...]
> 3) What are the basic parameters of context? Is the environment in
> which it gets used (a Web browser) enough? Or do we need the "goals,
> needs, and motivations of users" to really understand context and
> thereby evaluate any design?

I think a good model to document the "appropriateness" or usefulness of
a widget (or even patterns) should be addressed in a way similar to what
you can find in "Research-Based Web Design & Usability Guidelines" [1]:
as a set of relative value and documented evidence based on research or
other measurable indicators.

[1] http://www.usability.gov/pdfs/guidelines.html

--

regards,

------------------------------------------------------------
Nelson Rodríguez-Peña A.
Usabilidad, Accesibilidad, Arquitectura de Información,
Diseño de Interacción
blog: http://www.webstudio.cl/blog/
------------------------------------------------------------

15 Nov 2006 - 12:40pm
Austin Govella
2004

On 11/15/06, Dave Malouf <dave at ixda.org> wrote:
> Is design about "measurement"?
> I know there are "design awards" and such, which implies there is a
> differentiation of value.
> But "measure" implies quantification as opposed to qualification and I
> think of designs as having qualities that are 'critique-able'.

Just keeping us aware of our assumptions...

The assumtion is that value is about measurement, but it's not. Value
can be quantified or qualified, neither of which are discrete actions
with discrete outputs. Measure assumes something is discrete, so in
that case I agree: There's very little that is discrete about design.

Measurement, as a way of thinking about the value of design tries to
concretize and make discrete something that is inherently indiscrete.
This is obviously a bad way to go, winding up with such indiscretions
as Nielsen's suggestion that the entire web should look the same.

So discussing value leaves us with two better options: we can quantify
or qualify. Even then "quantification" suggests there's some unique
concrete quality to quantification when it's really best described as
a statistically significant qualification.

I like the idea of a critique because it priveleges everyone's
subjective qualification of a design, regardless of their specialty or
expertise.

> I'm not sure that "success" is really the
> final arbiter of "quality" of a design.

Success is entirely mutable and based on individual criteria. It might
be better to say that the value of a design has no real connection
with the success of a design.

--
Austin Govella
Thinking & Making: IA, UX, and IxD
http://thinkingandmaking.com
austin.govella at gmail.com

15 Nov 2006 - 12:48pm
Mark Schraad
2006

Metrics are important, especially to the positivists - namely most busines people and quantitative researchers. These are the folks bound up in a the quest for reliability. ie. if I reproduce this same experience 20 time over the next hundred years - will I get the same results? This is only one approach to what you are after.

Validity is another. (See Roger Martin of Rotman - he has written and talked on this issue of reliability vs validity extensively). Think about grounded theory or ethnography where you are looking for isolated instances with related and plausible explanations. These processes do indeed provide evidentury answers, but are not necasarily repeatable or quantifiable. That does not invalidate them.

Anytime you are investigating people in context there are a couple of things to remember. Most contextual situations will change over time. Most of what we as designers deal with - particularly on the web and in application interfaces are very short term. They last from a few days to a few years - but not to eternity. Secondly -as Dan mentioned, there are no control group or static line measures when it comes to behavior. Certainly there are majoritive tendoncies, but these are hit and miss.

Validity is not a substitue for reliability. It has great value, is yet another way of approaching a problem and is a critical component of how designers bring a different set of tools abd value to the world.

Mark

On Wednesday, November 15, 2006, at 11:43AM, "Austin Govella" <austin.govella at gmail.com> wrote:
>I've been trying to define a set of metrics along the same lines. So
>far, I have a set of metrics to measure different aspects of the
>implemented pattern. They're meant to be general, so we can get along
>to measuring without worrying about the specifics:
>
>1. Usable: combines completion time, error rate, learnability, etc.
>
>2. Accessible: device portability and how well it works without css or
>js dependencies
>
>3. Users like it: a simple rating of how much they do or don't enjoy the widget
>
>4. We like it: how much is it preferred by us (dev, design, eng,
>editorial, bus, etc.)

15 Nov 2006 - 12:48pm
LukeW
2004

In the interests of full disclosure, I'm going to excerpt a quote
from a separate thread that triggered a lot of my questions here:

"... I believe that our design patterns, discussions, and
justifications should be heavily influenced by the results of
cognitive psychology and the quantitative tools of interface design:
GOMs modeling, information-theoretic interface efficiency, Hick's
Law, Fitts' Law, etc. In computer science we use big-O notation to
compare algorithms, we should use the analogous tools for comparing
interface designs."

To me these are all "efficiency" measures. And hence, my explorations
of what other measures could be relevant as well. thanks~

On Nov 15, 2006, at 8:54 AM, Dave Malouf wrote:

>
> Is design about "measurement"?
> I know there are "design awards" and such, which implies there is a
> differentiation of value.
> But "measure" implies quantification as opposed to qualification and I
> think of designs as having qualities that are 'critique-able'.

::
:: Luke Wroblewski -[ www.lukew.com ]
:: Principal, LukeW Interface Designs
:: luke at lukew.com | 408.879.9826
::

15 Nov 2006 - 12:51pm
Mark Schraad
2006

Value can in fact be measured using utility and neary any scale of exchange (money, effort, time, etc). That is in fact the primary use of conjoint analysis outisde of attribute preference.

Mark

>The assumtion is that value is about measurement, but it's not. Value
>can be quantified or qualified, neither of which are discrete actions
>with discrete outputs. Measure assumes something is discrete, so in
>that case I agree: There's very little that is discrete about design.

15 Nov 2006 - 1:19pm
DanP
2006

> Is design about "measurement"? "measure" implies quantification as
> opposed to qualification and I
> think of designs as having qualities that are 'critique-able'.

Metrics. For some, a dreaded word :-) My opinion is "it depends."

Design can be measured and quantified, and it can also be critiqued
and qualified. But success is totally based on audience and
expectations.

Is the audience yourself, your clients/customers, or your
instructors? For instance, my instructors and employers "measure
success" through grades, performance reviews and pay. Some employ
quantifiable mathematics with structured and set methods to do so,
and some use critique and impressions. Some employ both.

In traditional "visual arts" design, this question is a bit easier to
answer - there are critiques. It has always been so, and it works. A
decent amount of subjectivity is expected and understood, and the
weight is on the subjective. In programming/code "design", metrics
and quantification often rule.

Is Interaction Design a fine-art based field, or is it scientific?
Everyone I ask has a different take on it depending on many
variables. Nothing wrong with that, but I suspect it requires the
ability to go back and forth between the two methods of determining
design success.

"Is design about measurement":

Architecture: Does the building fall down? F.E.A. is often employed.
Does the building look good? Back to fine art critique.

Industrial Design: Does the product function, and can it be produced?
Mock ups are often employed (materials, machine tools, etc..).
Does the product look good, and how does it make me feel? Back to
fine art.

Graphic Design: What objects "read" first, second and third in the
composition? Proofs are often employed.
Does the composition look good overall, and how does it make me feel?
Back to fine art.

I'm a visual designer by choice who can also quantify. My hope is
that the field offers a place for a visual person, and that my work
is weighted more towards that in general.

Regarding "quality", Persig dealt with this question pretty well in
Zen and the Art of Motorcycle Maintenance. It ultimately drove him to
be institutionalized, however.

All the best,
-Dan

-----------------------------------
Dan Peknik
San Jose State University
-----------------------------------

On Nov 15, 2006, at 8:54 AM, Dave Malouf wrote:

> I want to clarify/question something here:
>
> Is design about "measurement"?
> I know there are "design awards" and such, which implies there is a
> differentiation of value.
> But "measure" implies quantification as opposed to qualification and I
> think of designs as having qualities that are 'critique-able'.
>
> Now, I realize that there are "results" to designs and these
> results are
> definitely measurable and are probably the best indication of overall
> "success" of a design, but I'm not sure that "success" is really the
> final arbiter of "quality" of a design.
>

15 Nov 2006 - 2:10pm
russwilson
2005

LukeW wrote:
> 1) Is a widget or component or design pattern an "interaction design"
> in of itself? Or is it simply a "raw material" to be designed with?

I think it isn't. It might be an interface element, a component, just a widget. I believe that in order to call it interaction design it must be part of a flow or a more complex context of interaction. It is at the same level as a radio button or a text field.

-----------------------------------------

And not to diverge, but I am often asked what the difference
is between "interface" and "interaction" design. The above
seems to imply that an interface might be a widget, but an
interaction is a collection of widgets (or interface designs?)
applied to satisfy a specific goal.

Would others agree with this?

15 Nov 2006 - 3:09pm
.pauric
2006

Hello Wilson

I have little to add to the academic side of this discussion but I see a lot
of talk about what it is to be a component within a user interface. I'm not
sure there are any hard and fast rules here.

I have designed a 'widget' which is a virtual representation of the physical
device within the device's management UI.

On on the summary page it is used to display information about the device
such as utilisation, connected ports/users, e.g. status of components such
as modules/fans etc. As such, completes a flow.
It is clickable from the summary page which will take the user to a
configuration page specific to the area clicked, e.g. click on a port and
you are taken to the port setup page. It is now part of the menu.
It is also reused on high level protocol configuration pages and connects
these abstract protocols to their application on the device, e.g.
application of QoS/ACL etc. It is part of a flow, an element in the page.

If I were to measure its value, in addition to the very important
qualitative and qualitative criteria mentioned already I would look at how
its reuse has reduced user learning and development time.

I think the Spool Rule applies here, When is a widget not a widget? 'It
depends'

Regarding your question on Interface v Interaction design. "The above (your
previous email) seems to imply that an interface might be a widget, but an
interaction is a collection of widgets (or interface designs?) applied to
satisfy a specific goal."

An interface to me could be both a single or multiple widgets. But I do
agree with you that the flow through the widget(s) is the interaction, the
goal achieving part as you put it.

regards - pauric

17 Nov 2006 - 12:03am
Robert Reimann
2003

To me, interaction design is about the design of behavior, and the form
that enables that behavior, regardless of scale or scope.

Interface design is an older term, and one I don't particularly care for because
it is not very specific, and is as well a technology-centric term.
"User Interface" implies that the human is simply another peripheral
device to be interfaced with by the computer; i.e., the human's data
input must be responded to with machine output, etc. So while I
believe the terms are roughly equivalent, I think "interface design"
sends a message that works against what interaction designers are
trying to achieve, while also obscuring the key element of behavior.

Robert.

--
Robert Reimann
President, IxDA

Manager, User Experience
Bose Corporation
Framingham, MA

On 11/15/06, Wilson, Russell <Russell.Wilson at netqos.com> wrote:

> And not to diverge, but I am often asked what the difference
> is between "interface" and "interaction" design. The above
> seems to imply that an interface might be a widget, but an
> interaction is a collection of widgets (or interface designs?)
> applied to satisfy a specific goal.
>
> Would others agree with this?

17 Nov 2006 - 12:09am
Dan Saffer
2003

On Nov 15, 2006, at 11:10 AM, Wilson, Russell wrote:

> And not to diverge, but I am often asked what the difference
> is between "interface" and "interaction" design.

My answer has always been that interface design is the physical
expression of the interaction design.

Dan

17 Nov 2006 - 7:15pm
fred.welden at ...
2006

>From: LukeW <luke at lukew.com>
>To: Robert Reimann <rmreimann at gmail.com>
>Cc: discuss at ixda.org
>Sent: Wednesday, November 15, 2006 11:24:15 AM
>Subject: Re: [IxDA Discuss] IxD Metrics
>
>2) Can interaction design be measured out of context? Can you
>determine the "best" or "good" solutions without the context of use
>being known? To go back to the carousel example: is this a good
>interaction design despite what Web page or Desktop application it
>gets used in?
>3) What are the basic parameters of context? Is the environment in
>which it gets used (a Web browser) enough? Or do we need the "goals,
>needs, and motivations of users" to really understand context and
>thereby evaluate any design?
>
>I'm mostly throwing these questions out there for the sake of
>discussion. I don't actually have an answer formed :)
>

This draws on the response I posted under the "interface | interaction" titled thread.

An interaction is an instance of communication, so it always takes place in a complete context, not only of the environment in which it is used, but the specific circumstances (the user is signing up for a monthly service to be delivered to his home address, etc etc) in which it is used.

An interaction design is a reproducible set of interfaces and the semantics that connects them to facilitate that communication. How good is it can be measured by how well it facilitates the communication it is supposed to facilitate.

So the carousel is not an interaction design, it's a widget. A carousel that had some agreed-on semantics associated with it would be an interface. A carousel embedded in a process to collect some specific information from one entity and convey it to another would be part of an interaction design.

You need the goals, needs, and motivations of the users to understand context. You really need their life histories, the histories of the cultures they grew up in, and so on, but you have to make do with what you can get. (Sorry, I studied semantic analysis, learned that all meaning is derived from context, and I'm feeling pedantic this evening.)

But seriously I think you have to rate how good an interaction design is by how well it facilitates the specific communications your specific users make using it. That's pretty much the point behind personas, and back before that, use case analysis, no?

____________________________________________________________________________________
Sponsored Link

Mortgage rates near 39yr lows.
$310k for $999/mo. Calculate new payment!
www.LowerMyBills.com/lre

22 Nov 2006 - 6:40pm
niklasw
2005

Hi
Even if the heading says IxD metrics I am surprised that nowhere in
the discussion is the term aesthetics mentioned as a part of good
interaction design. It is touched upon as in engagement or delight,
referred to in other design disciplines as in it looks good and in how
it makes me feel.

We basically can't and shouldn't have to use parameters from other
design disciplines to describe good or bad IxD.

It is my opinion that there are parts unique to IxD that should be
described as being more or less aesthetic which then is one of the
components of evaluating if it is good or bad IxD. Loosely described
it could be described as that state of flow you get with some
interactive products. With some right away with some after a while and
it applies for both physical and software interactive products.

Measurable or not? I'm not sure... but I'm sure that it is there.

--Niklas

PS
Jonas Löwgren (http://webzone.k3.mah.se/k3jolo/) suggest describing it
as level of pliability but personally I'm not particularly fond of
that description.
DS

On 11/18/06, fred.welden at yahoo.com <fred.welden at yahoo.com> wrote:
> >From: LukeW <luke at lukew.com>
> >To: Robert Reimann <rmreimann at gmail.com>
> >Cc: discuss at ixda.org
> >Sent: Wednesday, November 15, 2006 11:24:15 AM
> >Subject: Re: [IxDA Discuss] IxD Metrics
> >
> >2) Can interaction design be measured out of context? Can you
> >determine the "best" or "good" solutions without the context of use
> >being known? To go back to the carousel example: is this a good
> >interaction design despite what Web page or Desktop application it
> >gets used in?
> >3) What are the basic parameters of context? Is the environment in
> >which it gets used (a Web browser) enough? Or do we need the "goals,
> >needs, and motivations of users" to really understand context and
> >thereby evaluate any design?
> >
> >I'm mostly throwing these questions out there for the sake of
> >discussion. I don't actually have an answer formed :)
> >
>
> This draws on the response I posted under the "interface | interaction" titled thread.
>
> An interaction is an instance of communication, so it always takes place in a complete context, not only of the environment in which it is used, but the specific circumstances (the user is signing up for a monthly service to be delivered to his home address, etc etc) in which it is used.
>
> An interaction design is a reproducible set of interfaces and the semantics that connects them to facilitate that communication. How good is it can be measured by how well it facilitates the communication it is supposed to facilitate.
>
> So the carousel is not an interaction design, it's a widget. A carousel that had some agreed-on semantics associated with it would be an interface. A carousel embedded in a process to collect some specific information from one entity and convey it to another would be part of an interaction design.
>
> You need the goals, needs, and motivations of the users to understand context. You really need their life histories, the histories of the cultures they grew up in, and so on, but you have to make do with what you can get. (Sorry, I studied semantic analysis, learned that all meaning is derived from context, and I'm feeling pedantic this evening.)
>
> But seriously I think you have to rate how good an interaction design is by how well it facilitates the specific communications your specific users make using it. That's pretty much the point behind personas, and back before that, use case analysis, no?
>
>
>
>
> ____________________________________________________________________________________
> Sponsored Link
>
> Mortgage rates near 39yr lows.
> $310k for $999/mo. Calculate new payment!
> www.LowerMyBills.com/lre
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org
>

Syndicate content Get the feed