Does eye-tracking carry any real meaning?

21 Nov 2007 - 10:56am
6 years ago
32 replies
2229 reads
Robert Hoekman, Jr.
2005

23 rules actionable lessons from eye-tracking studies:
http://tinyurl.com/yrhydu

I'm curious whether or not others on the list take this stuff seriously. I
haven't decided yet if I think eye-tracking is a useful tool, and although I
definitely have some opinions on the subject, I'd like some other
perspectives to form a better picture.

Several of these "actionable lessons" bother me. For example, "One-column
formats perform better in eye-fixation than multi-column formats."
*
*
This seems painfully obvious, and also completely useless. Of course a
single-column is going to perform better - you only have to look at a single
column rather than two! But how does this help us? I'm certainly not going
to start using single-column layouts all the time just because it's faster.
When it comes down to it, a few seconds difference in the time it takes to
digest a page isn't going to make or break a user's experience. What matters
is whether the content is worth it. What matters is whether the value in
exploring the page outweighs the difficulty in doing so.

Another: "Ads placed next to the best content are seen more often."

Again, painfully obvious. But does this mean that people actually "see" the
ads, or that they happen to fixate on them for more than a millisecond? Does
it mean they read them? Click them more often?

There are so many factors to consider in web design and usability, it's hard
to imagine that eye-tracking results can yield information that is truly
helpful.

-r-

Comments

21 Nov 2007 - 11:32am
Mark Schraad
2006

There is eye tracking equipment gathering dust in cog psych departments all over the country. Of course everytime a new head of research gets appointed in the corporate world... they trot out the eye tracking videos because they make for wow show and tells. Yawn.

Mark

On Wednesday, November 21, 2007, at 10:57AM, "Robert Hoekman, Jr." <robert at rhjr.net> wrote:
>23 rules actionable lessons from eye-tracking studies:
>http://tinyurl.com/yrhydu
>
>I'm curious whether or not others on the list take this stuff seriously. I
>haven't decided yet if I think eye-tracking is a useful tool, and although I
>definitely have some opinions on the subject, I'd like some other
>perspectives to form a better picture.
>
>Several of these "actionable lessons" bother me. For example, "One-column
>formats perform better in eye-fixation than multi-column formats."
>*
>*
>This seems painfully obvious, and also completely useless. Of course a
>single-column is going to perform better - you only have to look at a single
>column rather than two! But how does this help us? I'm certainly not going
>to start using single-column layouts all the time just because it's faster.
>When it comes down to it, a few seconds difference in the time it takes to
>digest a page isn't going to make or break a user's experience. What matters
>is whether the content is worth it. What matters is whether the value in
>exploring the page outweighs the difficulty in doing so.
>
>Another: "Ads placed next to the best content are seen more often."
>
>Again, painfully obvious. But does this mean that people actually "see" the
>ads, or that they happen to fixate on them for more than a millisecond? Does
>it mean they read them? Click them more often?
>
>There are so many factors to consider in web design and usability, it's hard
>to imagine that eye-tracking results can yield information that is truly
>helpful.
>
>-r-

21 Nov 2007 - 12:00pm
Patricia Garcia
2007

Personally, I think eye-tracking is only useful when testing a
specific interface. It's hard to draw general conclusions because
it will vary.

I also had a problem with this article in that it states where users
are more likely to look, (upper left corner - where we tend to place
the logo.)

Do we design because they are more likely to look there or do they
look there due to standards in design?

Users are less likely to notice graphics in the right pane, because
they are accustomed to ads being placed there. They start being
placed in the upper left corner and move the logo to the right (just
to make a point) than after awhile these eye-tracking studies will
show users tend to ignore the upper left corner and look to the upper
right.

For the most part I did find the article a "No, duh" for our
community. I think the only new piece of information that I became
conscious of is users tending to notice text before graphics. Again,
I think this varies. I've seen eye-tracking studies done for
specific interfaces and the results made sense for them.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22825

21 Nov 2007 - 12:27pm
Nancy Broden
2005

Eye-tracking feels like a desparate attempt to "scientifically" prove
the value of Design (with a big D). I don't have enough time to wade
into that one...

One question I do have for the eye-tracking supporters out there: if
the results culled from all the eye-tracking studies where implemented
wouldn't all of our websites look the same or at least very, very
similar?

> There are so many factors to consider in web design and usability, it's hard
> to imagine that eye-tracking results can yield information that is truly
> helpful.

-- Nancy
--------------------------------------
nancy.broden at gmail.com

21 Nov 2007 - 1:17pm
Nicholas Iozzo
2007

>From Nancy Broden:
One question I do have for the eye-tracking supporters out there: if
the results culled from all the eye-tracking studies where
implemented wouldn't all of our websites look the same or at least
very, very similar?
_____________

I do not consider myself an eye-tracking supporter anymore then I
consider myself a supporter of Allen wrenches (it is just a tool).
Also, as a consultant I have never encountered a situation in which I
felt eye tracking was the correct tool for the job. If I was back in
academia, I could imagine lots of times I would want to use it.

Those caveats out of the way, lots of standards and best practices
have evolved on where to place navigations and body content. But in
no way do all sites look the same. These studies mean something in
aggregate. You take the results from all of the studies and you use
your training and judgment to apply it.

If an eye tracking study just reports the results as a list of
commandments with none of the data or methodology behind it... then
we really can not use the results in our day-to-day job.

_________
>From Nancy Broden:
Eye-tracking feels like a desperate attempt to "scientifically"
prove the value of Design (with a big D) . I don't have enough time
to wade into that one...
_______

I can see coming to that conclusion if we are just using it to study
a small set of marketing sites or some e-commerce sites. But I
recently read a great article (sorry can not find the link, but I
hope someone on this list has it) where an eye tracking study was
used to aid in the design of software for radiologists. What they
found was that radiologists had higher dwell times on tumors that
they missed. They where able to determine some of the misses the
radiologists may have caught on their own if they where better able
to go back to past images. Given that the fast bulk of what they look
at are true negatives, their muscle memory kicks in and they mark an
image as negative before they fully processed it.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22825

21 Nov 2007 - 1:15pm
Murli Nagasundaram
2007

Good points. But this follows the old saw that to the guy with a hammer,
the whole world looks like a nail. All that the eye-tracking tells you is
that the user's eyes spend a lot of time looking at specific parts of the
screen/page. No more, no less. Eye tracking provides useful inputs once
one has already developed a couple of alternative design prototypes. It can
help one make design choices some way along the design proces, but
eye-tracking alone cannot drive design. Indeed, I don't know of any one
technique which by itself can or should drive design. Despite having a
strong techie background myself, I let my intuitions guide me in coming up
with rough cuts which then can be measured against various guidelines,
paradigms or methods.

I believe that guidelines are useful aids/heuristics for those who already
have an eye and a feel for design, and who therefore know when to respect or
reject received wisdom; but no amount of guidelines can turn a random person
into a designer.

-murli nagasundaram
www.murli.com

21 Nov 2007 - 12:18pm
Nick Quagliara
2007

It seems to me that if we boil everything down to efficiency and
heuristics then we become design robots. What about user experience
and pleasure? It's the intangibles that can really set a design
apart. I think that has to do with experience and tacit knowledge.
So yeah, I'm a bit wary of putting too much stock in eye-tracking
data.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22825

21 Nov 2007 - 2:35pm
Christopher Fahey
2005

> I'm curious whether or not others on the list take this stuff
> seriously. I
> haven't decided yet if I think eye-tracking is a useful tool, and
> although I
> definitely have some opinions on the subject, I'd like some other
> perspectives to form a better picture.

I've never read an eye-tracking insight that wasn't either (a)
patently obvious to anyone with a good sense for graphic design, or
(b) bad advice. The raw data from eyetracking is difficult enough to
draw meaningful conclusions from, especially if the person drawing
the conclusions doesn't know anything about graphic design, or about
the content being presented.

If you have a good sense for graphic design and you want to get a
good graphic design decision from a manager who has no sense
whatsoever about graphic design -- or from someone who has no respect
or trust in the expertise of graphic designers -- eyetracking may be
a good tool to apparently quantify an expert opinion. Otherwise, I am
incredibly skeptical -- it's not even like chicken soup, where it
doesn't hurt to try it, because a graphic design dolt can easily look
at eyetracking data and come up with manifestly bad advice.

Basically, if you have a good graphic designer that your organization
trusts I don't see how eyetracking will help you at all. If you don't
have a good graphic designer that is trusted by your organization,
use the money you'd spend on eyetracking to get one. If your
organization doesn't trust graphic designers, quit.

-Cf

Christopher Fahey
____________________________
Behavior
biz: http://www.behaviordesign.com
me: http://www.graphpaper.com

21 Nov 2007 - 2:50pm
Leisa Reichelt
2006

just a little sidenote:

it is fascinating to read the comments that people have left to the
originally quoted article - lots and lots of incredibly positive and
thankful comments with people talking of going back and redesigning
their sites as a result of this information.

it says a lot for the great demand out there for someone to make these
grand pronouncements about what works and what doesn't (with or
without context).

People do love their rules don't they.

I always find it a constant balancing act to try to give people this
sense of perceived value through pronouncements and actually getting
them to understand the context in which the design decisions are being
made.
________________________
Leisa Reichelt
Disambiguity.com

leisa.reichelt at gmail.com
+44 778 071 2129

21 Nov 2007 - 3:27pm
pnuschke
2007

Interesting article and discussion.

I've done a number of eyetracking studies and can say that there is
one thing that eyetracking is always great for- communication. Even if
you know that a problem exists, it is sometimes difficult to
communicate that understanding to people not as well versed in design
as we are. Eyetracking provides some great visuals that show the
problem in a concrete way.

Paul Nuschke
Human Factors Analyst
Electronic Ink

On Nov 21, 2007 10:56 AM, Robert Hoekman, Jr. <robert at rhjr.net> wrote:
> 23 rules actionable lessons from eye-tracking studies:
> http://tinyurl.com/yrhydu
>
> I'm curious whether or not others on the list take this stuff seriously. I
> haven't decided yet if I think eye-tracking is a useful tool, and although I
> definitely have some opinions on the subject, I'd like some other
> perspectives to form a better picture.
>
> Several of these "actionable lessons" bother me. For example, "One-column
> formats perform better in eye-fixation than multi-column formats."
> *
> *
> This seems painfully obvious, and also completely useless. Of course a
> single-column is going to perform better - you only have to look at a single
> column rather than two! But how does this help us? I'm certainly not going
> to start using single-column layouts all the time just because it's faster.
> When it comes down to it, a few seconds difference in the time it takes to
> digest a page isn't going to make or break a user's experience. What matters
> is whether the content is worth it. What matters is whether the value in
> exploring the page outweighs the difficulty in doing so.
>
> Another: "Ads placed next to the best content are seen more often."
>
> Again, painfully obvious. But does this mean that people actually "see" the
> ads, or that they happen to fixate on them for more than a millisecond? Does
> it mean they read them? Click them more often?
>
> There are so many factors to consider in web design and usability, it's hard
> to imagine that eye-tracking results can yield information that is truly
> helpful.
>
> -r-
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

21 Nov 2007 - 12:21pm
zack Frazier
2007

While it is true that many of the conclusions from eye-tracking
studies are obvious I'm not ready to lump them with "Gun-Toting
Drivers are More Prone to Road Rage."

What can we say about the behaviors that have obviously been
influenced directly by UXD over the years? For example:

Text attracts attention before graphics.
Readers ignore banners.
Fancy formatting and fonts are ignored.
One-column formats perform better in eye-fixation than multi-column
formats.
Text ads were viewed mostly intently of all types tested.

Wouldn't many of these behaviors tested differently in 1993 for a user
on Mosaic for the first time after only using LYNX? I doubt text ads
today would be as effective if LYNX had been exploited commercially
for years prior to the introduction of graphical browsers.

I like these studies because they provide even more validation for
common best practices. And obviously, user behaviors and attitudes are
not static.

Zack

21 Nov 2007 - 5:21pm
Chris Keane
2007

I think part of the problem with the article is that it frames eye
tracking as a tool for drawing sweeping conclusions, when it seems
far more appropriate for assessing a specific design and improving
performance in a manner similar to split testing (e.g. is this
heading getting ignored? What about if it were a little smaller?).
Eye tracking isn't relevant to branding issues or suggest solutions,
but it can give detailed feedback on a design's ergonomics and help
identify problems.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22825

21 Nov 2007 - 5:49pm
Rob Tannen
2006

Ah yes, eye tracking - it's like the "intelligent design" debate of
our field.

A few points to add:

-The reaction of "isn't that obvious" to some of the points
reminds me of reactions to findings from usability testing years ago.
Yes it may be apparently obvious, but that doesn't mean it's not
worth validating (see Freakanomics, for example).

-Eye tracking can be useful for diagnosing problems, not so much for
identifying them. For example, a viewer may miss a critical item on
the screen - eye tracking can reveal whether the element was visually
detected or overlooked and direct changes accordingly.

-There are some specific applications where eye tracking is
particularly useful. The radiology example was one. Another is when
we used eye tracking to determine whether a novel interface feature
(spatially expanding visual cue in the periphery) was affecting
visual scanning patterns.

Incidentally, in some cases attention precedes eye movements, and
vice-versa:
http://www.ingentaconnect.com/content/psocpubs/prp/2004/00000066/00000003/art00004

Happy Thanksgiving.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22825

21 Nov 2007 - 7:04pm
Robert Hoekman, Jr.
2005

>
> -Eye tracking can be useful for diagnosing problems, not so much for
> identifying them.

That's an interesting point. It does seems like eye-tracking would be
infinitely more valuable when used in conjunction with other things - like a
more traditional test. As in, you identify problems through regular ol'
"think out loud" testing and such, and then use the eye-tracking info to
help determine maybe where some of the problems start.

Hmm. As an idea/theory, I like it.

But even in that case, the eye-tracking would only be useful for
*some*information,
*some* of the time. Certainly not the broad, sweeping generalizations
summarized in these "actionable lessons". As such, it would be difficult to
justify the expense of the eye-tracking system unless you were a large
corporation with money to burn, or a UX research firm.

-r-

22 Nov 2007 - 6:08am
Adrian Howard
2005

On 21 Nov 2007, at 14:21, Chris Keane wrote:

> I think part of the problem with the article is that it frames eye
> tracking as a tool for drawing sweeping conclusions, when it seems
> far more appropriate for assessing a specific design and improving
> performance in a manner similar to split testing (e.g. is this
> heading getting ignored? What about if it were a little smaller?).

Yeah. I remember one particular project that, in hindsight, I wish I
had eye tracking data on.

Turned out that a underline on a heading was drawing the user towards
the "wrong" bit of the page. Once this was pointed out by the
(smarter than me) graphic designer the fix was trivial. Eye tracking
would have shown it up straight away.

It's just a tool.

Adrian

22 Nov 2007 - 4:54am
Anonymous

It is indeed fascinating to read the comments to the original article
- but I don't think you need to concern yourself with them too much:

"Thanks for the tips and advice...I hardly ever take into
consideration the audience%u2019s tendencies."

...

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22825

22 Nov 2007 - 11:18am
Jared M. Spool
2003

On Nov 21, 2007, at 7:04 PM, Robert Hoekman, Jr. wrote:

>> -Eye tracking can be useful for diagnosing problems, not so much for
>> identifying them.
>
>
> That's an interesting point. It does seems like eye-tracking would be
> infinitely more valuable when used in conjunction with other things
> - like a
> more traditional test.

Just like astrology.

Jared

22 Nov 2007 - 2:01pm
Jack L. Moffett
2005

But wouldn't the eye tracking experiment have taken a lot more time
and effort than consulting the graphic designer?

Jack

Sent from my iPhone

On Nov 22, 2007, at 6:08 AM, "Adrian Howard" <adrianh at quietstars.com>
wrote:

> Yeah. I remember one particular project that, in hindsight, I wish
> I had eye tracking data on.
>
> Turned out that a underline on a heading was drawing the user towards
> the "wrong" bit of the page. Once this was pointed out by the
> (smarter than me) graphic designer the fix was trivial. Eye tracking
> would have shown it up straight away.
>
> It's just a tool.

22 Nov 2007 - 2:16pm
Mark Schraad
2006

Yes. Any graphic designer worth their salt would know most of what
the article points out.

Another issue is that eye tracking does measure some valuable
sensation, perception cognition, but that does not necessarily equate
to actual user reaction and behavior.

Mark

On Nov 22, 2007, at 2:01 PM, Jack Moffett wrote:

> But wouldn't the eye tracking experiment have taken a lot more time
> and effort than consulting the graphic designer?
>
> Jack
>
> Sent from my iPhone
>
> On Nov 22, 2007, at 6:08 AM, "Adrian Howard" <adrianh at quietstars.com>
> wrote:
>
>> Yeah. I remember one particular project that, in hindsight, I wish
>> I had eye tracking data on.
>>
>> Turned out that a underline on a heading was drawing the user towards
>> the "wrong" bit of the page. Once this was pointed out by the
>> (smarter than me) graphic designer the fix was trivial. Eye tracking
>> would have shown it up straight away.
>>
>> It's just a tool.
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

22 Nov 2007 - 3:20pm
Pawson, Mark
2007

The first thing that struck me after reading this article was someone
has rehashed a bunch of design guidelines from Nielsen and other
course materials on homepage usability. Slap a title of Eyetracking
on it and a slow day for a writer just churned out an article....

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22825

22 Nov 2007 - 5:37pm
Adrian Howard
2005

On 22 Nov 2007, at 19:01, Jack Moffett wrote:

> But wouldn't the eye tracking experiment have taken a lot more time
> and effort than consulting the graphic designer?

If I already have my eye tracking equipment (I wish!) there isn't a
lot of extra cost in actually using it - and I don't always have
somebody cleverer than me to hand :-)

Having experts is better obviously. Hell, I'd have spotted it myself
on a good day. However, we all have bad days and having another tool
in the box is always useful. That list of "discoveries" on Seth's
blog was laughable, but I'm just as miffed by folk who think all eye
tracking is completely useless. Often misapplied and over-
generalised, but that's pretty much true of all research reporting
these days <sigh>.

The other interesting thing for me is that - as far as I can see -
there's nothing _seriously_ technical stopping eye-tracking gear
being considerably cheaper than it is at the moment. An IR source, a
decent camera or two, and some darn clever math. I expect more folk
doing research like http://talks.cam.ac.uk/talk/index/4650 on getting
eye tracking working with consumer level hardware.

Hell I can get a crappy head mounted tracker for about USD 10k. With
the exchange rate as it is at the moment that's less than the price
of a couple of high-end Mac Books in the UK!

What happens when you can get decent eye tracking equipment for a
couple of grand? If I had a bunch of eye tracking equipment I'm damn
sure I could find some useful ways to use it.

Cheers,

Adrian

22 Nov 2007 - 8:44pm
Stew Dean
2007

On 21/11/2007, Robert Hoekman, Jr. <robert at rhjr.net> wrote:
> 23 rules actionable lessons from eye-tracking studies:
> http://tinyurl.com/yrhydu
>
> I'm curious whether or not others on the list take this stuff seriously.

Simple answer is no, it's misleading. Text doesn't ATTRACT more
attention, it REQUIRES more attention. So whilst an image can be
glanced and understood quickly whilst you have to read text.

As with Jakob Nielsen's terrible 'Talking Heads are boring' article
what is happening is bad conclusion is being wrung from a fairly
meaningless set of data. In the talk head video people where listening
first, for example.

It's only real use is to test hypothesis, probably best in an academic
setting - not as a discovery tool. But then I even see lab based
usability testing as second fiddle to infield ethnographic testing -
labs are great for impressing clients but it's all smoke and mirrors -
well half silvered mirrors.

Eye tracking has no use on a real project, something I've learnt by
talking to those who have had experience of it first hand. I've even
been told that 'heat maps' are very little use as the time element is
missing, yet that's what clients like.

In short - it's snake oil.

--
Stewart Dean

22 Nov 2007 - 12:24pm
Kevin Lee
2007

An eye-tracking is the best way to "micro" analyze the pattern of
[you name anything]. At the end, the significance (or ROI) from micro
analyzing the "scanning" pattern of how people search for certain
information is insiginificant as the time involves in analyzing,
creating report, set up a meeting to report out, etc is completely
irrelevent to the needs of users. This is where 80/20 rule must be
applied. Is an eye-tracking helping us to solve 20% of problems that
contribute the success of 80%? I don't think so.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22825

22 Nov 2007 - 10:04pm
David Hall
2007

I actually think we're missing the point of why eye tracking is used to
frequently.

For consultancies selling IxD expertise it gives a 'oooh shiny' moment
for clients (and often for some practioners). Even if the commissioning
client staff member understands IxD it gives a recognisable 'scientific'
sheen to a report and therefore makes it easier to pry open the budget.

Or am I just being too cynical?

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Kevin Lee
Sent: 22 November 2007 17:25
To: discuss at ixda.org
Subject: Re: [IxDA Discuss] Does eye-tracking carry any real meaning?

An eye-tracking is the best way to "micro" analyze the pattern of
[you name anything]. At the end, the significance (or ROI) from micro
analyzing the "scanning" pattern of how people search for certain
information is insiginificant as the time involves in analyzing,
creating report, set up a meeting to report out, etc is completely
irrelevent to the needs of users. This is where 80/20 rule must be
applied. Is an eye-tracking helping us to solve 20% of problems that
contribute the success of 80%? I don't think so.

.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22825

________________________________________________________________
*Come to IxDA Interaction08 | Savannah*
February 8-10, 2008 in Savannah, GA, USA
Register today: http://interaction08.ixda.org/

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

23 Nov 2007 - 11:55am
Robert Hoekman, Jr.
2005

> For consultancies selling IxD expertise it gives a 'oooh shiny' moment
> for clients (and often for some practioners). Even if the commissioning
> client staff member understands IxD it gives a recognisable 'scientific'
> sheen to a report and therefore makes it easier to pry open the budget.

Jakob Nielsen is about to release a book on eye-tracking. One can only hope
that he addresses the appropriate contexts for use and how useful it really
is as a testing tool. But the simple fact that Nielsen is doing this means
that at least he thinks there's a whole lot more to it than "oooh shiny".

-r-

24 Nov 2007 - 6:18pm
hanif oneil
2007

Eye tracking is not the "end-all" solution, then again (from what
I've seen) it hasn't been promoted as such. It is an excellent
tool to help make designers more accountable to the functionality of
their designs. It also provides research teams further insights
beyond traditional "think out loud" methods. It provides data on
how users instinctively look at and interact with a web page, or
email campaign. This data does not offer feedback on whether a
design is good or not, instead it provides an understanding of how
well a design functioned: Did users see specific content or links?
And how long did it take them to see them in the interface? Or from
an analysis perspective: Are users' click patterns consistent to
their gaze pattern? What section of the page is experiencing a
considerable drop-off rate? For research teams this data will
always be qualitative, and its meaning defined by the goal(s) of
their design implements.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22825

25 Nov 2007 - 4:18pm
Jared M. Spool
2003

On Nov 24, 2007, at 3:18 PM, Hanif O'Neil wrote:

> This data does not offer feedback on whether a
> design is good or not, instead it provides an understanding of how
> well a design functioned: Did users see specific content or links?
> And how long did it take them to see them in the interface? Or from
> an analysis perspective: Are users' click patterns consistent to
> their gaze pattern? What section of the page is experiencing a
> considerable drop-off rate?

Unfortunately, this is a common misperception of eye-tracking. It
doesn't actually tell you any of those things.

It doesn't tell you what the user sees or doesn't see.

It only tells you, when calibrated well (something that's often not
done), where the user eye focuses. It's common for a user to focus
their eye on something and not see it. It's also common for a user to
see things they never focus on.

So, all you can tell is where the center of the retina pointed. And,
from a design standpoint, this tells you practically nothing.

Any inferences drawn from eye tracking data are influenced almost
100% from the inferrer. If a user gazes at a point on the screen for
a long period (whatever a "long period" is deemed to be), are they
gazing their because they are interested in that spot? Or because it
confused them? If they don't gaze at another point, is it because
they didn't see it? Or because they processed it from their
peripheral vision (or out of a focal area which is much wider than
the pixel point indicated by the eye tracker).

In my opinion, eye tracking a few advantages and a lot of
disadvantages. It doesn't tell design teams anything they couldn't
learn some other, far less expensive way. We regularly recommend our
clients avoid investing in it, stating its a waste.

Though, an eye tracker is cheaper than hiring a staff psychic, so if
those are your only two choices, I'd go with the eye tracker purely
to save money.

Jared

p.s. At UIE, we've been using eye trackers since 1996, so I do have
some experience with the tool. It's a great cognitive research tool,
but just not something we'd recommend for anyone doing production
design.

p.p.s. We haven't used psychics extensively, so I may be over
estimating their usefulness.

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks

25 Nov 2007 - 9:23pm
Robert Hoekman, Jr.
2005

> So, all you can tell is where the center of the retina pointed. And,
> from a design standpoint, this tells you practically nothing.

Cheers to that, Jared.

One can only hope that a UIE blog post is in order. Someone as knowledgeable
on the subject as yourself could do the world some good by refuting the list
that started this conversation, or at least debating the merits of eye
tracking.

You don't mind being egged on, do ya? ;)

-r-

24 Dec 2007 - 2:43pm
hanif oneil
2007

> Someone as knowledgeable on the subject as yourself could do the world
> some good by refuting the list that started this conversation, or at least
> debating the merits of eye tracking.
>
>

Eye-tracking provides unique methods to assess the impact of digital media
i.e., advertisements and web pages. Even still, the range of technology and
data resulting from research using eye-tracking varies beyond any
wide-sweeping criticism... The issue of accuracy Mr. Spool is suggesting
deals with what researchers call "visual acuity." Foveal vision constitutes
1-2 degrees of the eyes' visual field, which equates to roughly the area
covered by your thumb at arms length. For researchers using eye-tracking,
this often means that foveal vision is composed of an area of approximately
2cm in diameter (in other words one line top and bottom, and roughly a 1/2
word on each side). Anyone questioning whether peripheral vision is tracked
effectively, then dismissing an entire technology because of its relative
effectiveness to do so, is omitting a fact that peripheral vision is low
resolution. Which is a direct result of users eyes constantly focusing on
what falls within the high resolution [scan] area of the fovea. With that
said peripheral vision (is and) should be taken into account by researchers,
because it does play a role in–influencing gaze patterns and fixations.
Even as I write this I do not see how such an admission would invalidate
what eye tracking can do, or make it equal to what psychics do. Instead it
challenges usability experts to break open eye-tracking's usefulness as a
tool of research (some are up for the task more than others).

One debate that has lasted for quite some time is the relationship between
eye movement and cognitive processing. With this said, it is important to
note that I have not misleadingly suggested that where a user looks reflects
what as user is thinking. This is not a claim I am willing to make, as it
has been a worthwhile and largely contested area of research whereby many
theories avail.

Hanif O'Neil
Media Technician | Analyst

Skype: rawintellect
Office: +1 (415) 508 4116
Mobile: +1 (267) 879 9830

http://www.linkedin.com/in/honeil

25 Dec 2007 - 9:06pm
Anonymous

Totally agree with what Jared and others have said on the topic. The
plain truth is eye-tracking studies I've worked on have yielded
little to no more actionable data than less expensive methods have
provided. The practice probably has its place. I just haven't
found a good application of the technology yet.

Jake Zukowski
http://zukozee.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22895

26 Dec 2007 - 11:47am
Jeff Seager
2007

I studied a bit about eye-tracking when I was working for newspapers. We were concerned then mostly about the readers' "flow" as they scanned the page, and how our layouts might call attention to key elements such as informational graphics or advertising.

It seemed to me then that the only way you could draw valid conclusions from eye-tracking was if you also gathered subjective data on the user's cognitive process. What information did s/he retain a minute after reading? Five minutes later? An hour later? What did s/he recall about the images or the layout and design, and how or whether they reinforced the primary messages? How relevant did s/he perceive the material to be to his/her life before *and* after the experience? If that information is correlated with the eye-tracking data, you may have something useful.

I think it's also important to ask whether such studies are likely to merit their costs (time and money), compared to other user testing and the information already available to us about how to design effectively. In some rare cases, given an effective methodology, those costs will be justified. This may be comparable to a doctor who orders extensive testing "just in case" -- but not for all patients.

Is eye-tracking commonly used in modern interaction design, and if so, how? I'm curious to know.

Jeff Seager

P.S. Cheers to Jared on the Nilsson reference!

_________________________________________________________________
Don't get caught with egg on your face. Play Chicktionary!
http://club.live.com/chicktionary.aspx?icid=chick_wlhmtextlink1_dec

27 Dec 2007 - 6:46pm
Benjamin Ho
2007

I am actually intrigued by what Eye-tracking technology could offer.
I understand that the data is only as good as what kind of meaning we
put towards it - but then again, isn't that inherent in nearly
everything we do?

The users tell us like it is - but then we find out in research that
it isn't so. We go by the user's actions instead. So taking that
into consideration, shouldn't eye-tracking provide some sort of
validation in the physical sense?

And what about not really knowing what the problem is? While we may
be able to capture mouse-clicks, where the user looks first and in
those subsequent times, should we then just take a guess or use
eye-tracking?

I'm in the process of designing web applications (not websites) and
would like to know if anyone's used eye-tracking for that, taking
into consideration its flaws and strengths as discussed by Jared and
the others.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22895

28 Dec 2007 - 12:45am
Phil Chung
2007

Benjamin,

As you may know, eye tracking has traditionally been used to study topics such as banner blindness (or awareness) on websites and visual search (Are people seeing the critical warning signal on the interface? How do users search through the icons on their desktop?). You can find some examples on Anthony Hornof's site here:

http://www.cs.uoregon.edu/~hornof/publications.html

or this paper from a former labmate here:

http://chil.rice.edu/research/pdf/FleetwoodIcons.pdf

I'm guessing this is what you meant by: "shouldn't eye-tracking provide some sort of validation in the physical sense?" For instance, you might want to find out how frequently people look at the ads on Facebook (since it previously did not run ads). You could of course ask them informally if they noticed them, after running them through some tasks, but eyetracking would give you hard data (e.g., numbers...frequencies).

The linked examples are from academia -- given the amount of tedious post analysis eyetracking requires, in the commercial arena, I'd only use it if you really needed hard data to support your design / recommendations. For example, if you're testing safety (e.g., warning light) / business (e.g., banners) critical products.

I haven't been following this thread, so I apologize if I've just repeated what others have already stated. :-)

Phil Chung

----- Original Message ----
From: Benjamin Ho <benoh2 at yahoo.com>
To: discuss at ixda.org
Sent: Thursday, December 27, 2007 6:46:47 PM
Subject: Re: [IxDA Discuss] Does eye-tracking carry any real meaning?

I am actually intrigued by what Eye-tracking technology could offer.
I understand that the data is only as good as what kind of meaning we
put towards it - but then again, isn't that inherent in nearly
everything we do?

The users tell us like it is - but then we find out in research that
it isn't so. We go by the user's actions instead. So taking that
into consideration, shouldn't eye-tracking provide some sort of
validation in the physical sense?

And what about not really knowing what the problem is? While we may
be able to capture mouse-clicks, where the user looks first and in
those subsequent times, should we then just take a guess or use
eye-tracking?

I'm in the process of designing web applications (not websites) and
would like to know if anyone's used eye-tracking for that, taking
into consideration its flaws and strengths as discussed by Jared and
the others.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22895

________________________________________________________________
*Come to IxDA Interaction08 | Savannah*
February 8-10, 2008 in Savannah, GA, USA
Register today: http://interaction08.ixda.org/

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

Syndicate content Get the feed