If you can afford it I think this is problem than can truly benefit
from an eye-tracking quantitative survey--30 candidates. I'm
assuming that you're trying to solve for the basic flow and not
trying 4X versions of solutions, etc. I don't use quantitative
eye-tracking personally but saw an example with my vendor at the U of
MN where it was very useful to get insights about the action/reactions
We would like to test 5-6 different designs of a search results page
and want to know if users notice some elements or information on the
different designs. Are there best practices (besides counter
balancing) in usability test of search results pages? We have an
eye tracker that can be beneficial in the usability test.
I worked on usability testing of over 50 participants using the
knowledge base on a major software company's Web site and our results
should that a search results page that does NOT include the actual
page title, a page description, and matches the keywords used by users
when they create a search are unlikely to help a user find the
information they need.
This site accessed pages written by a variety of product groups within
this giant software company, but these groups were not consistent in
filling Title tags with titles that had meaning to users (not the kb
item #); including a clear, concise description of the pages content;
including synonyms in keywords; or making the version of a product a
prominent part of all three of these elements.
It is critical to make sure search is not simply matching the words
already on the pages, but that research is done to discover the actual
words and phrases users enter when doing a search. We also found that
giving users the option of limiting their search by product name,
product version, and other variables that can affect the relevance of
results was useful but only if users actually understood what their
choice of a variable (such as "filter") would do.
On Mon, May 11, 2009 at 5:47 AM, Shima Kazerooni <shkazerooni at yahoo.com> wrote:
> We would like to test 5-6 different designs of a search results page > and want to know if users notice some elements or information on the > different designs. Are there best practices (besides counter > balancing) in usability test of search results pages? We have an > eye tracker that can be beneficial in the usability test. > > Thanks! > > > > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > Posted from the new ixda.org >http://www.ixda.org/discuss?post=41852 > > > ________________________________________________________________ > Welcome to the Interaction Design Association (IxDA)! > To post to this list ....... discuss at ixda.org > Unsubscribe ................ http://www.ixda.org/unsubscribe > List Guidelines ............ http://www.ixda.org/guidelines > List Help .................. http://www.ixda.org/help >
Oh, and eye-tracking is not necessary if you ask people to practice
the speak-aloud protocol and to read the words of whatever they are
looking at on the page. If time-on-task is a consideration in your
testing, you can not use speak-aloud during the actual test, but
immediately after each task, sit with the user while they watch the
video you recorded and ask they to talk about what they were seeing
and thinking.
I share Jared Spool's skepticism about eye tracking's usefulness in
most usability studies of Web page use. Given the cost of the
equipment and how long it takes to actually understand how to
interpret the results, you can get the same results using tried and
true methods.
Mary
On Mon, May 11, 2009 at 2:40 PM, Mary Deaton <mmdeaton at mmdeaton.com> wrote:
> I worked on usability testing of over 50 participants using the > knowledge base on a major software company's Web site and our results > should that a search results page that does NOT include the actual > page title, a page description, and matches the keywords used by users > when they create a search are unlikely to help a user find the > information they need. > > This site accessed pages written by a variety of product groups within > this giant software company, but these groups were not consistent in > filling Title tags with titles that had meaning to users (not the kb > item #); including a clear, concise description of the pages content; > including synonyms in keywords; or making the version of a product a > prominent part of all three of these elements. > > It is critical to make sure search is not simply matching the words > already on the pages, but that research is done to discover the actual > words and phrases users enter when doing a search. We also found that > giving users the option of limiting their search by product name, > product version, and other variables that can affect the relevance of > results was useful but only if users actually understood what their > choice of a variable (such as "filter") would do. > > -- > Mary Deaton > Manager, STC Usability and User Experience Community, >http://www.stcsig.org/usability > Principal, Deaton Interactive Design >http://www.mmdeaton.com > Associate, SodaBlue Partners >http://www.sodabluepartners.com > > On Mon, May 11, 2009 at 5:47 AM, Shima Kazerooni <shkazerooni at yahoo.com> wrote: >> We would like to test 5-6 different designs of a search results page >> and want to know if users notice some elements or information on the >> different designs. Are there best practices (besides counter >> balancing) in usability test of search results pages? We have an >> eye tracker that can be beneficial in the usability test. >> >> Thanks! >> >> >> >> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . >> Posted from the new ixda.org >>http://www.ixda.org/discuss?post=41852 >> >> >> ________________________________________________________________ >> Welcome to the Interaction Design Association (IxDA)! >> To post to this list ....... discuss at ixda.org >> Unsubscribe ................ http://www.ixda.org/unsubscribe >> List Guidelines ............ http://www.ixda.org/guidelines >> List Help .................. http://www.ixda.org/help >> >
I have done a fair bit of usability research with web search engines
and how people deal with them. Curiously, our participants seemed to
make better relevance decisions using just the page titles alone than
the page titles along with something else. This implied that
abstracting information from the main text content (whether initial
mention, keyword embedded like Google, keyword extraction etc)
actually misleads searchers into thinking that the document is more
relevant than it actually is.
However, people disliked using titles alone more than using some
extra text from the document. Pictures didn't seem to play an
important role unless they were looking for a specific thing /
product / company and this was displayed.
So there seemed to be a balance: decisions of a document's relevance
to an information need were better made with less information, but
users were less happy with dealing with search like this.
Get the feed
Comments
On May 7, 2009, at 8:34 PM, Shima Kazerooni wrote:
> What are some of the best practices in usability test of search
> results pages? Do you use them?
Shima,
Your question isn't very clear.
Could you possibly explain what your situation is and what you're
hoping to learn? That might get you some helpful suggestions.
Jared
Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: @jmspool
UIE Roadshow: Seattle, Denver, DC in June: http://is.gd/gxwe
If you can afford it I think this is problem than can truly benefit
from an eye-tracking quantitative survey--30 candidates. I'm
assuming that you're trying to solve for the basic flow and not
trying 4X versions of solutions, etc. I don't use quantitative
eye-tracking personally but saw an example with my vendor at the U of
MN where it was very useful to get insights about the action/reactions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=41852
We would like to test 5-6 different designs of a search results page
and want to know if users notice some elements or information on the
different designs. Are there best practices (besides counter
balancing) in usability test of search results pages? We have an
eye tracker that can be beneficial in the usability test.
Thanks!
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=41852
I worked on usability testing of over 50 participants using the
knowledge base on a major software company's Web site and our results
should that a search results page that does NOT include the actual
page title, a page description, and matches the keywords used by users
when they create a search are unlikely to help a user find the
information they need.
This site accessed pages written by a variety of product groups within
this giant software company, but these groups were not consistent in
filling Title tags with titles that had meaning to users (not the kb
item #); including a clear, concise description of the pages content;
including synonyms in keywords; or making the version of a product a
prominent part of all three of these elements.
It is critical to make sure search is not simply matching the words
already on the pages, but that research is done to discover the actual
words and phrases users enter when doing a search. We also found that
giving users the option of limiting their search by product name,
product version, and other variables that can affect the relevance of
results was useful but only if users actually understood what their
choice of a variable (such as "filter") would do.
--
Mary Deaton
Manager, STC Usability and User Experience Community,
http://www.stcsig.org/usability
Principal, Deaton Interactive Design
http://www.mmdeaton.com
Associate, SodaBlue Partners
http://www.sodabluepartners.com
On Mon, May 11, 2009 at 5:47 AM, Shima Kazerooni <shkazerooni at yahoo.com> wrote:
> We would like to test 5-6 different designs of a search results page
> and want to know if users notice some elements or information on the
> different designs. Are there best practices (besides counter
> balancing) in usability test of search results pages? We have an
> eye tracker that can be beneficial in the usability test.
>
> Thanks!
>
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=41852
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>
Oh, and eye-tracking is not necessary if you ask people to practice
the speak-aloud protocol and to read the words of whatever they are
looking at on the page. If time-on-task is a consideration in your
testing, you can not use speak-aloud during the actual test, but
immediately after each task, sit with the user while they watch the
video you recorded and ask they to talk about what they were seeing
and thinking.
I share Jared Spool's skepticism about eye tracking's usefulness in
most usability studies of Web page use. Given the cost of the
equipment and how long it takes to actually understand how to
interpret the results, you can get the same results using tried and
true methods.
Mary
On Mon, May 11, 2009 at 2:40 PM, Mary Deaton <mmdeaton at mmdeaton.com> wrote:
> I worked on usability testing of over 50 participants using the
> knowledge base on a major software company's Web site and our results
> should that a search results page that does NOT include the actual
> page title, a page description, and matches the keywords used by users
> when they create a search are unlikely to help a user find the
> information they need.
>
> This site accessed pages written by a variety of product groups within
> this giant software company, but these groups were not consistent in
> filling Title tags with titles that had meaning to users (not the kb
> item #); including a clear, concise description of the pages content;
> including synonyms in keywords; or making the version of a product a
> prominent part of all three of these elements.
>
> It is critical to make sure search is not simply matching the words
> already on the pages, but that research is done to discover the actual
> words and phrases users enter when doing a search. We also found that
> giving users the option of limiting their search by product name,
> product version, and other variables that can affect the relevance of
> results was useful but only if users actually understood what their
> choice of a variable (such as "filter") would do.
>
> --
> Mary Deaton
> Manager, STC Usability and User Experience Community,
> http://www.stcsig.org/usability
> Principal, Deaton Interactive Design
> http://www.mmdeaton.com
> Associate, SodaBlue Partners
> http://www.sodabluepartners.com
>
> On Mon, May 11, 2009 at 5:47 AM, Shima Kazerooni <shkazerooni at yahoo.com> wrote:
>> We would like to test 5-6 different designs of a search results page
>> and want to know if users notice some elements or information on the
>> different designs. Are there best practices (besides counter
>> balancing) in usability test of search results pages? We have an
>> eye tracker that can be beneficial in the usability test.
>>
>> Thanks!
>>
>>
>>
>> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
>> Posted from the new ixda.org
>> http://www.ixda.org/discuss?post=41852
>>
>>
>> ________________________________________________________________
>> Welcome to the Interaction Design Association (IxDA)!
>> To post to this list ....... discuss at ixda.org
>> Unsubscribe ................ http://www.ixda.org/unsubscribe
>> List Guidelines ............ http://www.ixda.org/guidelines
>> List Help .................. http://www.ixda.org/help
>>
>
--
Mary Deaton
Manager, STC Usability and User Experience Community,
http://www.stcsig.org/usability
Principal, Deaton Interactive Design
http://www.mmdeaton.com
Associate, SodaBlue Partners
http://www.sodabluepartners.com
I have done a fair bit of usability research with web search engines
and how people deal with them. Curiously, our participants seemed to
make better relevance decisions using just the page titles alone than
the page titles along with something else. This implied that
abstracting information from the main text content (whether initial
mention, keyword embedded like Google, keyword extraction etc)
actually misleads searchers into thinking that the document is more
relevant than it actually is.
However, people disliked using titles alone more than using some
extra text from the document. Pictures didn't seem to play an
important role unless they were looking for a specific thing /
product / company and this was displayed.
So there seemed to be a balance: decisions of a document's relevance
to an information need were better made with less information, but
users were less happy with dealing with search like this.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=41852