Usability testing at conferences

31 Jul 2007 - 3:04pm
7 years ago
9 replies
2034 reads
russwilson
2005

I just returned from a conference where we tested several interaction

designs on 45 users (you can see a summary here along with a simple
breakdown

of what it cost us:
http://www.dexodesign.com/2007/07/usability-testing-at-conferences.html)
.

This was a very successful event for us; we came back with great
feedback

that will feed several changes in our designs.

Out of curiosity, do many others do any usability testing at conferences

in their domains (this was a networking conference - my company creates

network performance management products).

Best regards,

Russ

Russell Wilson | Director of Product Design
NetQoS, Inc. | 5001 Plaza on the Lake, Austin, TX 78746
512.334.3725 | russell.wilson at netqos.com
<mailto:russell.wilson at netqos.com>

NetQoS: Performance Experts
www.netqos.com <http://www.netqos.com/>

Comments

1 Aug 2007 - 12:18am
Thomas Vander Wal
2004

I found this to be one of the best ways to get usability feedback from
the actual people using the site and those that are the target
audience.

Some times we would take early betas to a conference to build
interest, but also would have a space set up to sit and watch people
interact with the site and get feedback. Most often the people who
want or need the information or service are going to be at the
conference on that subject. It is a great way to find out what is
standing between people wanting and needing what you are offering and
them getting to it or having the service become something they need.

All the best,
Thomas

On 7/31/07, Wilson, Russell <Russell.Wilson at netqos.com> wrote:
> I just returned from a conference where we tested several interaction
>
> designs on 45 users (you can see a summary here along with a simple
> breakdown
>
> of what it cost us:
> http://www.dexodesign.com/2007/07/usability-testing-at-conferences.html)
> .
>
>
>
> This was a very successful event for us; we came back with great
> feedback
>
> that will feed several changes in our designs.
>
>
>
> Out of curiosity, do many others do any usability testing at conferences
>
> in their domains (this was a networking conference - my company creates
>
> network performance management products).
>
>
>
> Best regards,
>
> Russ
>
>
>
>
>
>
>
> Russell Wilson | Director of Product Design
> NetQoS, Inc. | 5001 Plaza on the Lake, Austin, TX 78746
> 512.334.3725 | russell.wilson at netqos.com
> <mailto:russell.wilson at netqos.com>
>
> NetQoS: Performance Experts
> www.netqos.com <http://www.netqos.com/>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://beta.ixda.org/guidelines
> List Help .................. http://beta.ixda.org/help
> Unsubscribe ................ http://beta.ixda.org/unsubscribe
> Questions .................. list at ixda.org
> Home ....................... http://beta.ixda.org
>

2 Aug 2007 - 8:25am
Michele Marut
2005

Yes, we have used conferences for stakeholder/user interviews and concept
evaulations and/or walk throughs.
The advantages are that we can often get the right type of users from
different geographical regions as well as a mix of small to large
businesses.

One caveat is often the people attending the conferences are expert users
and/or have money that might make them use different accessories with the
product.
For example, some of the people we interviewed always brought a laptop which
could connect to the product into the field with them. This was not the
majority of the user segment and so we then needed to also look at those
using it without other accessories.

Michele Marut

2 Aug 2007 - 11:52am
Fred Beecher
2006

On 8/1/07, Thomas Vander Wal <vanderwal at gmail.com> wrote:
>
> I found this to be one of the best ways to get usability feedback from
> the actual people using the site and those that are the target
> audience.
>

It's also a great way to get access (for usability testing or any other user
research method) to audiences who are normally difficult to get access to,
e.g. physicians.

- Fred

5 Sep 2007 - 3:27pm
Pawson, Mark
2007

I have been asked to do some discount usability testing at a
conference coming in the fall. Discount because essentially we are
two UI guys who will be manning an understaffed booth at a trade
show. Our role is to preview the alpha release of the software and
note usability issues from the attendees. From experience at trade
shows we know we should only count on 15 minutes per attendee max.;
thus a formal hour long usability test is out of the question. We are
wondering if we could simply have one or two simple tasks that we
could give to interested attendees and record their impressions. Sort
of letting the user demo the product rather than us. At the end of
each day we would interpret the findings and adjust for the next
day.
Any suggestions, critiques or studies where this has been done would
be appreciated.
Thanks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://beta.ixda.org/discuss?post=18865

5 Sep 2007 - 3:36pm
Christopher Fahey
2005

> We are wondering if we could simply have one or two simple tasks
> that we could give to interested attendees and record their
impressions.
> Sort of letting the user demo the product rather than us.

Not knowing anything about the product, I'd suggest that you ask the
most basic questions imaginable, at least for starters. For example,
ask them in the first two seconds "What do you think this site/app
does?" or "What do you think you can do with this screen/page?" This
way you can help learn things that will help with the initial sales
process and adoption learning curves.

-Cf

Christopher Fahey
____________________________
Behavior
http://www.behaviordesign.com
me: http://www.graphpaper.com

On Sep 5, 2007, at 5:27 PM, Pawson at otto.dreamhost.com, Mark wrote:

> I have been asked to do some discount usability testing at a
> conference coming in the fall. Discount because essentially we are
> two UI guys who will be manning an understaffed booth at a trade
> show. Our role is to preview the alpha release of the software and
> note usability issues from the attendees. From experience at trade
> shows we know we should only count on 15 minutes per attendee max.;
> thus a formal hour long usability test is out of the question. We are
> wondering if we could simply have one or two simple tasks that we
> could give to interested attendees and record their impressions. Sort
> of letting the user demo the product rather than us. At the end of
> each day we would interpret the findings and adjust for the next
> day.
> Any suggestions, critiques or studies where this has been done would
> be appreciated.
> Thanks
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://beta.ixda.org/discuss?post=18865
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://beta.ixda.org/guidelines
> List Help .................. http://beta.ixda.org/help
> Unsubscribe ................ http://beta.ixda.org/unsubscribe
> Questions .................. list at ixda.org
> Home ....................... http://beta.ixda.org

5 Sep 2007 - 3:54pm
russwilson
2005

So you are planning on doing this in the booth with people
that walk up, correct? Are you going to record what the users
do with Morae (or something similar), or just take handwritten notes?

Bottom line (from my direct experience with this), you are going to have
to ask them if they would like to participate and offer them something
to do
so (money, t-shirt, whatever). If you "suck" people into a test, they
will
not be happy. :-)

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Pawson at otto.dreamhost.com
Sent: Wednesday, September 05, 2007 4:28 PM
To: discuss at lists.interactiondesigners.com
Subject: Re: [IxDA Discuss] Usability testing at conferences

I have been asked to do some discount usability testing at a
conference coming in the fall. Discount because essentially we are
two UI guys who will be manning an understaffed booth at a trade
show. Our role is to preview the alpha release of the software and
note usability issues from the attendees. From experience at trade
shows we know we should only count on 15 minutes per attendee max.;
thus a formal hour long usability test is out of the question. We are
wondering if we could simply have one or two simple tasks that we
could give to interested attendees and record their impressions. Sort
of letting the user demo the product rather than us. At the end of
each day we would interpret the findings and adjust for the next
day.
Any suggestions, critiques or studies where this has been done would
be appreciated.
Thanks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://beta.ixda.org/discuss?post=18865

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
List Guidelines ............ http://beta.ixda.org/guidelines
List Help .................. http://beta.ixda.org/help
Unsubscribe ................ http://beta.ixda.org/unsubscribe
Questions .................. list at ixda.org
Home ....................... http://beta.ixda.org

5 Sep 2007 - 6:26pm
Cwodtke
2004

I did this several years ago with Noel Franus, and I was amazed at how
successful it was. A couple hints-- recruit at the booth, but find a
quiet place to do the testing ( you get a lot of looky-loos, which
creates weird interactions.)

The other thing I found is people were surprisingly willing to give up
their time to us.. often beyond the ten-fifteen minutes the initial
promised. We nearly had to kick them out. These were the same people who
we couldn't get five minutes form in the course of their normal lives.

Pawson at otto.dreamhost.com wrote:
> I have been asked to do some discount usability testing at a
> conference coming in the fall. Discount because essentially we are
> two UI guys who will be manning an understaffed booth at a trade
> show. Our role is to preview the alpha release of the software and
> note usability issues from the attendees. From experience at trade
> shows we know we should only count on 15 minutes per attendee max.;
> thus a formal hour long usability test is out of the question. We are
> wondering if we could simply have one or two simple tasks that we
> could give to interested attendees and record their impressions. Sort
> of letting the user demo the product rather than us. At the end of
> each day we would interpret the findings and adjust for the next
> day.
> Any suggestions, critiques or studies where this has been done would
> be appreciated.
> Thanks
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://beta.ixda.org/discuss?post=18865
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://beta.ixda.org/guidelines
> List Help .................. http://beta.ixda.org/help
> Unsubscribe ................ http://beta.ixda.org/unsubscribe
> Questions .................. list at ixda.org
> Home ....................... http://beta.ixda.org
>

--
Christina Wodtke
Principal Instigator
415-577-2550

Business :: http://www.cucinamedia.com
Magazine :: http://www.boxesandarrows.com
Product :: http://www.publicsquarehq.com
Personal :: http://www.eleganthack.com
Book :: http://www.blueprintsfortheweb.com

cwodtke at eleganthack.com

6 Sep 2007 - 7:59pm
Dante Murphy
2006

This can work , but you have to adapt the methodology. Here are a couple of suggestions based on methods I've used in the past.

If you think your app will generate enough interest (it's an established product, your company has a good reputation, or it's a severely underserved market), then you can schedule "mini-previews" of specific segments of the appplication. Post a schedule at the booth...

10:00 am - Configuration
11:00 am - Remote Monitoring
Noon - Splitting the Atom
etc.

You get the idea. Then when the crowd gathers that is interested in splitting the atom, chat them up and screen for someone who feels like a real potential user, not just some nerd with nothing better to do. Sit her in front of your app and let her use that portion of the app. If she runs into trouble, or does something really unexpected, go to the larger audience for verbatims.

If you don't think you'll get that kind of traffic, then you and you partner may have to tag-team...one does a sit-down review with a single user, while the other fields questions and glad-hands.

Hope that helps, and good luck!

Dante

________________________________

From: discuss-bounces at lists.interactiondesigners.com on behalf of Pawson at otto.dreamhost.com
Sent: Wed 9/5/2007 5:27 PM
To: discuss at lists.interactiondesigners.com
Subject: Re: [IxDA Discuss] Usability testing at conferences

I have been asked to do some discount usability testing at a
conference coming in the fall. Discount because essentially we are
two UI guys who will be manning an understaffed booth at a trade
show. Our role is to preview the alpha release of the software and
note usability issues from the attendees. From experience at trade
shows we know we should only count on 15 minutes per attendee max.;
thus a formal hour long usability test is out of the question. We are
wondering if we could simply have one or two simple tasks that we
could give to interested attendees and record their impressions. Sort
of letting the user demo the product rather than us. At the end of
each day we would interpret the findings and adjust for the next
day.
Any suggestions, critiques or studies where this has been done would
be appreciated.
Thanks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://beta.ixda.org/discuss?post=18865

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
List Guidelines ............ http://beta.ixda.org/guidelines
List Help .................. http://beta.ixda.org/help
Unsubscribe ................ http://beta.ixda.org/unsubscribe
Questions .................. list at ixda.org
Home ....................... http://beta.ixda.org <http://beta.ixda.org/>

7 Sep 2007 - 7:23am
vutpakdi
2003

Pawson at otto.dreamhost.com wrote:
> I have been asked to do some discount usability testing at a
> conference coming in the fall. Discount because essentially we are
> two UI guys who will be manning an understaffed booth at a trade
> show. Our role is to preview the alpha release of the software and
> note usability issues from the attendees.
I think that doing so would be a really good idea, if only from the
public relations standpoint. Most of the people attending trade shows
in our industry will likely be the more senior folks in a position to
possibly influence their management. Current customers are also fairly
likely to stop by and want to take a look at what you have. Being a
current customer, they probably will be fairly open to give a quick
opinion, especially if they feel that they can influence the direction
of the next release and get a small bit of swag that they can't get
otherwise (say a Starbucks gift card for $5 or so).

Also, doing the "quickie" testing might net you some folks who would be
willing to do more in depth testing.

Given the short time frame, you probably won't be able to get more than
general impressions. Or, at least, you should be careful to avoid
assigning too much weight to feedback and results that you do get
(without factoring an adjustment due to the setting and circumstances).

Ron

Syndicate content Get the feed