Remote usability testing and participant feedback

27 Feb 2007 - 8:09pm
7 years ago
3 replies
987 reads
Jeff Axup
2006

Greetings,

It's been a while since I heard anything about the remote-usability testing
movement...

I've personally used online questionnaires, postcards, discussion forums and
blogs as tools for remotely eliciting user comments. I also hear that tech
support is commonly using tools to watch users' mouse and navigation actions
remotely with good results.
Also, I'd also be interested in knowing more about how site usage analysis
is being done and how designers of applications (as opposed to web-sites)
are getting and using remote quantitative usage data.
So all of the above methods would fall under the category of remote user
involvement/feedback.
Does anyone know of any recent books or other resources regarding this, or
have any personal experiences with the above?

Best Regards,
Jeff
____________________________________________________________________________
Jeff Axup Ph.D. Candidate - University of Queensland, Brisbane,
Australia
Principal Consultant, Mobile Community Design Consulting,
San Diego

Research: Mobile Group Research Methods, Social Networks, Group Usability
E-mail: axup <at> userdesign.com
Blog: http://mobilecommunitydesign.com
Moblog: http://memeaddict.blogspot.com
Academic: http://www.infenv.itee.uq.edu.au
____________________________________________________________________________

Comments

28 Feb 2007 - 8:45am
Trit Mulligan
2007

Most of the testing that I do is remote based using WebEx (soon we are
switching to Live Meeting)since our customers are spread across the
US/UK/Australia. The meetings are scheduled for no more than 1 hour.
Developers and QA are invited to join the meeting and I also use Morae
to record the sessions. The users are given control of the computer and
asked to perform specific tasks. The pros to this approach are that I
can control the environment as far as the app is concerned, I can
quickly get a study done using a handful of users.

I find that users don't seem to be as nervous because you're not sitting
down beside them staring at them. The downside is that by doing remote
testing you miss the face to face time which is especially helpful when
they are reading (labels, etc. - facial expressions can say a lot about
what a user is thinking). I also use the think-aloud method (must in
remote testing) and listening labs as a way to learn more about the user
experience. I follow this all up with some quick and dirty satisfaction
questions to measure quantitative data. I still have not had a lot of
luck with measuring qualitative data but hope to increase those
measurements this year.

We also have two additional initiatives planned for this year: we are
launching a sandbox environment and a design portal. The sandbox will
allow users to play in our apps on their own time and perform tasks and
give feedback without impacting their own environment. This will allow
us to increase sample sizes of users and hopefully get even more
feedback. The design portal will be a collaborative effort between our
users and designers to get additional feedback early in our process. The
portal will consist of sharing screens, talking directly to designers,
etc. We are excited about launching this new communication tool.

Trit Mulligan

Usability Engineer
Blackbaud, Inc.

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Jeff Axup
Sent: Tuesday, February 27, 2007 8:10 PM
To: IXDA list
Subject: [IxDA Discuss] Remote usability testing and participant
feedback

Greetings,

It's been a while since I heard anything about the remote-usability
testing movement...

I've personally used online questionnaires, postcards, discussion forums
and blogs as tools for remotely eliciting user comments. I also hear
that tech support is commonly using tools to watch users' mouse and
navigation actions remotely with good results.
Also, I'd also be interested in knowing more about how site usage
analysis is being done and how designers of applications (as opposed to
web-sites) are getting and using remote quantitative usage data.
So all of the above methods would fall under the category of remote user
involvement/feedback.
Does anyone know of any recent books or other resources regarding this,
or have any personal experiences with the above?

Best Regards,
Jeff
________________________________________________________________________
____
Jeff Axup Ph.D. Candidate - University of Queensland, Brisbane,
Australia
Principal Consultant, Mobile Community Design
Consulting, San Diego

Research: Mobile Group Research Methods, Social Networks, Group
Usability
E-mail: axup <at> userdesign.com
Blog: http://mobilecommunitydesign.com
Moblog: http://memeaddict.blogspot.com
Academic: http://www.infenv.itee.uq.edu.au
________________________________________________________________________
____
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org List Guidelines
............ http://listguide.ixda.org/ List Help ..................
http://listhelp.ixda.org/ (Un)Subscription Options ...
http://subscription-options.ixda.org/
Announcements List ......... http://subscribe-announce.ixda.org/
Questions .................. lists at ixda.org Home .......................
http://ixda.org/ Resource Library ........... http://resources.ixda.org

1 Mar 2007 - 9:57am
Barbara Ballard
2005

On 2/27/07, Jeff Axup <axup at userdesign.com> wrote:
> It's been a while since I heard anything about the remote-usability testing
> movement...
>
> Also, I'd also be interested in knowing more about how site usage analysis
> is being done and how designers of applications (as opposed to web-sites)
> are getting and using remote quantitative usage data.

We've developed a methodology that we're going to convert into a
product; it is outlined on pages 175-6 of my new book, Designing the
Mobile User Experience. Basically, if your application has any server
component at all, you be sure to have a rich log of events on the
server. When an event of interest happens, you send a SMS to the
phone number of the (registered) user. You can take a quick SMS
survey, or use a callback number to take the survey via VoiceXML.

Obviously this methodology works for beta testing scenarios, and is
not for traditional usability testing. On the other hand, it enables
an incredible array of real-use information, including usage patterns
over time and subsequent perceived usability.

There are tricks to the method, most especially restricting the
frequency with which you bother the user.

--
Barbara Ballard
barbara at littlespringsdesign.com 1-785-838-3003

6 Mar 2007 - 12:05pm
Lovellette, Shane
2007

In the past we have used both Webex and LiveMeeting with Morae for
remote testing. We now use UserVue (for purposes of full disclosure, we
develop UserVue and Morae).

I'm curious to know how many people do remote usability testing and what
tools they use. Why don't more people do it? Is it because the label
"usability testing" makes it seem too complicated?

Jeff brings up an interesting point. Does the design community view
usability testing as part of a broader user feedback category? How do
the majority of designers collect user feedback about designs early in
the process?

Thanks,

Shane Lovellette
TechSmith Corporation
s.lovellette at techsmith.com

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Jeff Axup
Sent: Tuesday, February 27, 2007 8:10 PM
To: IXDA list
Subject: [IxDA Discuss] Remote usability testing and participant
feedback

Greetings,

It's been a while since I heard anything about the remote-usability
testing
movement...

I've personally used online questionnaires, postcards, discussion forums
and
blogs as tools for remotely eliciting user comments. I also hear that
tech
support is commonly using tools to watch users' mouse and navigation
actions
remotely with good results.
Also, I'd also be interested in knowing more about how site usage
analysis
is being done and how designers of applications (as opposed to
web-sites)
are getting and using remote quantitative usage data.
So all of the above methods would fall under the category of remote user
involvement/feedback.
Does anyone know of any recent books or other resources regarding this,
or
have any personal experiences with the above?

Best Regards,
Jeff
________________________________________________________________________
____
Jeff Axup Ph.D. Candidate - University of Queensland, Brisbane,
Australia
Principal Consultant, Mobile Community Design
Consulting,
San Diego

Research: Mobile Group Research Methods, Social Networks, Group
Usability
E-mail: axup <at> userdesign.com
Blog: http://mobilecommunitydesign.com
Moblog: http://memeaddict.blogspot.com
Academic: http://www.infenv.itee.uq.edu.au
________________________________________________________________________
____
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
List Guidelines ............ http://listguide.ixda.org/
List Help .................. http://listhelp.ixda.org/
(Un)Subscription Options ... http://subscription-options.ixda.org/
Announcements List ......... http://subscribe-announce.ixda.org/
Questions .................. lists at ixda.org
Home ....................... http://ixda.org/
Resource Library ........... http://resources.ixda.org

Syndicate content Get the feed