Round-up of remote usability testing systems

16 Apr 2009 - 5:57pm
4 years ago
5 replies
4947 reads
Elizabeth Bacon
2003

Hiya folks,

Firstly, thanks to those who contributed to two recent threads here
about remote usability testing systems! I happen to be shopping for a
solution myself, and so I did a big round-up of the offerings that I
could discover. See: http://spreadsheets.google.com/ccc?key=pc-NAeDLmYFveCRlIejNuFA

On a high-level, there is the asynchronous, unmoderated, remote
system, available for a wide range of prices. Then there is a more do-
it-yourself synchronous, moderated, remote approach that's essentially
a screen-sharing & ideally recording solution, also available for a
wide range of prices. Plus Morae is on the list, as a step towards a
more complete overview of usability testing systems; Morae also plays
well with UserVue.

I'd be glad for people to expand on this material, which is a publicly
shared Google doc. I'll update it as I get more information from a
couple vendors, too. Enjoy!

Cheers,
Liz

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Vice-President, IxDA / www.ixda.org
CDO, Devise / www.devise.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Comments

16 Apr 2009 - 9:15pm
Joe Sokohl
2004

Nicely done!

This is very helpful and quite thorough.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=41325

17 Apr 2009 - 4:04am
James Page
2008

@Elizabeth Bacon
Thanks for putting the list up .

We will launching an update of our website in the next week or two, so until
then here is the missing data from your list for Webnographer.

All the best

James
http://blog.feralabs.com

Pricing
Currently our pricing is based on the time it needs for us to setup and
analysis. On average the time taken is about 1.5 days, but this can range
from 1 day to 5 depending on the complexity of both the test and the website
been tested, and number of participants. Our day rate is $1,200 per day.

Later in the year we will be launching a Self Service module. Pricing to
be announced.

Participants
Multiple ways of recruiting participants.

- Send a link via email, via a email list.
- Intercept users from your site.
- Use our panel.

Costs of participants are extra.

What designer provides
A link to a web address.

No code needs to be modified on the site been tested. Webnographer collects all
usability and user experience data, including click-streams and click-paths
without the participant having to download any extension or plugin. Works on
all common browsers including Safria, IE6 to 8, Firefox, and on all OS's
including Mac and PC

How it works
The Webnographer browser bar tracks the users click stream. This captures
all the user interactions with the site.

The software also allows questionnaires to be inserted into the test.

Currently we do the setup of the test, but we are developing a self service
module.

What analysis is provided
Webnographer Analysis provides metrics for the ISO Usability standard.
Satisfaction, Effectiveness, and Efficiency.

Webnographer provides answers about your users, such as:

- “How well does the system support the user goals?”
- “How efficient is the system for the user?”
- “Do users find the system effective for the given task?”
- “How satisfied are users with the system?”
- “Where do usability issues occurs?”
- “Why do usability issues occur?”

Additionally, Webnographer provides answers about business questions, such
as:

- “How can the system be improved?”
- “Where does improvement efforts need to be focussed?”
- “Which are the most important issues that should be fixed first?”

Reports that can be generated from the data include :-

- Task completion rate
- Time per task
- First clicks on page
- Missed Opportunities
- Click stream timeline
- Qualitative feedback with tagging ability
- Demographics
- Templates for standardised questionnaires, such as SUS
- Expectation Metric Learnability
- And many more

Currently Analysis is provided in either PowerPoint or Excell. But a web
interface will be shortly available.

Notes & Misc.
Contact us and we can organise a demo.

2009/4/16 Joe Sokohl <joe at sokohl.com>

> Nicely done!
>
> This is very helpful and quite thorough.
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=41325
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

17 Apr 2009 - 3:30pm
Craig Melbourne
2009

That great Liz, saved me a job.

I've always been a bit bemused why the Async tools are so expensive.
I looked at UserZoom but couldn't justify passing on the costs to
some of my smaller (yet worthy) clients so I ended up building my
own.

Hoping that Loop11 is *affordably priced*.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=41325

6 May 2009 - 1:11pm
Malia Zoghlin
2009

Thanks for posting this, it is quite helpful! I was just getting
started on putting my own research together when I found this
document. Really appreciate it!

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=41325

21 Sep 2010 - 6:28pm
leslie242
2010

Hey Liz,

I know some time has passed, but would you mind sharing A) which service(s) you ended up using and B) how the experience was?

I am currently conducting a similar evaluation for my work and would love to hear your perspective.

Thanks so much!

Syndicate content Get the feed