usability testing equipment for mobile devices

18 Jan 2009 - 10:26pm
5 years ago
6 replies
3645 reads
Juan Ruiz
2007

Hello IxDA'rs

We are looking to purchase equipment for usability testing on mobile devices. I have done my research and there are 3 methods for this type of testing:

- Fixed device positioning: this setup is that the mobile device sits on a fix location with a webcam attached to the device via a metal neck or other type of support. The webcam points to the mobile device, recording the screen and the user's interaction with the device. A separate video camera on a fix location records the user's facial expressions and broad interaction.

http://www.gotomobile.com/archives/diy-gotomobiles-mobile-cam

- Movable positioning: A dual camera is attached to the mobile device. One camera records the screen and user interaction with the mobile, and the second camera records the facial expression as well as the positioning of the device in relation to the user.

http://www.littlespringsdesign.com/analysis/utest/

- VNC server on Mobile phone: the VNC server software has to be installed on the mobile device. The VNC server will share the mobile's screen to a pc. An external video camera on a fix location records the facial expressions of the user.

Have you purchased these type of equipment? Have you purchased a different type of equipment for these test? Any recommendation will be helpful.

Regards,
-Juan

Juan Ruiz
Senior User Experience Architect
-------------------------------------------------

hyro

P 1300 800 500 (in Australia)
D +61 2 9215 4279
M +61 421 448 780
E juan.ruiz at hyro.com<mailto:juan.ruiz at hyro.com>
W www.hyro.com<http://www.hyro.com>

AUSTRALIA | CHINA | HONG KONG | NEW ZEALAND | THAILAND
---------------------------------------------------------------------------------------------------------
Hyro is Hiring! For excellent career opportunities visit http://jobs.hyro.com
---------------------------------------------------------------------------------------------------------
ASX : HYO

Comments

18 Jan 2009 - 11:03pm
Anonymous

Hi Juan
What is it that you aim to test? While I personally haven't used any
of the three tools you list here so can't comment on them
specifically, I can say that having a fixed device limits you from
testing anything to do with being "mobile". Best practice
approaches to mobile usability really depend on what aspects you are
trying to get feedback on so with more info people might have
different experiences to offer.

Thanks and good luck with the testing
Penny

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=37344

19 Jan 2009 - 6:06am
Janne Kaasalainen
2008

Hi,

Without knowing much about what you do and why, but since you seem to
be after lab tests: Over here we at times use a small, custom video
camera about the size of a rubber eraser and have it fixed to view the
mobile screen. With this setup the mobile is not fixed itself, it only
has a wire or two coming out. Or, case pending, the TV-out to video
recorder may work as well (and usually to an extra TV display for
observing). Further recordings as needed, but those ought not to be
anything unusual (e.g. a video camera for the user). Most of the time
the software based screen recording apps haven't been up to speed for
us.

Disclaimer; the setup mentioned above does not work with all sort of
tests even in labs. Depending on your needs that may matter little or
a lot. One still needs to customize their procedures to fit the
questions they seek answer to.

Best regards,

Janne Kaasalainen

20 Jan 2009 - 7:00am
Harry Brignull
2004

Hi Juan,

Beware of fixed device positioning - this will effect the way the device is
used. Consider a traditional candybar handset - the user normally holds the
device and presses buttons with the same hand (single handed interaction).
If you fix this device to a table, then users will find it more natural to
"type" by poking the buttons with their forefingers. A problem of ecological
validity.

In the past I've used a mobile testing kit put together by
http://www.tracksys.co.uk/ (UK company) - composed of off-the-shelf
components nicely packed into a flight case. It was a perfectly good set up,
but if I was setting up a mobile lab now, I'd probably use the new version
of Morae since it supports multiple cameras. (and the VNC set up you refer
to, with a bit of tinkering).

There are lots of different possible camera configurations you can consider.
Try googling "Helmet camera" or "bullet camera" for some possibilities...

Hope that helps

Harry

--
Dr. Harry Brignull
User Experience Consultant
http://www.90percentofeverything.com
+ 44 (0)7920 474784

20 Jan 2009 - 11:24am
Barbara Ballard
2005

On Sun, Jan 18, 2009 at 9:26 PM, Juan Ruiz <Juan.Ruiz at hyro.com> wrote:
> We are looking to purchase equipment for usability testing on mobile devices.
>
> - Movable positioning: A dual camera is attached to the mobile device. One camera records the screen and user interaction with the mobile, and the second camera records the facial expression as well as the positioning of the device in relation to the user.
>
> http://www.littlespringsdesign.com/analysis/utest/

We ended up adding a third camera. One points at the screen, one at
the keypad (or the rest of the screen if a candybar phone), one at the
face. Run the whole thing through a quad-processor security thing (I'd
have to ask our camera geek about the details).

Tip: do NOT use wireless cameras. GSM will cause signal degradation.
Tip: if building it yourself, be sure to use lightweight metal or
plastic. Users start ignoring the sled soon enough, but not if it's
super heavy.

We do lots of testing on feature phones, and you really don't want to
add any software load to them (if you could get VNC in the first
place). We don't "upgrade" participants to nicer phones because those
aren't the phones they are used to, and then you have device & OS
issues compounding your application & website issues. If you're
testing the device itself, you've got more flexibility.

~~~~
Barbara Ballard
barbara at littlespringsdesign.com 1-785-838-3003

20 Jan 2009 - 5:47pm
Juan Ruiz
2007

Harry, I didn't think on using a "helmet camera" or "bullet
camera" before. I'm going to do more research about it and see if
it is feasible for usability testing on mobile devices. Thanks for
sharing this.

Barbara, thanks for the tip on NOT using wireless cameras. GSM will
cause signal degradation. Sometime for me to keep in mind.

Thanks,
-Juan

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=37344

21 Jan 2009 - 12:46am
baruag
2007

Hi Juan

Apart from the above methods, I've also managed to use low-tech
methods to conduct mobile usability tests successfully.

Method 1:
External camera positioned behind the user and looks the participants
shoulder.
The focus of the camera is zoomed in on the phone, enough to see the
screen as well as notice the participants physical interaction with
the device Having this kind of frame of focus really helps as it
helps understand the actions outside the screen as well as on the
screen, for example: hesitation to press a button or pressing the
back button after going to the wrong screen.

Method 2:
If testing an application and not the device itself, the N95 supports
a video out from the device. This way the Video Out can be plugged
directly into the observation screen (tv/monitor) and a recording
device. The users actions on the small screen are displayed on the
television/monitor in real time.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=37344

Syndicate content Get the feed