Hack your own usability studio (long recipe, low price)

26 Sep 2007 - 3:30pm
6 years ago
6 replies
1232 reads
Fredrik Matheson
2005

At work we use Morae (http://techsmith.com/morae.asp) for user
testing. Quirks aside, it works pretty well.

The other day all our copies of Morae were busy and I needed a way to
let colleagues observe the use of a website in the room next door.
Basic requirements were:

1. a computer that the user would browse from (PC or Mac, it didn't matter)
2. some way to show what was on the user's screen on another computer
3. video of the user (one or more)
4. audio of what the user was doing
5. hopefully, a way to record all of this

Morae has lots of nice touches like keylogging, a yellow circle around
the mouse (to aid visibility) and click hinting. It also has a way to
mark events with keys on the observer's computer ("X" for problem, "L"
for login, or whatever you prefer). They're valuable features but you
can manage without them.

1: Since we had a few iSight cameras for the Mac, it made sense to use
it for observation and use the spare PC we had as the user's machine.
Therefore, the procedure for the observer machine below is a Mac.

2: VNC, or Virtual Network Computing, lets you access another
computer's screen from your own. That was the easiest way to see what
the user was doing. VNC uses a server on the host computer, and a
viewer, on your computer.
More information about VNC: http://www.cae.wisc.edu/site/public/?title=fswhatvnc

The test subject was on a PC laptop and the observer machine was a Mac.
On the PC, I used TightVNC (http://www.tightvnc.com/) as a server.
On the Mac, I used JollysFastVNC as a client
(http://www.jinx.de/JollysFastVNC.html).

We had to open a few ports in the PC's software firewall. VNC uses
ports 5900–5909 (network experts: feel free to correct me) and made
sure that the TightVNC server updated the entire screen often (300
milliseconds) so that there wouldn't be a lot of lag. Then we found
the IP address of the PC, typed it into JollysFastVNC and voilà, we
had our user's screen.

3. An iSight camera on a magnetic mount was placed on a camera tripod
and aimed at the user. We linked the standard firewire cable from the
iSight to a firewire hub with a separate power supply and a five meter
long cable. This let us watch the user from a separate room.

For software, you can use iChat or Quicktime, but I wanted a smaller
viewer. EvoCam (http://www.evological.com/evocam.html) gives you lots
of control of the image from the iSight. If you have a MacBook (Pro)
with an integrated iSight camera, EvoCam will let you easily select
which iSight to get the signal from, and if you want to use multiple
cameras, it'll let you do that as well.

OK, so now a user could test, we could watch what happened on-screen
and we could see the user working. What about sound?

4. Since iSight cameras have microphones we had sound in. But we had
to get it out. LineIn (http://www.rogueamoeba.com/freebies/) from
Rogue Amoeba lets you pass the sound through directly to the speakers.
The MacBook Pro's speakers were good enough.

5. In the end we didn't record all of the sessions but ScreenMimic
(http://www.polarian.com/) did a great job of recording the MacBook
Pro screen, which showed the tester's screen and environment.

So: computers, cameras, tripods and long cables aside, the whole
"usability lab" cost about 100 USD. I'd love to hear what setups
you've used to the same effect. Guerilla user testing stories are
welcome.

- Fredrik

PS: You could, of course, re-distribute the observer computer screen
using VNC (Vine VNCserver worked pretty well –
http://sourceforge.net/projects/osxvnc/) but I don't know how you'd
distribute the audio. Tips are welcome.

Comments

27 Sep 2007 - 2:28am
Alexander Baxevanis
2007

Hi Fredrik,
that's a great idea, and I can confirm that it works, as I've also tried it.
My need was to record the screen of a mobile phone. The solution was to
install a VNC server in the phone (yes, such a thing exists, see for example
this one for Symbian mobiles: http://www.m-shell.net/Products-mVNCFull.aspx)
and record the screen of a laptop that was running a VNC viewer and a webcam
viewer. Worked great (apart from a few problems with Audio/Video sync) and
it's the least intrusive way of recording a mobile screen that I've seen so
far.

Cheers,

Alex

On 9/26/07, Fredrik Matheson <fredrik.matheson at gmail.com> wrote:
>
> At work we use Morae (http://techsmith.com/morae.asp) for user
> testing. Quirks aside, it works pretty well.
>
> The other day all our copies of Morae were busy and I needed a way to
> let colleagues observe the use of a website in the room next door.
> Basic requirements were:
>
> 1. a computer that the user would browse from (PC or Mac, it didn't
> matter)
> 2. some way to show what was on the user's screen on another computer
> 3. video of the user (one or more)
> 4. audio of what the user was doing
> 5. hopefully, a way to record all of this
>
> Morae has lots of nice touches like keylogging, a yellow circle around
> the mouse (to aid visibility) and click hinting. It also has a way to
> mark events with keys on the observer's computer ("X" for problem, "L"
> for login, or whatever you prefer). They're valuable features but you
> can manage without them.
>
> 1: Since we had a few iSight cameras for the Mac, it made sense to use
> it for observation and use the spare PC we had as the user's machine.
> Therefore, the procedure for the observer machine below is a Mac.
>
> 2: VNC, or Virtual Network Computing, lets you access another
> computer's screen from your own. That was the easiest way to see what
> the user was doing. VNC uses a server on the host computer, and a
> viewer, on your computer.
> More information about VNC: http://www.cae.wisc.edu/site
> /public/?title=fswhatvnc
>
> The test subject was on a PC laptop and the observer machine was a Mac.
> On the PC, I used TightVNC (http://www.tightvnc.com/) as a server.
> On the Mac, I used JollysFastVNC as a client
> (http://www.jinx.de/JollysFastVNC.html).
>
> We had to open a few ports in the PC's software firewall. VNC uses
> ports 5900–5909 (network experts: feel free to correct me) and made
> sure that the TightVNC server updated the entire screen often (300
> milliseconds) so that there wouldn't be a lot of lag. Then we found
> the IP address of the PC, typed it into JollysFastVNC and voilà, we
> had our user's screen.
>
> 3. An iSight camera on a magnetic mount was placed on a camera tripod
> and aimed at the user. We linked the standard firewire cable from the
> iSight to a firewire hub with a separate power supply and a five meter
> long cable. This let us watch the user from a separate room.
>
> For software, you can use iChat or Quicktime, but I wanted a smaller
> viewer. EvoCam (http://www.evological.com/evocam.html) gives you lots
> of control of the image from the iSight. If you have a MacBook (Pro)
> with an integrated iSight camera, EvoCam will let you easily select
> which iSight to get the signal from, and if you want to use multiple
> cameras, it'll let you do that as well.
>
> OK, so now a user could test, we could watch what happened on-screen
> and we could see the user working. What about sound?
>
> 4. Since iSight cameras have microphones we had sound in. But we had
> to get it out. LineIn (http://www.rogueamoeba.com/freebies/) from
> Rogue Amoeba lets you pass the sound through directly to the speakers.
> The MacBook Pro's speakers were good enough.
>
> 5. In the end we didn't record all of the sessions but ScreenMimic
> (http://www.polarian.com/) did a great job of recording the MacBook
> Pro screen, which showed the tester's screen and environment.
>
> So: computers, cameras, tripods and long cables aside, the whole
> "usability lab" cost about 100 USD. I'd love to hear what setups
> you've used to the same effect. Guerilla user testing stories are
> welcome.
>
> - Fredrik
>
> PS: You could, of course, re-distribute the observer computer screen
> using VNC (Vine VNCserver worked pretty well –
> http://sourceforge.net/projects/osxvnc/) but I don't know how you'd
> distribute the audio. Tips are welcome.
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://beta.ixda.org/guidelines
> List Help .................. http://beta.ixda.org/help
> Unsubscribe ................ http://beta.ixda.org/unsubscribe
> Questions .................. list at ixda.org
> Home ....................... http://beta.ixda.org
>

28 Sep 2007 - 10:23am
Rob Tannen
2006

You can also utilize Morae to capture mobile device screens if you can
"simulcast" the screen on a PC. For example, we used ActiveSync to
capture real-time screen feeds for in-car GPS navigation from PocketPCs
and Smartphones:

http://www.techsmith.com/morae/interview/rtannen.asp

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Alexander Baxevanis
Sent: Thursday, September 27, 2007 3:29 AM
To: Fredrik Matheson
Cc: IxDA Discuss
Subject: Re: [IxDA Discuss] Hack your own usability studio (long
recipe,low price)

Hi Fredrik,
that's a great idea, and I can confirm that it works, as I've also tried
it.
My need was to record the screen of a mobile phone. The solution was to
install a VNC server in the phone (yes, such a thing exists, see for
example
this one for Symbian mobiles:
http://www.m-shell.net/Products-mVNCFull.aspx)
and record the screen of a laptop that was running a VNC viewer and a
webcam
viewer. Worked great (apart from a few problems with Audio/Video sync)
and
it's the least intrusive way of recording a mobile screen that I've seen
so
far.

Cheers,

Alex

29 Sep 2007 - 2:54pm
Michael Lisboa
2007

Fredrik,

We use a similar setup:

20" iMac. A great machine as it has a nice big display built into a
pretty compact form factor. The built in iSight camera actually
produces a fairly good picture and a surprisingly clear audio
recording. We also set up a second mini-dv video camera to record
wide shots to capture body language, etc.

Macbook. For observation we use Apple's Remote Desktop software. OSX
has this capability built in. After testing various free VNC apps, we
decided to just go ahead and buy Apple's Remote Desktop administrator
software. If the testing machine is a PC we simply use Microsoft's
free Remote Desktop software. Both are very reliable and easy to use.
Another option we've considered (but haven't tested) is Quicktime
Broadcaster, which would allow for live streaming of recordings.

Desktop recording with iShowU. We tested a bunch of desktop recording
solutions. iShowU (http://www.shinywhitebox.com/home/home.html) is
hands down the easiest and most reliable software out there.

For remotely observing audio, we simply use a wireless intercom
system we got for $75 from Radio Shack.

After using this setup a few times, we've found it to be useful and
convenient. Having had too many experiences with the traditional
usability lab setup - expensive, sterile, and technologically a pain
in the ass - this cobbled together solution we use is a pleasure to
work with: portable, affordable and it works every time.

The best part is, once we've taken all the collected footage to
editing and post and create a DVD, you'd think we had a $15,000+ lab
setup.

Michael Lisboa

On Sep 28, 2007, at 8:23 AM, Rob Tannen wrote:

>
> You can also utilize Morae to capture mobile device screens if you can
> "simulcast" the screen on a PC. For example, we used ActiveSync to
> capture real-time screen feeds for in-car GPS navigation from
> PocketPCs
> and Smartphones:
>
> http://www.techsmith.com/morae/interview/rtannen.asp
>
>
>
> -----Original Message-----
> From: discuss-bounces at lists.interactiondesigners.com
> [mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
> Alexander Baxevanis
> Sent: Thursday, September 27, 2007 3:29 AM
> To: Fredrik Matheson
> Cc: IxDA Discuss
> Subject: Re: [IxDA Discuss] Hack your own usability studio (long
> recipe,low price)
>
> Hi Fredrik,
> that's a great idea, and I can confirm that it works, as I've also
> tried
> it.
> My need was to record the screen of a mobile phone. The solution
> was to
> install a VNC server in the phone (yes, such a thing exists, see for
> example
> this one for Symbian mobiles:
> http://www.m-shell.net/Products-mVNCFull.aspx)
> and record the screen of a laptop that was running a VNC viewer and a
> webcam
> viewer. Worked great (apart from a few problems with Audio/Video sync)
> and
> it's the least intrusive way of recording a mobile screen that I've
> seen
> so
> far.
>
> Cheers,
>
> Alex
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://beta.ixda.org/guidelines
> List Help .................. http://beta.ixda.org/help
> Unsubscribe ................ http://beta.ixda.org/unsubscribe
> Questions .................. list at ixda.org
> Home ....................... http://beta.ixda.org

29 Sep 2007 - 5:34pm
Fredrik Matheson
2005

Thanks for the Apple Remote Desktop tip. I considered it as well but
wasn't sure how to couple it to a PC. Now I know how – and I'll give
it a try.

Using an intercom for observing audio: ingenious. How much simpler can it get?

I agree about the output; given a nice frame rate from the user's
machine you can get smooth, high-quality video quite easily. Morae's
ability to log events, show mouse clicks and capture keyboard input is
great and the video editor is pretty decent. That said, Final Cut
Express does video editing quite well too.

I'm curious: does Apple Remote Desktop have any features that make the
remote machine's mouse more visible on the observer screen? That's one
thing I'd really like to have in the screen capture videos.

- Fredrik

29 Sep 2007 - 9:25pm
Todd Warfel
2003

We use almost the exact same setup, but open an iChat session on the
remote Mac, then mute the mic in the observation room to broadcast
the audio from the testing room to the observation room. We use a set
of speakers plugged into the Mac for sound.

On Sep 29, 2007, at 3:54 PM, Michael Lisboa wrote:

> 20" iMac. A great machine as it has a nice big display built into a
> pretty compact form factor. The built in iSight camera actually
> produces a fairly good picture and a surprisingly clear audio
> recording. We also set up a second mini-dv video camera to record
> we shots to capture body language, etc.
>
> Macbook. For observation we use Apple's Remote Desktop software.
> OSX has this capability built in. After testing various free VNC
> apps, we decided to just go ahead and buy Apple's Remote Desktop
> administrator software. If the testing machine is a PC we simply
> use Microsoft's free Remote Desktop software. Both are very
> reliable and easy to use.
> Another option we've considered (but haven't tested) is Quicktime
> Broadcaster, which would allow for live streaming of recordings.
>
> Desktop recording with iShowU. We tested a bunch of desktop
> recording solutions. iShowU (http://www.shinywhitebox.com/home/
> home.html) is hands down the easiest and most reliable software out
> there.
>
> For remotely observing audio, we simply use a wireless intercom
> system we got for $75 from Radio Shack.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

29 Sep 2007 - 6:42pm
Michael Lisboa
2007

Unfortunately, as far as I know Remote Desktop (Mac or Windows)
doesn't have any built in feature to increase visibility or the mouse
cursor for observers. In the past, I've added a "halo" around the
cursor in post through tracking. However, a fast and dirty solution
might be to use the operating system's accessibility feature to
increase the mouse cursor size (on Mac OS it's in the Universal
Access preferences), although this simply increases the cursor size
on the testing machine, making it more visible for everybody involved
-- including your test participants.

Michael Lisboa

Syndicate content Get the feed