Usability testing software for Mac?

31 Jan 2007 - 2:11pm
7 years ago
4 replies
3156 reads
Josephine Choi
2005

Hi everyone,

My company's working on a new Mac product that we will be doing usability
testing on. We use Morae for our Windows products and like the features it
provides.

We're looking to find something comparable for testing Mac software. We are
doing testing on-site, so remote testing software is not necessary.
However, we would like reliable screen capture. Remote observer capability
and real-time marking/flagging would also be nice to haves. For those who
have experience with testing Mac software, would you be able to share your
experiences and process? What products do you use?

Thanks for your input,
Josephine

PS - Also posted on the sig-ia disc group, sorry if you receive this twice!

Comments

6 Feb 2007 - 4:49pm
Jay Zipursky
2005

Hi Josephine,

Has anyone replied off-list? The only "product" I know of is something
called VisualMark by UsersFirst (http://www.usersfirst.com/). It's "beta",
but I'm not sure it's under active development anymore. Maybe the author
reads this list and can chime in...

Anyway, it's definitely not a competitor to Morae and is quite rough from
what I've seen. I've never used it for a test.

You may want to try rigging up something like a VNC connection to the test
computer and use Morae to record the screen remotely. (I believe this is how
VisualMark works, actually.) This is not the best solution, but it may be
better than nothing.

Let me know what you end up doing.

Jay
Kodak

On 1/31/07, Josephine Choi <josephine.c.choi at gmail.com> wrote:
>
> Hi everyone,
>
> My company's working on a new Mac product that we will be doing usability
> testing on. We use Morae for our Windows products and like the features
> it
> provides.
>
> We're looking to find something comparable for testing Mac software. We
> are
> doing testing on-site, so remote testing software is not necessary.
> However, we would like reliable screen capture. Remote observer
> capability
> and real-time marking/flagging would also be nice to haves. For those who
> have experience with testing Mac software, would you be able to share your
> experiences and process? What products do you use?
>
> Thanks for your input,
> Josephine
>
> PS - Also posted on the sig-ia disc group, sorry if you receive this
> twice!
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org
>

5 Mar 2007 - 6:47pm
Josephine Choi
2005

Hi Jay,

Sorry for the late response, but I hope better late than never.

I have had a few responses on and off-list, let me see if I can capture them.

Visual Mark - I got this response a few times. I checked it out and
the software looks great, but I couldn't get it hooked up in time as
it required Mac to record the session and another computer. I only
had 1 Mac, and was conducting testing on it.

Ovo - Seemed promising as well, but I couldn't get the set up nor
funding in time for the test. I signed up for a free trial for Ovo
Logger, but I didn't see any help on using it on a Mac, so I made a
quick decision to put it on the back burner. Still, it comes highly
recommended by someone who has set it up and conducted several
successful Mac studies on it.

Vine viewer - remote desktop viewer. I tried setting this up, but
there was considerable lag between what I was seeing on my Windows
machine and what was happening on the Mac. In addition, the cursor
wasn't visible.

Apple Remote Desktop - another remote desktop viewer, but I must
admit, I didn't try this.

Redstone Eggplant - another one that I didn't investigate too deeply
due to time.

SnapZPro - for screen capture - it had by far the best fidelity - no
lag and it caught all mouse clicks. You could also record audio.
Unfortunately, it took forever for me to save/encode the file (on the
order of hours!) so it wasn't feasible for me to use.

There are also other screen capture tools for the Mac that I tried.
They saved/encoded faster than SnapZPro, but still on the order of
almost an hour. They lagged and the video quality was quite poor,
skipping over mouse clicks, etc.

So after all this investigation and desperately trying to get Snapzpro
to encode faster, I hacked a user study set-up (another suggestion):

I set up WebEx conferences and had a Windows machine with Morae and
the Mac testing computer in the same conference. I shared the Mac
desktop (make sure it's the host), and thus I was able to record the
Mac desktop off the Windows machine using Morae. In addition I was
able to use Morae's video and audio capturing.

There was a bit of lag, but I could still see the cursor moving and
most mouse clicks. It kept up pretty well with real time. The main
problem I encountered was with network connectivity - my windows
machine got disconnected from the conference or couldn't sign on.
Still, it was a good fix for my needs.

Hope this helps! Feel free to email me if you have any other questions.

- Josephine

On 2/6/07, Jay Zipursky <jay.zipursky at gmail.com> wrote:
> Hi Josephine,
>
> Has anyone replied off-list? The only "product" I know of is something
> called VisualMark by UsersFirst (http://www.usersfirst.com/). It's "beta",
> but I'm not sure it's under active development anymore. Maybe the author
> reads this list and can chime in...
>
> Anyway, it's definitely not a competitor to Morae and is quite rough from
> what I've seen. I've never used it for a test.
>
> You may want to try rigging up something like a VNC connection to the test
> computer and use Morae to record the screen remotely. (I believe this is how
> VisualMark works, actually.) This is not the best solution, but it may be
> better than nothing.
>
> Let me know what you end up doing.
>
> Jay
> Kodak
>
>
> On 1/31/07, Josephine Choi <josephine.c.choi at gmail.com > wrote:
> >
> > Hi everyone,
> >
> > My company's working on a new Mac product that we will be doing usability
> > testing on. We use Morae for our Windows products and like the features
> it
> > provides.
> >
> > We're looking to find something comparable for testing Mac software. We
> are
> > doing testing on-site, so remote testing software is not necessary.
> > However, we would like reliable screen capture. Remote observer
> capability
> > and real-time marking/flagging would also be nice to haves. For those who
> > have experience with testing Mac software, would you be able to share your
> > experiences and process? What products do you use?
> >
> > Thanks for your input,
> > Josephine
> >
> > PS - Also posted on the sig-ia disc group, sorry if you receive this
> twice!
> >
> ________________________________________________________________
> > Welcome to the Interaction Design Association (IxDA)!
> > To post to this list ....... discuss at ixda.org
> > List Guidelines ............ http://listguide.ixda.org/
> > List Help .................. http://listhelp.ixda.org/
> > (Un)Subscription Options ...
> http://subscription-options.ixda.org/
> > Announcements List .........
> http://subscribe-announce.ixda.org/
> > Questions .................. lists at ixda.org
> > Home ....................... http://ixda.org/
> > Resource Library ........... http://resources.ixda.org
> >
>
>

5 Mar 2007 - 7:08pm
Josh
2006

Josephine,
Thanks for the insight. I've been looking (on and off) for reasonable
solutions to this same issue for a couple of months.

On the topic of hacked solutions for usability testing.

At a previous company, we had a flash communication server and used flash w/
webcams to record and broadcast the usability tests. It was a pretty quick
hack and we didn't have time to add in nice stuff like the ability to add
notes to keyframes. The video quality was decent and we were able to use
multiple webcams (one for the screen and another for the person's face), and
it allowed everyone in the company to view the live broadcast. The audio
quality was poor, but it was most likely a product of the webcams. I'm going
to see if I can dig up some more details (it was 3 years ago). At the time,
it seemed like it could have been built out to be a decent inexpensive
solution.

--
Josh Viney
EastMedia Group
http://www.eastmedia.com

6 Mar 2007 - 10:49pm
Pete Gordon - U...
2004

Hello.

Any questions, comments or anything on VisualMark don't hesitate to
contact me pete [at] usersfirst.com .

Although, VisualMark can capture from anything running a VNC Server
(Windows 95, Me, 2000, XP, Mac 7,8,9, Linux, AIX, etc.,etc.). It is
not perfect when capturing a Mac screen because of the dependency on
the VNC Server on the user machine (OSX VNC). Redstone Eggplant and
Vine Viewer, which I believe are the same thing under the hood (both
are done by Redstone) and I believe both are VNC clients that go
against a VNC Server running on the Mac, are not pretty, no cursor
and long lag. VisualMark is similar to the Redstone products when
using a VNC Server on a Mac OS X box. I haven't had a lot of demand
to capture the Mac as of yet.

SnapZPro is great for screen capturing on Mac--but as you saw the
rendering takes forever.

I had been working on integrating a hardware product (VGA2USB,
www.epiphan.com) to improve the screen capture capabilities of
VisualMark and make it even more suitable for multiple camera
ethnographic studies in peoples homes and remote locations using a
MacBook and the user computer, POS device, kiosk, or whatever--but
the work on integrating VGA2USB has stalled as issues have arisen,
and other priorities/features have emerged. I think using the
hardware (VGA2USB) solution integrated with VisualMark would be the
ideal solution for capturing from all platforms and avoiding the Mac
and Linux capture lag. Let me know your thoughts, on or off list.
If there is a lot of need for a solution, I'll get a solution asap
(but the issues are not trivial--smile). There have been three big
competing priorities for VisualMark, 1. VGA2USB, 2. Wireless 802.11b
cameras, 3. Tobii integration 4. VNC Performance improvements on Mac
and Linux. Feel free to vote on your favorite improvements for
VisualMark or add to the list and send me your favorite feature
requests to features at usersfirst.com.

http://www.epiphan.com/products/product.php?pid=1

<PLUG>
I do have customers using VisualMark and having a lot of success with
it--drop me a line if your interested and I'll tell you more.

VisualMark is a two computer User Experience solution. The
VisualMark lab runs on Mac OS X and allows you to capture multiple
cameras, audio, and the users screen together into a simple single
Quicktime Movie. In addition, it synchronizes Observation Notes and/
or Task Lists that link directly to your marked observations within
the Quicktime movie using cross-platform standards based HTML and
Quicktime Reports.
</PLUG>

Thanks!
Pete Gordon
pete [at] usersfirst.com
Users First
Columbus, Ohio USA
614.360.1919 [ office ]
866.392.5222 [ fax ]

"The test of the machine is the satisfaction it gives you. There
isn't any other test. If the machine produces tranquillity it's
right. If it disturbs you it's wrong until either the machine or
your mind is changed." -- Zen and the Art of Motorcycle Maintenance

On Mar 5, 2007, at 6:47 PM, Josephine Choi wrote:

> Hi Jay,
>
> Sorry for the late response, but I hope better late than never.
>
> I have had a few responses on and off-list, let me see if I can
> capture them.
>
> Visual Mark - I got this response a few times. I checked it out and
> the software looks great, but I couldn't get it hooked up in time as
> it required Mac to record the session and another computer. I only
> had 1 Mac, and was conducting testing on it.
>
> Ovo - Seemed promising as well, but I couldn't get the set up nor
> funding in time for the test. I signed up for a free trial for Ovo
> Logger, but I didn't see any help on using it on a Mac, so I made a
> quick decision to put it on the back burner. Still, it comes highly
> recommended by someone who has set it up and conducted several
> successful Mac studies on it.
>
> Vine viewer - remote desktop viewer. I tried setting this up, but
> there was considerable lag between what I was seeing on my Windows
> machine and what was happening on the Mac. In addition, the cursor
> wasn't visible.
>
> Apple Remote Desktop - another remote desktop viewer, but I must
> admit, I didn't try this.
>
> Redstone Eggplant - another one that I didn't investigate too deeply
> due to time.
>
> SnapZPro - for screen capture - it had by far the best fidelity - no
> lag and it caught all mouse clicks. You could also record audio.
> Unfortunately, it took forever for me to save/encode the file (on the
> order of hours!) so it wasn't feasible for me to use.
>
> There are also other screen capture tools for the Mac that I tried.
> They saved/encoded faster than SnapZPro, but still on the order of
> almost an hour. They lagged and the video quality was quite poor,
> skipping over mouse clicks, etc.
>
> So after all this investigation and desperately trying to get Snapzpro
> to encode faster, I hacked a user study set-up (another suggestion):
>
> I set up WebEx conferences and had a Windows machine with Morae and
> the Mac testing computer in the same conference. I shared the Mac
> desktop (make sure it's the host), and thus I was able to record the
> Mac desktop off the Windows machine using Morae. In addition I was
> able to use Morae's video and audio capturing.
>
> There was a bit of lag, but I could still see the cursor moving and
> most mouse clicks. It kept up pretty well with real time. The main
> problem I encountered was with network connectivity - my windows
> machine got disconnected from the conference or couldn't sign on.
> Still, it was a good fix for my needs.
>
> Hope this helps! Feel free to email me if you have any other
> questions.
>
> - Josephine
>

Syndicate content Get the feed