Mobile phones usability research - Advices on video recording

6 Nov 2007 - 3:50pm
6 years ago
9 replies
1532 reads
maglez@btintern...
2006

Hello,

I am going to run a usability research on the BlackBerry Curve, for which I
want to video record the device screen for a later detailed analysis.

I got a couple of camcorders, cheap camcorders, to discover that the
resolution they offer is so poor that they are of no value for this
research.

Apart from using a good quality camcorder, do you guys have any other advice
on how to set up a test that requires video recording of a small screen?

I am going to use to cameras, one recording the interaction with the device,
the other recording the user's faces, for later synchronize and see what
part of the interface gives more problems. Any advice on video
synchronization?

Any other comment?

Thanks a lot.

Miguel Gonzalez.

Comments

6 Nov 2007 - 4:46pm
Barbara Ballard
2005

On 11/6/07, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> I am going to run a usability research on the BlackBerry Curve, for which I
> want to video record the device screen for a later detailed analysis.
>
> I got a couple of camcorders, cheap camcorders, to discover that the
> resolution they offer is so poor that they are of no value for this
> research.

For our clients, we use a sled with three cameras attached: face,
screen, and keypad. These are tiny security cameras with different
lenses on them. We run the three cameras, plus a wideshot camera, into
a portable quad mixer (targeted at the security folks, again, but
works fine for us). We then record the four images.

Synchronization is very difficult. My photo specialist explained it to
me once, and I don't have it fully integrated. The explanation
included the fact that different cameras run at slightly different
speeds, causing mismatches to accumulate.

I've got a presentation up on slideshare on usability testing, it
includes some pictures of different setups.
http://www.slideshare.net/barbaraballard/mobile-usability-testing

--
Barbara Ballard
barbara at littlespringsdesign.com 1-785-838-3003

6 Nov 2007 - 6:24pm
maglez@btintern...
2006

Interesting Barbara, I saw your slides and son I was looking for Elmo
cameras :-)

As you have tried them, Can you say that the cameras are able to capture the
mobile screen in details, with good quality?

Thanks.

6 Nov 2007 - 8:23pm
Barbara Ballard
2005

On 11/6/07, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> Interesting Barbara, I saw your slides and son I was looking for Elmo
> cameras :-)
>
> As you have tried them, Can you say that the cameras are able to capture the
> mobile screen in details, with good quality?
>

Scott Weiss can probably articulate Elmo benefits better; in my
experience they are good for flat screens (candy bar phones but not
clamshells) and they inhibit natural use of the phone. The camera
quality is plenty good for usability work. I've used them for
(surprise!) presentation work.

--
Barbara Ballard
barbara at littlespringsdesign.com 1-785-838-3003

7 Nov 2007 - 12:19am
Jeff Axup
2006

Miguel,

Can you provide a little more detail on what the purpose of the test is?

If you are doing "usability testing of software" such as watching tasks, and
are interested in the detailed interaction, you are probably headed down the
right track and Barbara's recommended tools are great.

However, if you're looking for the higher-level behaviors of using the
device in a natural social/physical context there are other more appropriate
methods/tools.

Also, there is a growing trend towards not using video camcorders in general
for usability testing. Increasingly people are using screen capture
utilities paired with audio recorders (and sometimes a web-cam in addition).
This can still get a lot of usability data, but be a lot less cumbersome and
less stressful for the user since they don't have a video camera in their
face. Also works better for remote testing. I'm not sure if software like
this is available on many mobile devices.

Cheers,
Jeff

On 11/6/07, Barbara Ballard <barbara at littlespringsdesign.com> wrote:
>
> On 11/6/07, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> > Interesting Barbara, I saw your slides and son I was looking for Elmo
> > cameras :-)
> >
> > As you have tried them, Can you say that the cameras are able to capture
> the
> > mobile screen in details, with good quality?
> >
>
> Scott Weiss can probably articulate Elmo benefits better; in my
> experience they are good for flat screens (candy bar phones but not
> clamshells) and they inhibit natural use of the phone. The camera
> quality is plenty good for usability work. I've used them for
> (surprise!) presentation work.
>
> --
> Barbara Ballard
> barbara at littlespringsdesign.com 1-785-838-3003
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

--
Thanks,
Jeff
________________________________________________________________________________
Jeff Axup, Ph.D.
Principal Consultant, Mobile Community Design Consulting, San Diego

Research: Mobile Group Research Methods, Social Networks, Group Usability
E-mail: axup <at> userdesign.com
Blog: http://mobilecommunitydesign.com
Moblog: http://memeaddict.blogspot.com

"Designers mine the raw bits of tomorrow. They shape them for the present
day." - Bruce Sterling
________________________________________________________________________________

7 Nov 2007 - 2:49am
maglez@btintern...
2006

Hi Jeff,

I already tried screen capture. The device I am going to do my research on
is the BlackBerry Curve, http://tinyurl.com/3ctk2w, I found two screen
capture for BlackBerry devices, http://tinyurl.com/2qeles and
http://tinyurl.com/3e3mp6 but those screen capture isn't good enough, the
way they work is by taking screenshots, but they only take screenshots every
2 seconds, 1 second or 0.5 second. Because it results too intense, many
times the user has to press a key twice for the device to capture the user
interaction.

Jeff, I am also interested in a higher level research, I would like to be
able to analyse user's behaviour in a social context. What would you
recommend?

Let me give more details...

I have been working on IT for the last 13 years, I started on Graphics
Design, then Web and Multimedia Development, later I started with more
serious programming with OOP languages and then continuing through the path
of GUI, Usability and User Experience.

I am one of those persons that learned everything through books and
experimentation, no formal qualifications. Now I want to get a Master in
Software Engineering which I am doing through the University of Oxford.

For my latest subject, Requirements Engineering, one of the tasks I have to
do for my essay is a usability research on a piece of technology of my
choice. I have chosen the BlackBerry Curve and I am going to focus my
research on the email functionality, and the user as a novice user. As the
BlackBerry Curve is trying to break the trend of only corporate users and
trying to reach individuals, as they also feel the push from other smart
phones, even more when the iPhone is out. Already in America, this week in
Europe (England and Germany I believe).

As a social behaviour, I think I could focus on the way this device brings
closer communication between people, I got this device two weeks ago and I
have noticed a huge difference as the phone doesn't make much different
between SMS and emails, and so shortening the response time for emails.

Usually, we could say that a phone call is answered immediately, a SMS
within 5 minutes but an email can take hours to be answered, smart phones
brings emails to the level of SMS.

Thanks a lot for your comments. More comments?

Miguel Gonzalez.

8 Nov 2007 - 11:29am
pnuschke
2007

Miguel,

I just ran a study where I had to record the screens of such devices.
If you want to be able to read the screen, you have to get a high
definition camcorder. I used a Sony hard-drive camcorder so that it
would be easier to send the files to our clients. However, the file
are so big from a two hour session that I will have to ship them on an
external hard drive because they don't fit on a DVD! The compression
algorithms unfortunately loose too much fidelity. I would highly
recommend only recording certain tasks this way.

Paul Nuschke
Electronic Ink

On Nov 6, 2007 3:50 PM, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> Hello,
>
> I am going to run a usability research on the BlackBerry Curve, for which I
> want to video record the device screen for a later detailed analysis.
>
> I got a couple of camcorders, cheap camcorders, to discover that the
> resolution they offer is so poor that they are of no value for this
> research.
>
> Apart from using a good quality camcorder, do you guys have any other advice
> on how to set up a test that requires video recording of a small screen?
>
> I am going to use to cameras, one recording the interaction with the device,
> the other recording the user's faces, for later synchronize and see what
> part of the interface gives more problems. Any advice on video
> synchronization?
>
> Any other comment?
>
> Thanks a lot.
>
> Miguel Gonzalez.
>
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

8 Nov 2007 - 6:48pm
maglez@btintern...
2006

Thanks a lot Paul,

Yes, I got a couple of bullet cameras and a high definition Sony, the
difference is astonishing.

I will use the Sony for the screen and the bullet cameras for the user's
face and the keyboard.

Again, thanks a lot.

Miguel Gonzalez.

-----Original Message-----
From: Paul Nuschke [mailto:plnii11 at gmail.com]
Sent: 08 November 2007 16:30
To: Miguel
Cc: IxDA Discuss
Subject: Re: [IxDA Discuss] Mobile phones usability research - Advices on
video recording

Miguel,

I just ran a study where I had to record the screens of such devices.
If you want to be able to read the screen, you have to get a high
definition camcorder. I used a Sony hard-drive camcorder so that it
would be easier to send the files to our clients. However, the file
are so big from a two hour session that I will have to ship them on an
external hard drive because they don't fit on a DVD! The compression
algorithms unfortunately loose too much fidelity. I would highly
recommend only recording certain tasks this way.

Paul Nuschke
Electronic Ink

On Nov 6, 2007 3:50 PM, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> Hello,
>
> I am going to run a usability research on the BlackBerry Curve, for which
I
> want to video record the device screen for a later detailed analysis.
>
> I got a couple of camcorders, cheap camcorders, to discover that the
> resolution they offer is so poor that they are of no value for this
> research.
>
> Apart from using a good quality camcorder, do you guys have any other
advice
> on how to set up a test that requires video recording of a small screen?
>
> I am going to use to cameras, one recording the interaction with the
device,
> the other recording the user's faces, for later synchronize and see what
> part of the interface gives more problems. Any advice on video
> synchronization?
>
> Any other comment?
>
> Thanks a lot.
>
> Miguel Gonzalez.
>
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

15 Nov 2007 - 3:12am
maglez@btintern...
2006

Hi Jeff,

I ran a series of interviews last weekend to get an initial idea of where
problems may arise, it was quite interesting. I am going to do more specific
tests this weekend focusing on what I found last time. I am waiting to
finish those to post my experiences, some results and possible improvements.

Miguel.

From: meme.addict at gmail.com [mailto:meme.addict at gmail.com] On Behalf Of Jeff
Axup
Sent: 15 November 2007 01:54
To: Miguel
Subject: Re: [IxDA Discuss] Mobile phones usability research - Advices on
video recording

Hi Miguel,
Feel free to follow up with me privately.
It might be interesting to do part of your research qualitative and half
quantitative. the cameras will help you track specific details during preset
tasks. if you want to know more about natural usage you'll need to shadow
some people or use surveys or something similar. Sometimes the bigger social
issues like social and physical context can be more important than the
detailed task efficiency issues.
Cheers,
Jeff

On Nov 6, 2007 11:49 PM, Miguel <miguel.gonzalez at o2.co.uk> wrote:

Hi Jeff,

I already tried screen capture. The device I am going to do my research on
is the BlackBerry Curve, http://tinyurl.com/3ctk2w, I found two screen
capture for BlackBerry devices, http://tinyurl.com/2qeles and
http://tinyurl.com/3e3mp6 but those screen capture isn't good enough, the
way they work is by taking screenshots, but they only take screenshots every
2 seconds, 1 second or 0.5 second. Because it results too intense, many
times the user has to press a key twice for the device to capture the user
interaction.

Jeff, I am also interested in a higher level research, I would like to be
able to analyse user's behaviour in a social context. What would you
recommend?

Let me give more details...

I have been working on IT for the last 13 years, I started on Graphics
Design, then Web and Multimedia Development, later I started with more
serious programming with OOP languages and then continuing through the path
of GUI, Usability and User Experience.

I am one of those persons that learned everything through books and
experimentation, no formal qualifications. Now I want to get a Master in
Software Engineering which I am doing through the University of Oxford.

For my latest subject, Requirements Engineering, one of the tasks I have to
do for my essay is a usability research on a piece of technology of my
choice. I have chosen the BlackBerry Curve and I am going to focus my
research on the email functionality, and the user as a novice user. As the
BlackBerry Curve is trying to break the trend of only corporate users and
trying to reach individuals, as they also feel the push from other smart
phones, even more when the iPhone is out. Already in America, this week in
Europe (England and Germany I believe).

As a social behaviour, I think I could focus on the way this device brings
closer communication between people, I got this device two weeks ago and I
have noticed a huge difference as the phone doesn't make much different
between SMS and emails, and so shortening the response time for emails.

Usually, we could say that a phone call is answered immediately, a SMS
within 5 minutes but an email can take hours to be answered, smart phones
brings emails to the level of SMS.

Thanks a lot for your comments. More comments?

Miguel Gonzalez.

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto: <mailto:discuss-bounces at lists.interactiondesigners.com>
discuss-bounces at lists.interactiondesigners.com] On Behalf Of Jeff
Axup
Sent: 07 November 2007 05:19
To: IxDA Discuss
Subject: Re: [IxDA Discuss] Mobile phones usability research - Advices on
video recording

Miguel,

Can you provide a little more detail on what the purpose of the test is?

If you are doing "usability testing of software" such as watching tasks, and

are interested in the detailed interaction, you are probably headed down the
right track and Barbara's recommended tools are great.

However, if you're looking for the higher-level behaviors of using the
device in a natural social/physical context there are other more appropriate
methods/tools.

Also, there is a growing trend towards not using video camcorders in general
for usability testing. Increasingly people are using screen capture
utilities paired with audio recorders (and sometimes a web-cam in addition).
This can still get a lot of usability data, but be a lot less cumbersome and
less stressful for the user since they don't have a video camera in their
face. Also works better for remote testing. I'm not sure if software like
this is available on many mobile devices.

Cheers,
Jeff

On 11/6/07, Barbara Ballard < <mailto:barbara at littlespringsdesign.com>
barbara at littlespringsdesign.com> wrote:
>
> On 11/6/07, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> > Interesting Barbara, I saw your slides and son I was looking for Elmo
> > cameras :-)
> >
> > As you have tried them, Can you say that the cameras are able to capture
> the
> > mobile screen in details, with good quality?
> >
>
> Scott Weiss can probably articulate Elmo benefits better; in my
> experience they are good for flat screens (candy bar phones but not
> clamshells) and they inhibit natural use of the phone. The camera
> quality is plenty good for usability work. I've used them for
> (surprise!) presentation work.
>
> --
> Barbara Ballard
> barbara at littlespringsdesign.com 1-785-838-3003
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

--
Thanks,
Jeff
____________________________________________________________________________

____
Jeff Axup, Ph.D.
Principal Consultant, Mobile Community Design Consulting, San Diego

Research: Mobile Group Research Methods, Social Networks, Group Usability
E-mail: axup <at> userdesign.com
Blog: http://mobilecommunitydesign.com
Moblog: http://memeaddict.blogspot.com
<http://memeaddict.blogspot.com>

"Designers mine the raw bits of tomorrow. They shape them for the present
day." - Bruce Sterling
____________________________________________________________________________
____
________________________________________________________________
*Come to IxDA Interaction08 | Savannah*
February 8-10, 2008 in Savannah, GA, USA
Register today: http://interaction08.ixda.org/

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

________________________________________________________________
*Come to IxDA Interaction08 | Savannah*
February 8-10, 2008 in Savannah, GA, USA
Register today: http://interaction08.ixda.org/

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

--
Thanks,
Jeff
____________________________________________________________________________
____
Jeff Axup, Ph.D.
Principal Consultant, Mobile Community Design Consulting, San Diego

Research: Mobile Group Research Methods, Social Networks, Group Usability

E-mail: axup <at> userdesign.com
Blog: http://mobilecommunitydesign.com
Moblog: http://memeaddict.blogspot.com

"Designers mine the raw bits of tomorrow. They shape them for the present
day." - Bruce Sterling
____________________________________________________________________________
____

3 Mar 2008 - 2:37am
maglez@btintern...
2006

Hello everyone...

I finished my research and submitted my assay though I haven't got my mark
yet :-(

I would like to thankful all of you that contributed with your comments. As
a reminder, I was running a research on the BlackBerry Curve mobile phone
for which I was asking for advices on how to record the tiny screen.

>From all the comments I remember Barbara's on multiple sources
synchronization for which I got a Quad, a device that get up to 4 difference
camera sources and mixed all together. This eliminates the synchronization
issue and also allows me to see all the cameras on one single screen.

To record the mobile screen I used a high definition Sony camera that I
borrowed from a friend. I just have to turn off any auto adjustment like
light and focus, since the user had the device on their hands, some
movements made the camera to auto adjust light and focus, making things very
bad for a second or two.

A very good choice was the bullet cameras I bought, those cameras are so
small that participants ignored them very soon; they showed no concern about
the cameras once they started the task. At first, while I asked them about
their consent to be recorded, they used to look at the camera and send a
salutation :-) but they forgot about the camera very soon as it was
unnoticeable.

All the cameras, bullet cameras and Sony camera were plugged to a Quad mixer
which output were to a digital recorder. This digital recorder is again
small so it was very easy for me to have it in from of me to monitor that
the recording was fine.

I am very happy with the equipment; it was like my little TV studio, my own
research lab.

For those interested, I am selling all this equipment on eBay
http://tinyurl.com/2pvd3t and I know many of you may be interested on it.

Again, thank you very much to all of you that drop me a comment.

Miguel Gonzalez.

-----Original Message-----
From: Paul Nuschke [mailto:plnii11 at gmail.com]
Sent: 08 November 2007 16:30
To: Miguel
Cc: IxDA Discuss
Subject: Re: [IxDA Discuss] Mobile phones usability research - Advices on
video recording

Miguel,

I just ran a study where I had to record the screens of such devices.
If you want to be able to read the screen, you have to get a high
definition camcorder. I used a Sony hard-drive camcorder so that it
would be easier to send the files to our clients. However, the file
are so big from a two hour session that I will have to ship them on an
external hard drive because they don't fit on a DVD! The compression
algorithms unfortunately loose too much fidelity. I would highly
recommend only recording certain tasks this way.

Paul Nuschke
Electronic Ink

On Nov 6, 2007 3:50 PM, Miguel <miguel.gonzalez at o2.co.uk> wrote:
> Hello,
>
> I am going to run a usability research on the BlackBerry Curve, for which
I
> want to video record the device screen for a later detailed analysis.
>
> I got a couple of camcorders, cheap camcorders, to discover that the
> resolution they offer is so poor that they are of no value for this
> research.
>
> Apart from using a good quality camcorder, do you guys have any other
advice
> on how to set up a test that requires video recording of a small screen?
>
> I am going to use to cameras, one recording the interaction with the
device,
> the other recording the user's faces, for later synchronize and see what
> part of the interface gives more problems. Any advice on video
> synchronization?
>
> Any other comment?
>
> Thanks a lot.
>
> Miguel Gonzalez.
>
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

Syndicate content Get the feed