User Research: Three user groups, five hours. What would you do?

2 Feb 2009 - 2:06pm
5 years ago
17 replies
4967 reads
jmcatee
2008

I am being given the opportunity to visit a client of our software to
do some user research. I have never been on site with any of our
clients before.

At this client we have 3 user groups two who do different tasks and
one super user who can do everything plus config.

I only have five maybe six hours depending on flights being on time
etc.

We have full access to both groups all day but I would like to send
them some sort of schedule so they are not just sitting around
waiting on us.

Goals:
1. Understand the users and how they use the software.
2. Uncover any pain points.
3. Bring back recommendations for changes.

Right now my plan is to do a contextual inquiry and watch one member
of each group work through their tasks using the software. Then move
to either an interview or a focus group of what they think about the
software.

As far as time most of it will be spent with the individual users and
not the super user.

This is the first time the company has sent anyone to do something
like this.I am hoping it goes well and we can bring back good
information so we can do some more of these visits.

Advice is appreciated!

Comments

2 Feb 2009 - 2:22pm
ELISABETH HUBERT
2007

Curious as to how long you are planning for the individual interviews.
This may determine you're follow up activity. I think the approach of
watching them in their environment and taking notes first hand is
great. You should be able to fulfill most of your goals with just
that one activity, but with the time constraints i can see how you'd
want to find out more info.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=38073

2 Feb 2009 - 2:41pm
Dana Chisnell
2008

Jamie,

I think this is a super plan. You might want to come up with 2-4
Big Questions that you want to observe for during the individual
sessions. This doesn't seem like very much, but the point is to
observe as much as possible rather than interviewing. Specifics
usually reveal themselves along the way during the session.

You could also consider asking the participants for artifacts that
relate to what you want to find out, like reports or forms that are
inputs to their tasks or reports, emails, or whatever that might be
outputs of their tasks. Think about looking at how your product fits
into their overall work.

Get someone to help you take notes, if at all possible. Running
these kinds of sessions and observing closely takes a lot of
concentration and effort. If you have someone else to jot down notes,
it can really free you up to ask follow-up questions and just be
present.

Consider taking digital photographs of your users' work spaces and
any forms or other artifacts that they use that you can't take copies
away with you.

Also consider recording the sessions on digital audio, even if you
do have a note taker.

Closing the day with a group discussion is a great idea. Again, I
recommend having 1 or 2 topics that you want to talk about as a group,
remembering that each person will take a minute or two to answer the
question or contribute to the discussion about each topic you ask about.

What are you going to do with the data you collect?

Dana
:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::
Dana Chisnell
desk: 415.392.0776
mobile: 415.519.1148

dana AT usabilityworks DOT net

www.usabilityworks.net
http://usabilitytestinghowto.blogspot.com/

On Feb 2, 2009, at 11:06 AM, Jamie McAtee wrote:

> I am being given the opportunity to visit a client of our software to
> do some user research. I have never been on site with any of our
> clients before.
>
> At this client we have 3 user groups two who do different tasks and
> one super user who can do everything plus config.
>
> I only have five maybe six hours depending on flights being on time
> etc.
>
> We have full access to both groups all day but I would like to send
> them some sort of schedule so they are not just sitting around
> waiting on us.
>
> Goals:
> 1. Understand the users and how they use the software.
> 2. Uncover any pain points.
> 3. Bring back recommendations for changes.
>
> Right now my plan is to do a contextual inquiry and watch one member
> of each group work through their tasks using the software. Then move
> to either an interview or a focus group of what they think about the
> software.
>
> As far as time most of it will be spent with the individual users and
> not the super user.
>
> This is the first time the company has sent anyone to do something
> like this.I am hoping it goes well and we can bring back good
> information so we can do some more of these visits.
>
> Advice is appreciated!
>
>
> ________________________________________________________________
> Reply to this thread at ixda.org
> http://www.ixda.org/discuss?post=38073
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

2 Feb 2009 - 3:59pm
Nicholas Iozzo
2007

I have found it is generally better to open the day with a group
meeting. No matter how much explaining you do ahead of time, you will
still likely be scheduled to meet with the wrong folks.

The group meeting will allow the managers to have their say and give
them a forum to tell you many things. You can then use that to start
asking specific task questions. Usually during the course of these
questions, they will say "Well, Sue does that". Great time to say,
"I'd love to spend some time with Sue later today then to see how
she does it."

I have found that during the course of this opening meeting, names of
individuals get brought up as the person who does X. This is the best
way for you to then select whom you want to meet with and learn more
about task x.

Of course, as the prior posted mentioned. Preparation is the most
important thing to do. Know what you want to observe, know how many
events you want to observe, have research questions you want
answered. Prepare a study guide....

During the course of the day, you will be presented with more
opportunities then you have time to follow-up on. You need to have
spent time developing your study guide so you can make on-the-fly
decisions on how to best use your time.

Even if your research techniques are all about not interfering with
the user and letting it naturally flow. If you have not made
decisions about what you want to learn, then you will not learn
anything.

This is more practical then ideal. Ideally you would have lots of
time to spend with everyone, so you will be able to learn all you
can. Practically, you have a very limited amount of time to spend
with a limited number of folks. So you have to plan on how to use it
wisely.

Get clearance before you even bring out any recording devices. Many
companies do not like it.

Good luck.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=38073

2 Feb 2009 - 4:42pm
jmcatee
2008

@Elizabeth I was planning on talking to the user after I spend some
time observing them. I am not sure how many people they will really
free up for us so if we have a number of people in that group I might
do a focus group instead.

@Dana What we are trying to do is understand our user base more and
take that to propose changes to the software. We know there are some
issues and others that seem like issues but may not be. I have been
through the full work flow with the business analyst for this product
so I know the way we think they do things but I am not sure if this is
the right model. Thanks for the suggestions I am working on the plan
right now and your ideas really help.

@Nicholas I was thinking of doing a meeting in the morning to explain
who we are and why we are here. I hadn't thought of using it to
figure out exactly who to target but that is a good idea. I am
planning on asking if I can record the sessions in some manner so I
can share them with the rest of the UI team and others people who
work on this product.

Thanks everyone for your great feedback. If anyone else has thoughts
or experiences please share.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=38073

2 Feb 2009 - 5:53pm
Jeremy Yuille
2007

at the risk of sounding like I'm throwing tech at the problem ;-)

I totally agree with the note taking warnings above, and have found the livescribe pen to be very helpful for handling audio with the occasional note (synced to the recording)

to reflect and begin writing up, I then usually replay the recording straight from the pen as soon as possible - and add observational notes etc - these get added to the recording at that timestamp.

it's a little hard to describe in text, but very effective at being able to record people unobtrusively while being able to put in markers for things to go back to.

disclaimer: i'm in no way affiliated or anything with the company.. but I do like their product. (although the pen's form is a little annoying to use, the function had me sold :)

2 Feb 2009 - 5:53pm
Jeremy Yuille
2007

at the risk of sounding like I'm throwing tech at the problem ;-)

I totally agree with the note taking warnings above, and have found
the livescribe pen to be very helpful for handling audio with the
occasional note (synced to the recording)

to reflect and begin writing up, I then usually replay the recording
straight from the pen as soon as possible - and add observational
notes etc - these get added to the recording at that timestamp.

it's a little hard to describe in text, but very effective at being
able to record people unobtrusively while being able to put in
markers for things to go back to.

disclaimer: i'm in no way affiliated or anything with the company..
but I do like their product. (although the pen's form is a little
annoying to use, the function had me sold :)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=38073

2 Feb 2009 - 6:34pm
Jared M. Spool
2003

On Feb 2, 2009, at 11:06 AM, Jamie McAtee wrote:

> This is the first time the company has sent anyone to do something
> like this.I am hoping it goes well and we can bring back good
> information so we can do some more of these visits.
>
> Advice is appreciated!

You've gotten a lot of good advice here.

Here's some additional resources we've put together at UIE on this
topic:

Field Research Fundamentals: An Interview with Kate Gomoll
http://is.gd/i9Gy
(Article)

Spoolcast: An Interview with Kate Gomoll on Field Studies
http://is.gd/i9GE
(Podcast)

The Field Study Handbook
http://is.gd/i9GN
(Report available for purchase)

Hope that helps,

Jared

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: jmspool
UIE Web App Summit, 4/19-4/22: http://webappsummit.com

2 Feb 2009 - 6:48pm
Jared M. Spool
2003

On Feb 2, 2009, at 2:12 PM, Katie Albers wrote:

> I feel compelled to reiterate my note-taking plea here: Don't do it!
> Note-taking splits your attention and tends to change the behavior
> of the subject. It's aurally, visually and actively intrusive.
> Note-taking is evil.
> Use a tape recorder or a web cam or a small video camera you can
> mount on the cubicle wall and aim, or similar (depending on what
> behaviors you're particularly studying) but if at all possible,
> don't take notes. Much of what you would take notes on can be
> translated into sound simply by asking questions. Then schedule time
> between sessions when you can jot down your notes and aides de memoir.
>
> I realize that contemporary note-taking is in some cases simply
> unavoidable, but make sure that you really need to do it in this
> case before automatically incorporating it.

I'd argue that note taking is very valuable and, when properly done,
very important to both the observer and the participant.

(As an aside: In this case, the "subject" is not the person you're
observing, it's the software you're studying. The formal name in
phenomenalogical ethnographic studies is "informant", but many of us
just use "participant". Or their first name, which feels less
impersonal.)

Trying to remember everything you see, especially in an 5 to 6 hour
session, also splits your attention. Tape recorders, web cams, and
video cameras change the participants behaviors as much, if not more,
than note taking.

When I'm doing field studies, I prefer to take a small audio recorder.
(I'm in love with the Olympus LS-10, though we often use bulkier
Marantz PMD-660s.) However, I still take my trusted Moleskine large-
size reporter's notebook, for which I take most of my notes. I would
not take notes on a laptop or palm-sized keyboard.

If you've never taken notes in a live interview before, I recommend
you practice it. It's a learned skill and practicing definitely
improves it. Rehearsing your site visit by watching fellow colleagues,
taking notes, then writing up your daily summary -- repeating that
process a couple of times -- is a great way to work the kinks out and
get some practice.

Jared

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: jmspool
UIE Web App Summit, 4/19-4/22: http://webappsummit.com

2 Feb 2009 - 7:00pm
SemanticWill
2007

I love you guys and your polemics - what ever happened to "it
depends..."

Now it's Note taking is evil and

Eye tracking is voodoo.

RED, ACD, GDD, UCD: It all reminds me of the religious arguments
people used to get into between kung-fu, aikido, aikijujitsu,
shotokan, judo, wingchun, as to which "style" was the best/most
effective/most versatile. only neophiles got in to those arguments.
the masters never did :-) New Practitioners used so spend as much time
discussing their art/style and often more time discussing, than doing.
We had an old saying: Shut up and practice. Same with [insert TLA
Silver bullet methodology here] - the process that can be spoken is
not the ultimate process. Those who speak, do not know, and those who
know, do not speak - the master shows by doing, all else is void and
emptiness.
~ will

"Where you innovate, how you innovate,
and what you innovate are design problems"

--------------------------------------------------------------------------------
Will Evans | User Experience Architect
tel: +1.617.281.1281 | will at semanticfoundry.com
http://blog.semanticfoundry.com
aim: semanticwill
gtalk: semanticwill
twitter: semanticwill
--------------------------------------------------------------------------------

On Feb 2, 2009, at 3:48 PM, Jared Spool wrote:

>
> On Feb 2, 2009, at 2:12 PM, Katie Albers wrote:
>
>> I feel compelled to reiterate my note-taking plea here: Don't do it!
>> Note-taking splits your attention and tends to change the behavior
>> of the subject. It's aurally, visually and actively intrusive.
>> Note-taking is evil.
>> Use a tape recorder or a web cam or a small video camera you can
>> mount on the cubicle wall and aim, or similar (depending on what
>> behaviors you're particularly studying) but if at all possible,
>> don't take notes. Much of what you would take notes on can be
>> translated into sound simply by asking questions. Then schedule
>> time between sessions when you can jot down your notes and aides de
>> memoir.
>>
>> I realize that contemporary note-taking is in some cases simply
>> unavoidable, but make sure that you really need to do it in this
>> case before automatically incorporating it.
>
> I'd argue that note taking is very valuable and, when properly done,
> very important to both the observer and the participant.
>
> (As an aside: In this case, the "subject" is not the person you're
> observing, it's the software you're studying. The formal name in
> phenomenalogical ethnographic studies is "informant", but many of us
> just use "participant". Or their first name, which feels less
> impersonal.)
>
> Trying to remember everything you see, especially in an 5 to 6 hour
> session, also splits your attention. Tape recorders, web cams, and
> video cameras change the participants behaviors as much, if not
> more, than note taking.
>
> When I'm doing field studies, I prefer to take a small audio
> recorder. (I'm in love with the Olympus LS-10, though we often use
> bulkier Marantz PMD-660s.) However, I still take my trusted
> Moleskine large-size reporter's notebook, for which I take most of
> my notes. I would not take notes on a laptop or palm-sized keyboard.
>
> If you've never taken notes in a live interview before, I recommend
> you practice it. It's a learned skill and practicing definitely
> improves it. Rehearsing your site visit by watching fellow
> colleagues, taking notes, then writing up your daily summary --
> repeating that process a couple of times -- is a great way to work
> the kinks out and get some practice.
>
> Jared
>
> Jared M. Spool
> User Interface Engineering
> 510 Turnpike St., Suite 102, North Andover, MA 01845
> e: jspool at uie.com p: +1 978 327 5561
> http://uie.com Blog: http://uie.com/brainsparks Twitter: jmspool
> UIE Web App Summit, 4/19-4/22: http://webappsummit.com
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

2 Feb 2009 - 7:29pm
Todd Warfel
2003

I'd completely agree w/Jared's comments below. Good note taking is
very, very important. So important in fact that we developed our own
framework to track everything in. Record anytime you can. I use a
small Leica to photograph any time I can.

On Feb 2, 2009, at 3:48 PM, Jared Spool wrote:

> Trying to remember everything you see, especially in an 5 to 6 hour
> session, also splits your attention. Tape recorders, web cams, and
> video cameras change the participants behaviors as much, if not
> more, than note taking.
>
> When I'm doing field studies, I prefer to take a small audio
> recorder. (I'm in love with the Olympus LS-10, though we often use
> bulkier Marantz PMD-660s.) However, I still take my trusted
> Moleskine large-size reporter's notebook, for which I take most of
> my notes. I would not take notes on a laptop or palm-sized keyboard.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

2 Feb 2009 - 7:40pm
Katie Albers
2005

First of all, let me admit that I never take written notes. Never
have. Not in classes and not in my practice. It isn't that I'm bad at
it (apparently I take excellent notes) but my neurological makeup
makes it a very bad idea; I learn almost entirely through my senses,
especially my ears. My statement on note-taking springs from two causes:

(1) on the few occasions where I've been forced to take notes, I
literally had no idea, later, what had happened. The paper was
undoubtedly much the wiser, I couldn't put anything on it in a context
that made it make sense to me...and I spent years working with
professionals on how to "get over" this "limitation" and my brain
still works that way...change didn't work (and I can't imagine why
anyone thought it would). On the other hand, I have a stunningly good
memory. But I rarely encounter anyone who is willing to consider that
there may be other ways of creating a record. In any case, Jared,
please let me beg off another try at incorporating note-taking again.
It's like asking a deaf person to listen more carefully.

(2) I find that note taking is a default that works against any
further examination of alternative methods of recording one's
reactions, on the spot observations, questions, etc. Often those are
better recorded through another means, but people don't look for any
other means because they'll just take notes. In any case, it's always
another distraction, and how many do you want to tolerate?

As far as terminology, this is one of those cases when I consider the
user's actual behavior to be the subject of the trial, although
obviously their behavior is in reference to the trial of the software.
Therefore, although I call the manipulators in a one-on-one software
test "testers" in order to emphasize their agency; I refer to them in
a contextual enquiry as "subjects" because it's their reality I want
to get into, not the software's. I can readily see arguments in
opposition, but these terms have always worked for me.

I tend to make my statement's definitive and contrary simply because
that tends to encourage active consideration, even if it comes in the
form of serious and substantial disagreement.

And as far as Will's question: yes, it depends...but that means you
have to consider alternatives. I admit I find it somewhat entertaining
that anyone would consider either Jared or me as neophytes -- and I'm
very flattered by the grouping: I have less experience than Jared and
I've been at it 15 years. I think I'll take it as a compliment
though...It's a long time since I've been called neo-anything :)

kt

Katie Albers
Founder & Principal Consultant
FirstThought
User Experience Strategy & Project Management
310 356 7550
katie at firstthought.com

On Feb 2, 2009, at 3:48 PM, Jared Spool wrote:

>
> On Feb 2, 2009, at 2:12 PM, Katie Albers wrote:
>
>> I feel compelled to reiterate my note-taking plea here: Don't do it!
>> Note-taking splits your attention and tends to change the behavior
>> of the subject. It's aurally, visually and actively intrusive.
>> Note-taking is evil.
>> Use a tape recorder or a web cam or a small video camera you can
>> mount on the cubicle wall and aim, or similar (depending on what
>> behaviors you're particularly studying) but if at all possible,
>> don't take notes. Much of what you would take notes on can be
>> translated into sound simply by asking questions. Then schedule
>> time between sessions when you can jot down your notes and aides de
>> memoir.
>>
>> I realize that contemporary note-taking is in some cases simply
>> unavoidable, but make sure that you really need to do it in this
>> case before automatically incorporating it.
>
> I'd argue that note taking is very valuable and, when properly done,
> very important to both the observer and the participant.
>
> (As an aside: In this case, the "subject" is not the person you're
> observing, it's the software you're studying. The formal name in
> phenomenalogical ethnographic studies is "informant", but many of us
> just use "participant". Or their first name, which feels less
> impersonal.)
>
> Trying to remember everything you see, especially in an 5 to 6 hour
> session, also splits your attention. Tape recorders, web cams, and
> video cameras change the participants behaviors as much, if not
> more, than note taking.
>
> When I'm doing field studies, I prefer to take a small audio
> recorder. (I'm in love with the Olympus LS-10, though we often use
> bulkier Marantz PMD-660s.) However, I still take my trusted
> Moleskine large-size reporter's notebook, for which I take most of
> my notes. I would not take notes on a laptop or palm-sized keyboard.
>
> If you've never taken notes in a live interview before, I recommend
> you practice it. It's a learned skill and practicing definitely
> improves it. Rehearsing your site visit by watching fellow
> colleagues, taking notes, then writing up your daily summary --
> repeating that process a couple of times -- is a great way to work
> the kinks out and get some practice.
>
> Jared
>
> Jared M. Spool
> User Interface Engineering
> 510 Turnpike St., Suite 102, North Andover, MA 01845
> e: jspool at uie.com p: +1 978 327 5561
> http://uie.com Blog: http://uie.com/brainsparks Twitter: jmspool
> UIE Web App Summit, 4/19-4/22: http://webappsummit.com

2 Feb 2009 - 8:01pm
Jared M. Spool
2003

On Feb 2, 2009, at 4:40 PM, Katie Albers wrote:

> (1) on the few occasions where I've been forced to take notes, I
> literally had no idea, later, what had happened. The paper was
> undoubtedly much the wiser, I couldn't put anything on it in a
> context that made it make sense to me...and I spent years working
> with professionals on how to "get over" this "limitation" and my
> brain still works that way...change didn't work (and I can't imagine
> why anyone thought it would). On the other hand, I have a stunningly
> good memory. But I rarely encounter anyone who is willing to
> consider that there may be other ways of creating a record. In any
> case, Jared, please let me beg off another try at incorporating note-
> taking again. It's like asking a deaf person to listen more carefully.

Katie, with all due respect, I could care less whether *you* take
notes or not. If note taking doesn't work for you, then don't do it.
(Though, if you're recording the session, don't you take notes off the
recordings? What do you do with them if you're not taking notes off of
them?)

However, there is a long gap between you deciding note taking doesn't
work for *you* and declaring them evil and suggesting that nobody
should ever do it.

Having spent many hours in my career training researchers to take good
notes, I can tell you that (a) good note taking is not a natural
talent, it's a learned skill, (b) most people don't learn in school
how to take good research notes (or even passable ones), and (c) it
takes practice to become proficient at it.

That said, I still prefer it to retrospectively trying to glean useful
information from recordings post facto. It doubles the research time
(at a minimum -- in fact, it could be longer for some types of
studies), it loses context, and it boring as all get out. There is
nothing that can turn a great, exciting, fun research project into
something completely dreadful as transcribing notes off of recordings.

> (2) I find that note taking is a default that works against any
> further examination of alternative methods of recording one's
> reactions, on the spot observations, questions, etc. Often those are
> better recorded through another means, but people don't look for any
> other means because they'll just take notes. In any case, it's
> always another distraction, and how many do you want to tolerate?

Again, this is just a lack of training. Good researchers and
journalists learn to take notes while being introspective and
interrogative. This isn't rocket science, but it isn't natural
either. It takes learning a skill and practicing it.

What you're describing is poor note taking practice. It takes no
skills to do a crappy job at anything you put your mind to. (Damn. I
say this so often that I've decided to call it Spool's First Law of
Competency.)

> As far as terminology, this is one of those cases when I consider
> the user's actual behavior to be the subject of the trial, although
> obviously their behavior is in reference to the trial of the
> software. Therefore, although I call the manipulators in a one-on-
> one software test "testers" in order to emphasize their agency; I
> refer to them in a contextual enquiry as "subjects" because it's
> their reality I want to get into, not the software's. I can readily
> see arguments in opposition, but these terms have always worked for
> me.

For someone who is concerned about the behavior changing effects of
note taking, have you ever noticed how people respond when they
discover you've referred to them as a "subject"? They get a real
glassy-eyed Am-I-A-Rat-In-A-Maze look. It's very dehumanizing, in my
opinion. It's not at all empowering.

In user research, I'd be happy to see the word "subject" just banished
from the vocabulary. ("Testers" is another one I wouldn't mind seeing
go by the way side.) Partner, participant, and collaborators are all
better terms.

That's my $0.02.

Jared

3 Feb 2009 - 12:46pm
Jared M. Spool
2003

On Feb 3, 2009, at 8:32 AM, Jonathan Abbett wrote:

>> What you're describing is poor note taking practice. It takes no
>> skills to
>> do a crappy job at anything you put your mind to. (Damn. I say this
>> so often
>> that I've decided to call it Spool's First Law of Competency.)
>
> Can you recommend any resources on how to take notes well during an
> interview?

That's a really good question. In my experience, it's a core
competency of any journalism or ethnography program.

I've seen good stuff over the years, but am hard pressed to put my
finger on anything current. Years ago, we had Ellen Isaacs (a UX
researcher who was a trained journalist) do a full-day seminar on the
topic at the User Interface Conference, but she's disappeared from the
UX world of late.

Over the years, I've trained our researchers and dozens of clients.
It's not hard, but there are definitely tricks and techniques to
learn. And practice is really what makes it work.

I'll research it some more and see if I can find something current.

Jared

3 Feb 2009 - 1:17pm
jmcatee
2008

I guess my response earlier got lost.

I am planning on taking notes and hoping for the chance to do audio.
I was thinking about video but as others have mentioned that might
make the user feel strange and alter the results. I still have to
talk to our contact at the location about audio recording.

I will have one other person going with me. He is a business analyst
and has asked me to let him know what I need him to do. If we do a
focus group I'll probably have him do notes since in my experience
it has been awkward to facilitate and take notes.

On note taking I usually do general point notes or things that really
stand out and then use the tape for direct quotes or clarifications.
My journalism degree comes in handy with note taking, interviews and
focus groups.

@Jonathan To the rest of your points. I'll be doing a contextual
inquiry since we want to observe how the software is being used vs
the model of how we think it is being used. Plus either interviews or
a focus groups depending on the number of people they are making
available in each user group. No resources for transcription, our UI
team is 3 people. The on site time is a one shot deal in and out the
same day. We have to fly to the location so we have to cram as much
as we can into one session. We could probably do some follow up
e-mails but I doubt we will be able to go back.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=38073

3 Feb 2009 - 2:29pm
Jared M. Spool
2003

On Feb 3, 2009, at 10:17 AM, Jamie McAtee wrote:

> I am planning on taking notes and hoping for the chance to do audio.
> I was thinking about video but as others have mentioned that might
> make the user feel strange and alter the results. I still have to
> talk to our contact at the location about audio recording.

Don't worry too much about your participants feeling strange. Having
someone poking around their office, asking questions, and taking
notice is strange enough. It's not like it'll make too much additional
difference.

That said, video can be (a) against their company policies (you'll
need to check) and (b) more obstrusive. Hand-held cameras are very
noticeable and a constant reminder of recording. A tripod or wall
mounted camera is less so, but the angles are often weird and not
helpful. (Getting the participant *and* the screen into the frame is
damn near impossible.)

Here's what I like to do: I take notes in my Moleskine pad. I record
with discussion with one of our audio recorders. I bring my Canon
SD630 which has both still photo and video capabilities. I take still
photos of artifacts that are interesting (post-its on monitor, desk
set up, manuals available, instructions on the wall) and at least one
short video (1-2 minutes) to help me remember the person when I'm
looking through my notes.

I always ask permission before turning on the audio recorder, taking
pictures, or shooting the video. I always explain what we're going to
do with that information and tell them it's optional but helpful to
me. 99% of the time they say yes.

> I will have one other person going with me. He is a business analyst
> and has asked me to let him know what I need him to do. If we do a
> focus group I'll probably have him do notes since in my experience
> it has been awkward to facilitate and take notes.

Perfect. You should both take turns at facilitating and note taking.

> @Jonathan To the rest of your points. I'll be doing a contextual
> inquiry since we want to observe how the software is being used vs
> the model of how we think it is being used. Plus either interviews or
> a focus groups depending on the number of people they are making
> available in each user group.

I like using these questions as a framework to guide the visit: http://is.gd/ig4b
(Of course, you'd want to tailor the questions to your specific
product and how it's used.)

> No resources for transcription, our UI
> team is 3 people.

If you're not in a rush, Castingwords.com has really high-quality,
budget transcription services: http://castingwords.com/store/

> The on site time is a one shot deal in and out the
> same day. We have to fly to the location so we have to cram as much
> as we can into one session. We could probably do some follow up
> e-mails but I doubt we will be able to go back.

I always find it helpful to ask, just as we're finishing up, if we can
contact them in a couple of weeks for follow-ups. I've found that you
can even use screen sharing technology (like GotoMeeting.com) to get
specific "show me how you do X?" questions answered, quite
effectively. It's especially useful when you've subsequently visited
other sites and have questions about whether the patterns emerging are
more widespread than you first realized.

Hope that helps,

Jared

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: jmspool
UIE Web App Summit, 4/19-4/22: http://webappsummit.com

4 Feb 2009 - 8:35pm
Jared M. Spool
2003

On Feb 4, 2009, at 1:52 AM, Francis Norton wrote:

> I am trying to improve my life-long poor note-taking skills and
> would be be very interested in your recommendations. I've currently
> trying to adopt Cornell Notes (http://en.wikipedia.org/wiki/Cornell_Notes
> ) which is definitely an improvement on what I've been doing so far.

The Cornell system is for studying.

Taking notes during interviewing is a different set of skills, because
(as Katie aptly pointed out) you need to stay present in the moment
and be working on the dialogue with the participant *while* you're
making the notes.

The best resources are going to either be from journalism or
ethnography. Studying note taking techniques won't work as well, nor
will note taking techniques used for interrogation (practiced by
police). They are different animals.

Jared

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: jmspool
UIE Web App Summit, 4/19-4/22: http://webappsummit.com

5 Feb 2009 - 8:22am
Adrian Howard
2005

On 3 Feb 2009, at 19:29, Jared Spool wrote:
[snip]
> Don't worry too much about your participants feeling strange. Having
> someone poking around their office, asking questions, and taking
> notice is strange enough. It's not like it'll make too much
> additional difference.
>
> That said, video can be (a) against their company policies (you'll
> need to check) and (b) more obstrusive. Hand-held cameras are very
> noticeable and a constant reminder of recording.
[snip]

Something I've noticed is that having "normal" cameras (video with
digital camera, mobile phone, flip cam) as opposed to a "professional"
looking device helps with (b).

Ambling around with an expensive "professional" camera must mean this
is... "Important". Recording an interaction snippet with the digital
camera that looks like the one the user has at home - not so much.

Of course folk notice you recording - but seem less self conscious
about it.

Adrian

Syndicate content Get the feed