Guiding an interface evaluation session

2 Nov 2011 - 7:50am
4 years ago
4 replies
1327 reads

Dear all,

I'm planning a session with some users who are experiencing a problem with an existing complex editing interface (aimed at structured data capture). What I wanted to achieve is to identify where the key problems lie so I can work on a redesign: ideally via one to one sessions but this isn't possible as the team is very distributed so we need to do it as a group in a central location.

I was hoping to concentrate primarily on observation as the users worked their way through a typical task. However, it seems the users would like to use the session primarily as an opportunity to air their frustrations and potentially make suggestions for specific improvements (some of this will be useful I'm sure).

I just wondered if anyone had any experience, tips or pointers for managing a session like this and to keep (potentially strong minded) people "on track", guiding them away from specific solutions to help define the problem. I'm more used to structured usability testing so it's a more unknown area to me I have to admit.





2 Nov 2011 - 9:58am
Linda M

My experience with leading sessions like these is that when a participant wants to stop and gripe about current features, you need to let him or her air these issues and feel completely heard before you'll make any further progress. By probing the complaints, you can sometimes get to relevnat information. Ask them why this is an issue (if it's not already blatantly obvious), ask how often this happens and how this process fits into their routine work,  have them show you the process on the UI.

From there, you can usually get them back on track - but also be aware that how you want to talk about the interface may not align to how users use it or want to use it. there's an art to guiding the conversation to yield useful information while allowing the participant to lead.

same for feature requests - acknowledge the request, but don't take it at face value either. Probe about how the participant uses the system that would cause this feature to be useful.

Good luck!

2 Nov 2011 - 10:45am
Neicole Crepeau

Observation would be more helpful for your purposes, as you know. However, you may still be able to use this session to identify key problems. I had a similar group session a while back. I let people begin by just brainstorming a list of items they'd like to improve, for about fifteen minutes, just to let them get it out of their system. It was a bubble-up approach, not a lot of discussion, just throw things out to put on the list.

Then I had them walk me through the task from beginning to end and we diagrammed the task workflow as we talked. We brought up the app so I could see parts of the process. By walking step-by-step through the task, we were able to identify many of the usability issues, in context. You might find that a similar process will give you some of the information you would have gotten from observation. It also can help you keep people focused on the problems, as you keep reminding them that the goal is to move through the process beginning to end and diagram it, not design new solutions.

Good luck!

2 Nov 2011 - 11:21am

I think it would help if you communicate beforehand that the session will be in *TWO* parts; one where they can express frustration and give ideas and another where you watch them in action. I'd try to arrange a break between the two, so they can mentally separate themselves from their solutions (so that their behavior with interface isn't influenced by the earlier conversation).

Since the users are scattered, is remote observation (screen sharing, et. al) a possibility? 

3 Nov 2011 - 5:03am

Dear all,

Thanks for the replies -- they've been a great help. I think the 2-part session approach is probably going to work the best -- as I agree it's important to allow the users space for making their views known. In practice it may be more difficult to separate out the demonstration of what's causing them specific problems from a more general exploration of the interface. Well, I'll see what comes out of it and gather my impressions in case that's of use to others.

I think the remote observation idea definitely could come in handy--maybe at a later stage when testing redesigns. I'm certainly keen to get away from having group sessions.

Thanks again!

Syndicate content Get the feed