I am a UI designer for a start up company and I'm conducting a
usability test for our web product for the first time.
I've read a few books including "handbook of usability testing"
and "a practical guide to usability testing", and I've read a ton
of resources online.
The product we are testing is a web widget which is going to be
integrated in other websites (blogs, portals, etc.). We have a
working prototype where we have embedded the widget in a fully
functional mocked-up website.
We are interested in testing two things:
1- Whether or not people will notice our widget at all and will
interact with it
2- Knowing what the widget is, how easily can they use the features
the widget offers
The success of the product largely depends on the first point. The
problem is that its almost impossible to write tasks for that.
So I have designed the test to have 2 parts: in the first part, I
want to give the user about 10 minutes to free-roam the website, and
think-out-loud, to see how long it would take for them to notice the
widget. I also want to observe how long it will take for them to
interact with the feature, and see if they understand (and use) the
features of the widget on their own.
In part 2, I will give them particular tasks related to features of
the widget and see how well they can interact with the UI (if they
haven't figured out by now what the widget is, I will tell them)
The only issue is, they might have already done some of the tasks in
part 2 when they were free-roaming in part 1, and I'm not sure how
to handle that.
So, for the usability experts here, do you think this is a good
approach? I would definitely appreciate your advice and comments.