I'm conducting a usability study of a U.S army corps of engineers website.
The most important component is a task analysis. In this case we sit along
with the subject and have a dialogue with them as they go along with the
task (talk to them about what they are thinking, try getting their thoughts
out), and give them hints if they are stuck etc, until they get to the
result (all info is captured in the coding sheets).
Another research I recently noticed used a diametrically different approach.
They just give the task to the subjects and left them on their own. In this
case there would be success/failure to get to the result etc. No hints were
given and the users could give up when they wanted.
Which of these approaches would you prefer? In the first case, the data
collected would be really rich and deep...but it isn't natural becauase of
all the hints and dialogues. Any other de/merits of either methodology?