Heuristic Evaluation on an iPad app

13 Jul 2010 - 1:40pm
3 years ago
4 replies
3614 reads
Thomas Davies
2008

I recently conducted a heuristic evaluation on an iPad game.

I wanted to do some sort of evaluation that wasn't connected to a degree assignment of mine! I emailed it to the company behind the game, but haven't got a reply for two weeks so I thought I would make it public.

The first few pages provide a brief explanation of the evaluation method used, just in case the company wasn't aware of heuristic evaluations. The rest of the document then explains 5 faults I found with the app, and recommendations on how to resolve them.

It would be great if I could get some feedback from you guys, any tips on how to improve future attempts? I know the evaluation is very short and there were still many other issues that I didn't document but I just wanted to get some real experience of doing evaluations outside of university.

This link takes you to the document on Scribd.

Regards,

Thomas

Website: http://thomasdavies.tumblr.com/

Assignments: http://thomasdavies.tumblr.com/assignments

Comments

13 Jul 2010 - 5:05pm
Chaunsee at aol.com
2007

Hello Thomas

What set (or sets) heuristics did you use for your evaluation?

With an iPad game (I am an iPad owner) I could envision mobile heuristics, game heuristics (you might use a heuristic about "challenge) and an adjustable skill level),general heuristics and human factors principles).

Chauncey

Sent from my iPhone

On Jul 13, 2010, at 5:03 PM, Thomas Davies wrote:

> I recently conducted a heuristic evaluation on an iPad game [1]. > > I wanted to do some sort of evaluation that wasn't connected to a degree assignment of mine! I emailed it to the company behind the game, but haven't got a reply for two weeks so I thought I would make it public. > > The first few pages provide a brief explanation of the evaluation method used, just in case the company wasn't aware of heuristic evaluations. The rest of the document then explains 5 faults I found with the app, and recommendations on how to resolve them. > > It would be great if I could get some feedback from you guys, any tips on how to improve future attempts? I know the evaluation is very short and there were still many other issues that I didn't document but I just wanted to get some real experience of doing evaluations outside of university. > > This link takes you to the document on Scribd. [2] > > Regards, > > Thomas > > Website: http://thomasdavies.tumblr.com/ [3] > > Assignments: http://thomasdavies.tumblr.com/assignments [4] > >

13 Jul 2010 - 5:42pm
Thomas Davies
2008

I used Nielsen's 10 Usability Heuristics. They were created in the early 1990s, so clearly not focused on modern tablets such as the iPad, yet I do feel they still apply.

This evaluation stayed away from the playability of the game, concentrating solely on the usability of the actual app.

Game heuristics is something I researched a little for one of my first year assignments, and is an area I am definitely looking into at the moment as well - especially when it comes to mobile devices such as the iPhone and iPad.

Thomas

14 Jul 2010 - 7:05pm
pjohnkeane
2008

Hi Thomas,
Doing a guerilla, real world evaluation is a great way to get some experience - super idea, and you seem to have done a fair job. But first off, I'd say that your evaluation actually highlights some of the general criticisms levelled at heuristic evaluation, namely that it tends to identify superficial issues (such as, in your case, the UI patterns being used for toggle controls). 
Next, I agree that Nielsen's classic set of heuristics are still useful guidelines for general usability (that's kind of the point!) but you also need to be careful about how you interpret the heuristics, especially in the context of the particular application you're reviewing. Take "consistency and standards", for example: a number of your issues identify that the UI patterns used in the app don't conform with the UI patterns used in other iPad (and iOS) apps (i.e. external consistency) ... but in the case of this app, is that more or less important than conforming with the UI patterns used elsewhere in the app (i.e. internal consistency)? Or is the distinctive look and feel of the app important in immersing the user in the game - helping to remind the user that they're playing a game, not tracking their productivity in a to do list app or checking their email or social networking or using a map or etc? In the context of general user experience, are your severity ratings still valid? (N.B. I'm unfamiliar with game heuristics, so I don't know if they cover things like this)
Finally, I'm not sure that you explain yourself and your method well enough in the opening section. I don't recall even seeing the phrase 'heuristic evaluation', so if your client didn't know about it before, they probably still don't! You talk about the heuristics themselves, but not much about the method, how you apply it in practice, its validity, etc. That would be important if you wanted your results - the guts of your report - to be taken seriously.
Actually, that wasn't finally, but this is... It's useful to drop in some positive comments into your report too. After all, the developers you've just anonymously sent this list of bugs to spent time and money and possibly sweat and tears creating this app. Give them some love. For more on this, read this Ask Tog column (from 2001!): http://www.asktog.com/columns/047HowToWriteAReport.html
Hope this is useful.
Kind regards,
John

On 14 July 2010 01:56, Thomas Davies <thomasdavies@me.com> wrote:

I used Nielsen's 10 Usability Heuristics [1]. They were created in the early 1990s, so clearly not focused on modern tablets such as the iPad, yet I do feel they still apply.

This evaluation stayed away from the playability of the game, concentrating solely on the usability of the actual app.

Game heuristics is something I researched a little for one of my first year assignments [2], and is an area I am definitely looking into at the moment as well - especially when it comes to mobile devices such as the iPhone and iPad.

Thomas

((
15 Jul 2010 - 7:11am
Thomas Davies
2008

I really appreciate your advice and will take note of the points you have raised when I conduct another evaluation.

Thank you,

Thomas

Syndicate content Get the feed