Software for Multivariate Usability Testing
14 Oct 2011 - 12:30pm
8 replies
8111 reads
Powers
2008
Does anyone have any suggestions for software that allows for multivariate testing? I've got a project that have numerous paths a user can take and standard click throughs wont cut it. Based on user selections from initial steps the later steps may change and the software would need to allow for this.
Get the feed
Comments
JP, Would you need something as powerful as SPSS or SAS? They offer basic packages but I've found they can be too much of a good thing for a lot of the UX qual + quant challenges we address. When I was in healthcare-medicine I got my own copy of SPSS and now use perhaps 2% of it, a few times a year. The R-Project software is free, works well . . . http://www.r-project.org/. Would PC Calc be too lightweight? http://www.pcalc.com/english/about.html - Andrew
Hi! What is multivariate testing?
I do eye tracking test. I have mirametrix eye tracker with my own software and tobii eye tracker. But I don't understand what is multivariate....
Andrei,
Multivariate testing is an effectiveness test of more than one design variable, often tested in a live environment. In the case of websites 50% of the visitors will see one version of the page design, the other half will see another version of the same page. By analyzing the visitor clicking or navigation behavior the effectiveness of the design alternatives is determined.
- Yohan
The software developed by my company, Ovo Studios, can probably support this. We allow you to configure any number of experimental treatments and assign a different to different users. You could vary task order, which tasks/scenarios you want the user to work with, which URL you want to present to the user as a stimulus, etc.
Outside of this scripted approach, you can manually run users through tasks in any order you wish, which should support your "user selections from earlier steps influence later steps" use case.
Check out either Ovo Solo or Ovo Logger at www.ovostudios.com.
wager slowly
Five Card Stud at this online casino is just one of the poker games that are here for your gambling and entertainment pleasure but you have to wager slowly.
Depending on the budget, but generally you can do this with some tools like Google Analytics + Google Website Optimizer as well.
As usual, it all depends on your research goals. While pretty powerful, multivariate testing does not typically offer much usability data. That's my personal experience. It tells you which design variable performed best (i.e. sold more, got more registrants, etc.), but it does not tell you the fundamental usability/UX question: WHY. That's why many researchers combine multivariate with usability testing (and web analytics, surveys, ethnographic studies, etc.).
I'd offer you our unmoderated remote user testing methodology and products at UserZoom. You can certainly test 2 design versions over hundreds of users and collect usability metrics, success ratios, click heatmaps, clickstreams, and responses to a follow up (post task) questionnaire. More info at www.userzoom.com
Good luck!
Multi-variate analysis is a multiple ANOVA. Most usability studies do not use sufficient participants and groups to be suitable for ANOVA. In most cases researchers better spend their usability dollars on several studies rather than on studies with 15+ participants per group. One might ask what is the nature of your study that it requires multi-variate analysis? Is this really a relevant technique for the variables of interest?
Geoff Willcher
-----Original Message----- From: Alfonso de la Nuez Sent: Friday, December 09, 2011 2:56 AM To: gwbando@msn.com Subject: Re: [IxDA] Software for Multivariate Usability Testing
As usual, it all depends on your research goals. While pretty powerful, multivariate testing does not typically offer much usability data. That's my personal experience. It tells you which design variable performed best (i.e. sold more, got more registrants, etc.), but it does not tell you the fundamental usability/UX question: WHY. That's why many researchers combine multivariate with usability testing (and web analytics, surveys, ethnographic studies, etc.).
I'd offer you our unmoderated remote user testing methodology and products at UserZoom. You can certainly test 2 design versions over hundreds of users and collect usability metrics, success ratios, click heatmaps, clickstreams, and responses to a follow up (post task) questionnaire. More info at www.userzoom.com
Good luck!