Survey Design | Usability Issues

20 Jan 2010 - 3:08pm
4 years ago
8 replies
2567 reads
Linda Chen
2010

Hi Everyone,

I am in the process of designing a survey for work. The survey consists of
45 questions to be conducted online using a the Checkbox tool (
http://www.checkbox.com/ which seems to be down at the moment).

Because it is a lengthy survey, I am worried about a couple of things:
1. Does the number of question per page matter? If so what is the average
number of questions a participant should answer per page before they lose
patience?
2. Alternatively, what is the maximum number of pages a survey should have?
3. 30 of the 45 questions will be requesting the answers to be on a scale of
1 - 5. Each of these ratings will have an associate description. What is a
good length (e.g. 4 - 5 words) for these rating descriptions?
3. And lastly, are there any good books / online resources that I could use
to answer these questions?

Any response is much appreciated!

Take care,

lin.

Comments

20 Jan 2010 - 3:29pm
Sharon Greenfield5
2008

Yeah, I suggest picking up a book about survey writing for research. I
like Survey Research Methods by Fowler.
Also, simple small thing which can make a big difference: make sure to
the user how many pages/what page they are on.

On Jan 20, 2010, at 12:08 PM, Linda/JingYing Chen wrote:

> Hi Everyone,
>
> I am in the process of designing a survey for work. The survey
> consists of
> 45 questions to be conducted online using a the Checkbox tool (
> http://www.checkbox.com/ which seems to be down at the moment).
>
> Because it is a lengthy survey, I am worried about a couple of things:
> 1. Does the number of question per page matter? If so what is the
> average
> number of questions a participant should answer per page before they
> lose
> patience?
> 2. Alternatively, what is the maximum number of pages a survey
> should have?
> 3. 30 of the 45 questions will be requesting the answers to be on a
> scale of
> 1 - 5. Each of these ratings will have an associate description.
> What is a
> good length (e.g. 4 - 5 words) for these rating descriptions?
> 3. And lastly, are there any good books / online resources that I
> could use
> to answer these questions?
>
> Any response is much appreciated!
>
> Take care,
>
> lin.

20 Jan 2010 - 3:48pm
Linda Chen
2010

Thanks everyone for your responses! A couple of things I left out in
the background:
1. The survey was tried with a set of participants. It received good
responses, but largely due to other reasons.
2. The survey questions have been categorized around 5 dimensions.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48492

20 Jan 2010 - 8:37pm
Shona Dawson
2010

Hi Lin,
Since I work for Checkbox Survey I hope I can help you. Firstly, we
have a wealth of resources on our website - blog posts, webinars, and
training videos that should provide you with lots of tips and tricks
on how to conduct your survey. This blog post in particular should
help you out:
http://www.checkbox.com/blog/2009/07/12-tips-on-how-to-conduct-your-first-online-survey/
1. You should limit the number of questions on a page to around 6.
Respondents may lose patience if they have to scroll through lengthy
pages. Try and keep questions grouped per page too, so that each
page begins a new topic area.
2. A survey can be as long as you like as long as you keep it
relevant. If you have a lengthy survey, make sure you alert your
respondents in an introduction as to how long it should take to
complete. This will allow them to decide whether they will take the
survey right away, or put it aside for when they have more time.
Using a progress bar will also help guide your respondents as to how
far through the survey they are.
3. For your rating questions - keep the scale as simple as possible
in terms of wording (2-3 words if you can). If this is an issue,
then use a number scale, 1-5 and then at the top of each page insert
a 'message' describing what the numeric ratings mean.
You could also use a matrix item to condense the space and number of
questions on a page if all your rating scales are the same. For
example, you say the questions have been categorized around 5
dimensions - maybe you could have 5 matrix items (one per page) to
clearly set this out.

I hope this has been of some help and I recommend checking out our
blog posts for more Checkbox tips and tricks.

Good Luck

Shona

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48492

20 Jan 2010 - 3:28pm
Anonymous

Lin,

I'm a bit wary to specifically answer your individual questions, as
I am unaware of the context and flow of the overall survey, but I
highly recommend purchasing Luke Wroblewski's "Web Form Design -
Filling in the Blanks"

http://rosenfeldmedia.com/books/webforms/

It's an excellent resource for solving these kinds of questions,
regardless of things like form content, length, etc.

Best of luck.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48492

21 Jan 2010 - 11:36am
AlokJain
2006

Linda,

Here are some thoughts:

>> Because it is a lengthy survey, I am worried about a couple of
>> things:
>> 1. Does the number of question per page matter? If so what is the
>> average
>> number of questions a participant should answer per page before
>> they lose
>> patience?

The number of Qns per page do matter. keep in mind your respondent
profile - if they are busy people a shorter version of the survey
would work better. To increase completion rate tell your respondents
upfront the time it is going to take them to take the survey. the
number of questions per page should be guided by the fact that the
questions have to be grouped according to their context - move them to
the next page if the context/ topic changes. Avoid situations which
requires the respondents to do a lot of scrolling.

>> 2. Alternatively, what is the maximum number of pages a survey
>> should have?

This is tied to 1 above.

>> 3. 30 of the 45 questions will be requesting the answers to be on a
>> scale of
>> 1 - 5. Each of these ratings will have an associate description.
>> What is a
>> good length (e.g. 4 - 5 words) for these rating descriptions?

Why dont you try using the matrix type question - you can have three
such questions (one on each page) each having ten attributes. You do
not have to limit yourself to 4 - 5 words for the description and you
may use a complete sentence of 10 - 20 words.

Regards
Alok Jain
http://insightify.com

22 Jan 2010 - 11:47am
Donna Colarossi
2009

I agree the length is not that important as long as users know up
front how much time they need to commit.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48492

24 Jan 2010 - 10:31pm
Chauncey Wilson
2007

The Fowler book is a very good book. I like the very practical book:

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet,
mail, and mixed-mode surveys: The tailored design method (Third
Edition). New York, NY: Wiley. This is perhaps the best and
comprehensive book on survey design. This is the latest book from Don
Dillman, a guru of questionnaire and survey design. Dillman's book is
full of practical advice backed up by decades of research.

The comments by others are very good. The length of the survey will
depend on how usable the survey is, the motivation of your
respondents, the relevance of the questions, and the trust that people
have in your survey. The Dilllan book has many tips on how to ensure
trust and indicate how the survey will benefit the respondents.

Thanks,
Chauncey

On Wed, Jan 20, 2010 at 3:29 PM, live <human.factor.one at gmail.com> wrote:
> Yeah, I suggest picking up a book about survey writing for research. I like
> Survey Research Methods by Fowler.
> Also, simple small thing which can make a big difference: make sure to the
> user how many pages/what page they are on.
>
> On Jan 20, 2010, at 12:08 PM, Linda/JingYing Chen wrote:
>
>> Hi Everyone,
>>
>> I am in the process of designing a survey for work. The survey consists of
>> 45 questions to be conducted online using a the Checkbox tool (
>> http://www.checkbox.com/ which seems to be down at the moment).
>>
>> Because it is a lengthy survey, I am worried about a couple of things:
>> 1. Does the number of question per page matter? If so what is the average
>> number of questions a participant should answer per page before they lose
>> patience?
>> 2. Alternatively, what is the maximum number of pages a survey should
>> have?
>> 3. 30 of the 45 questions will be requesting the answers to be on a scale
>> of
>> 1 - 5. Each of these ratings will have an associate description. What is a
>> good length (e.g. 4 - 5 words) for these rating descriptions?
>> 3. And lastly, are there any good books / online resources that I could
>> use
>> to answer these questions?
>>
>> Any response is much appreciated!
>>
>> Take care,
>>
>> lin.
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

1 Feb 2010 - 8:56am
Carolyn Craven
2010

Hi Lin

I work with Shona at Checkbox Survey and wanted to add one more
resource to help you with your surveys. We have recently published a
free e-book, "Survey Creation Made Easy by Checkbox Survey", which
is a compilation of our staff and customers' favorite advice, tips,
and tricks. You can receive your free copy by filling out a contact
form at http://customer.checkboxonline.com/ebook1.aspx. Please also
feel free to inquire about our professional services at
www.checkbox.com or ask us for advice on Twitter @checkbox_survey.
Best of Luck!

Carolyn Craven, COO Checkbox Survey Solutions

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48492

Syndicate content Get the feed