Using scales for user input/feedback

23 May 2007 - 12:29pm
6 years ago
3 replies
501 reads
juliecab
2007

Hello,

Our company designs health information that is found on many web
portals, health plans and hospital websites. We are in the process of
redesigning a portion of this work that are a number of interactive
features/topics geared to help consumers make difficult health
decisions. There is a wealth of medical knowledge and input under the
model we are building, however, the nuance of how to gather consumer
ratings for how they are leaning on certain decisions is proving to be a
problem.

The basic pieces of information include statement(s) about a personal
feeling you may have about this medical decision, and then you are asked
to rate how important that particular issues is to you.

For example, in a decision tool about deciding between breast-conserving
surgery or a mastectomy to treat breast cancer, you find the following
statement:

Example:

"I am worried that the cancer will come back in my breast, even though I
know it will not affect my chance of survival."

The standard way in medical circles to discuss gaining feedback on this
is using a likert scale of saying 1= not very important and 5 = very
important.

What we find as challenges/have tried as options are the following:

* Unless the scale for reference on 1 to 5 appears above each
statement (this particular decision tool has 10 unique statements to
evaluate, the likelihood that the individual can get confused on the
scale is high. Additionally, the screen real estate required to
individually label each statement/value pair is pretty high (and screen
real estate is at a premium)

* Using "stars" ala netflix or other consumer rating systems
tested very poorly.

* Using text based drop down ratings (instead of 1 to 5,
providing 5 textual values, i.e. not very important, less important,
neutral, more important, very important) work well, but don't provide
the user as nice of a visual confirmation of how all of these various
statements compare to each other.

* Using a small slider scale control seems potentially cleanest,
but carries the same labeling problem as the 1-5 likert scale...and a
little of the screen real estate issue.

Does anyone have any examples of solving this kind of issue in the past
or can point to good research regarding this? The research we've found
does not clear cut offer that one of the methods above is clearly
testing better than another.

Thanks in advance!

Julie Cabinaw

Healthwise, Inc.

Comments

23 May 2007 - 1:11pm
Dante Murphy
2006

Julie-

Can you paginate the questions using a FLASH application or AJAX? This will prevent you from creating a single, very long form, and will also give you the flexibility to alter the number of choices or their meaning on a question-by-question basis if you need to.

You can always display the tabulated results on a summary page and allow the user to confirm or edit before submitting. This model is well adopted and understood in e-commerce.

Hope that helps,
Dante

_______________________________________________________

Dante Murphy |  Director of Information Architecture

D I G I T A S  H E A L T H  | www.digitashealth.com

-----Original Message-----

Hello,

Our company designs health information that is found on many web
portals, health plans and hospital websites. We are in the process of
redesigning a portion of this work that are a number of interactive
features/topics geared to help consumers make difficult health
decisions. There is a wealth of medical knowledge and input under the
model we are building, however, the nuance of how to gather consumer
ratings for how they are leaning on certain decisions is proving to be a
problem.

< snip >

Example:

"I am worried that the cancer will come back in my breast, even though I
know it will not affect my chance of survival."

The standard way in medical circles to discuss gaining feedback on this
is using a likert scale of saying 1= not very important and 5 = very
important.

What we find as challenges/have tried as options are the following:

* Unless the scale for reference on 1 to 5 appears above each
statement (this particular decision tool has 10 unique statements to
evaluate, the likelihood that the individual can get confused on the
scale is high. Additionally, the screen real estate required to
individually label each statement/value pair is pretty high (and screen
real estate is at a premium)

* Using "stars" ala netflix or other consumer rating systems
tested very poorly.

* Using text based drop down ratings (instead of 1 to 5,
providing 5 textual values, i.e. not very important, less important,
neutral, more important, very important) work well, but don't provide
the user as nice of a visual confirmation of how all of these various
statements compare to each other.

* Using a small slider scale control seems potentially cleanest,
but carries the same labeling problem as the 1-5 likert scale...and a
little of the screen real estate issue.

Does anyone have any examples of solving this kind of issue in the past
or can point to good research regarding this? The research we've found
does not clear cut offer that one of the methods above is clearly
testing better than another.

Thanks in advance!

Julie Cabinaw

23 May 2007 - 1:16pm
Trip O'Dell
2007

Hi Julie - really interesting question.

I'm not sure if there is a good way to rate those statements in the
way you are proposing because you're essentially trying to measure
two different things with the same metric. Specifically you're
asking the patient to measure their level of agreement with a
statement that is measuring two qualities - 1) Inclination between
two procedures and 2) the patient's level of anxiety.

This approach is an artifact of paper-based inventories used off
line. One of the limitation of that approach is that its forced to
use generic statements and measure agreement. Online you can give the
user an opportunity to map their persona on a simple ajax or flash
widget (see attached example)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: breast.jpg
Type: image/jpeg
Size: 13005 bytes
Desc: not available
Url : http://lists.interactiondesigners.com/pipermail/discuss-interactiondesigners.com/attachments/20070523/36e34939/attachment.jpg
-------------- next part --------------

Allowing the user to drag a pointer around the grid and see the
personal statement change gives you more accurate data while enabling
the user to make more accurate personal statements as opposed to
having to agree with someone else's statement. This approach also
gives you the added benefit of being able to measure more per
question and reduce the overall real estate. Its sort of a "where
does it hurt?" approach.

Just an idea.

Good luck
Trip

23 May 2007 - 2:20pm
juliecab
2007

I realized in retrospect that I picked the poorest example question to
share, as we've reviewed all the others and had sent this one back for
rewriting to the content team already. Plus the bigger picture is
necessary to context. My apologies.

However, I am really interested in this concept of a quadrant
approach...the trick gets to multiple statements that have to be
evaluated from a medical perspective here.

i.e. -- the complete list of examples (including the bad first one which
doesn't "balance" well with its opposite partner, but...) is here, which
might provide a better sense of the complete scenario, albeit it not
properly designed for user presentation yet.

http://usability.healthwise.org/decisionpoint/breastcancer/index.htm#per
sonalfeelings

Thanks for the food for thought!

Julie

-----Original Message-----
From: Trip O'Dell [mailto:tripodell at mac.com]
Sent: Wednesday, May 23, 2007 12:17 PM
To: IxDA Discuss
Cc: Julie Cabinaw
Subject: Re: [IxDA Discuss] Using scales for user input/feedback

Hi Julie - really interesting question.

I'm not sure if there is a good way to rate those statements in the

way you are proposing because you're essentially trying to measure

two different things with the same metric. Specifically you're

asking the patient to measure their level of agreement with a

statement that is measuring two qualities - 1) Inclination between

two procedures and 2) the patient's level of anxiety.

This approach is an artifact of paper-based inventories used off

line. One of the limitation of that approach is that its forced to

use generic statements and measure agreement. Online you can give the

user an opportunity to map their persona on a simple ajax or flash

widget (see attached example)

Syndicate content Get the feed