Designing a long list of items that people mustchoose from.

1 Sep 2009 - 10:05am
5 years ago
7 replies
660 reads
Amy Jones
2009

Have you thought of doing a survey and having people self-identify their
occupation? No constraints, just a text field (or two- one for industry
and one for occupation). You could even do it as a fill in the blank,
eg, "I work as a _____ in the ______ industry."
You'd need a fairly large sample size, but it sounds like you may be
working with a well-defined audience.

Once you have a large list of how people think of their industry and
occupation, you can normalize it and that becomes your list.

You're never going to have a truly exhaustive list, though (unless
you're dealing with a very constrained system), so the choice becomes
having people not answer or answer incorrectly, vs having an optional
"not listed" selection and having them write-in their occupation if they
don't see it on the list. The first means you'll get less accurate
data, the second means you'll get more data that will be hard to do
anything with, so there are trade-offs either way.

Good luck!

--Amy Jones

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Paul Trumble
Sent: Tuesday, September 01, 2009 9:34 AM
To: discuss at ixda.org
Subject: [IxDA Discuss] Designing a long list of items that people
mustchoose from.

All:

I'm looking for some advice, examples or even recommendations of who
might
be good at solving this particular problem for us.

In the context of a longish multi-page web form we have a need for the
user
to tell us at a fairly granular level what their occupation is. the
total
length of the list is long, more than 1,000 choices. The accuracy of
the
answer is pretty important to our business as is our desire not to stop
the
users flow through the application because of either the difficulty or
perceived intrusiveness of the question. I should add that most users
don't
view the question as being necessary based on their understanding of
what
they are filling out.

Currently we use an introductory question (labeled currently 'industry',
but
in the past 'line of work' - the better version) to narrow down the list
of
occupations that are presented to the individual. This approach may
well be
the best solution to a difficult problem, but it brings a little
emotional
and cognitive overhead with it. Regularly when we observe users they
will
grumble that we are asking the same question twice, less so with the
'line
of work' label I believe.

Part of the problem with we have with this approach is that the choices
in
the industry list are not very good. The selections for industry are
confusing and users don't always grasp that if their occupation is not
showing up as a choice the solution to the problem might be to choose a
different industry. The actual list has some regulatory constraints and
a
fair amount of internal political baggage.

We are looking for a way to develop a new taxonomy that might make the
process more understandable to the user, while preserving the level of
detail the business requires. Card-sorting doesn't seem like a good
tool
here since ultimately we need to know how individual users categorize
their
own occupation, not how they classify a list of occupations with which
they
have varying degrees of familiarity. Because of the regulatory
constraints
we can't experiment with different versions of the list at any given
time.
We've used surveys to test particular taxonomies in the past. Generally
surveys have proved a good way to rule things out, not develop something
that works well.

What thoughts do you all have on this? I really haven't found any
examples
of folks who do something similar well. I'm interested in advice, or if
you
know someone (or if you are someone) who could do a good job of putting
a
new taxonomy together that would be good too. We may well have to bring
in
the magic aura of expertise that only consultants possess in order to
sell
any changes.

You can email me directly or reply to the list. Thanks in advance.

Paul Trumble

--
Time flies like an arrow. Fruit flies like a banana. - Groucho Marx

http://www.trumbling.com/
http://www.flickr.com/photos/paultrumble/
http://www.twitter.com/trumbling
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

Comments

1 Sep 2009 - 10:31am
bminihan
2007

Go to berecruited.com and register as a high school athlete. While
doing so, you'll be asked to select your high school from a list of
25,000 of them. You'll notice, though, that you have to type its
name, and once done, your dropdown list narrows to show schools in
your state, then just those in your area.

You could do something similar with the industry selection...perhaps
narrow your industries down to just five or so, then let people type
their occupation, displaying possible options via ajax below.

I ran a site similar to berecruited, wherein we *needed* the user's
high school, but had a real hard time getting people to find it among
a huge list. Seems similar to your problem...

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=45250

1 Sep 2009 - 4:32pm
Paul Trumble
2004

Actually we did an a/b test with an auto-complete feature, with disastrous
results. Personally I think the lack of an agreed upon vocabulary killed
it. While you know what your high school is called, there might be 25 ways
to name your job.

Thanks for the ideas. Let me know if you have any others.

Paul

On Tue, Sep 1, 2009 at 4:31 AM, Bryan Minihan <bjminihan at nc.rr.com> wrote:

> Go to berecruited.com and register as a high school athlete. While
> doing so, you'll be asked to select your high school from a list of
> 25,000 of them. You'll notice, though, that you have to type its
> name, and once done, your dropdown list narrows to show schools in
> your state, then just those in your area.
> --
>

1 Sep 2009 - 7:25pm
Sascha Brossmann
2008

On Tue, Sep 1, 2009 at 23:32, Paul Trumble<paultrumble at gmail.com> wrote:
> Actually we did an a/b test with an auto-complete feature, with disastrous
> results. Personally I think the lack of an agreed upon vocabulary killed
> it. While you know what your high school is called, there might be 25 ways
> to name your job.

hm… If the auto-completer just triggers suggestions based on more or
less literally matching terms: yes, certainly. But maybe it would be
possible to have it search within semantic fields instead? And how
about elevating the usefulness of suggestions further with some social
metrics of your users and the like?

Further: maybe drilling down through possibilities could get easier
when employing facets for classification instead of a taxonomy.

> Thanks for the ideas. Let me know if you have any others.

Let's see. But I'd like to ask more questions first:

* What kind of information about the users do you already have at this point?
* Is it absolutely crucial to request this information exactly at this
time? Maybe it could be easier to collect the required information
later.
* How do the users come to register and what happens afterwards?
* Could you maybe outline the whole registration process as currently
planned? As far and detailed as possible?
* On a side note: a large multi-step/-screen form does not seem like
the optimal way to get good follow-through/conversion rates ;-)

To come up with better ideas for this would probably also require some
more knowledge about your prospective users, their goals, business
objectives etc. I don't know how much about those you are willing or
able to share without having NDAs signed, though. (Given the
importance you hinted at, I guess that we might touch some
sensible/strategic business issues here.)

Cheers,

Sascha
--
& : create

https://www.xing.com/profile/Sascha_Brossmann
http://www.linkedin.com/in/brsma
http://twitter.com/brsma

1 Sep 2009 - 7:29pm
Sascha Brossmann
2008

> sensible/ business issues here.)

Oops, false friend… s/sensible/sensitive/
I hope, they're sensible, nonetheless ;-)

2 Sep 2009 - 8:59am
Juan Lanus
2005

Hi Paul,

Firstly, I would look at the existing information (industry, occupation) to
detect strange numbers: too high or too low. These might be indicating a
misplaced occupation or a wrong labeled industry. this could be useful for
fixing classification error or a complete waste of time, but I could not
resist doing the analysis.

Secondly, I would make the selection process as transparent to the user as
possible, so they clearly realize that the "industry" question is a means to
help them by narrowing the choices in the second question.
I would consider adding an "all of them" option in the industry question.
I would make the reaction to an industry selection change highly performant,
sub-second, through the use of some simple javascript, so the user can
experiment without the pain of annoying waits, and without losing the focus.

This is to help a user that does not share the mental model of those who
made the taxonomy. For example, I work for a software factory and might find
things under consulting, software or outsourcing. Because "industry" is not
a parent classification for "occupation", think of the guy who works as a
gardener in a pharmaceutical laboratory. Or an accountant in a casino.

Additionally, depending on the characteristics of the audience, I could set
an accelerator so they can type words or parts of words and get the list
shortened by means of al AJAX script. Like thus:
http://www.tecnosol.com.ar/ui/CG012.htm
Notice that for the accelerator to be useful it must find occurences of the
typed text not only at the beginning of the list items but anywhere,
including inside words. It is very important for this feature to be useful
that it beared zero time learning, and also that it could be bypassed
without noticing it.

If applicable I would also implement a multihierarchy in order to be able to
add an occupation into more then one industry.
--
Juan Lanus

On Tue, Sep 1, 2009 at 12:05, Amy Jones <ajones at convio.com> wrote:

> Have you thought of doing a survey and having people self-identify their
> occupation? No constraints, just a text field (or two- one for industry
> and one for occupation). You could even do it as a fill in the blank,
> eg, "I work as a _____ in the ______ industry."
> You'd need a fairly large sample size, but it sounds like you may be
> working with a well-defined audience.
>
> Once you have a large list of how people think of their industry and
> occupation, you can normalize it and that becomes your list.
>
> You're never going to have a truly exhaustive list, though (unless
> you're dealing with a very constrained system), so the choice becomes
> having people not answer or answer incorrectly, vs having an optional
> "not listed" selection and having them write-in their occupation if they
> don't see it on the list. The first means you'll get less accurate
> data, the second means you'll get more data that will be hard to do
> anything with, so there are trade-offs either way.
>
> Good luck!
>
> --Amy Jones
>
>
> -----Original Message-----
> From: discuss-bounces at lists.interactiondesigners.com
> [mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
> Paul Trumble
> Sent: Tuesday, September 01, 2009 9:34 AM
> To: discuss at ixda.org
> Subject: [IxDA Discuss] Designing a long list of items that people
> mustchoose from.
>
> All:
>
> I'm looking for some advice, examples or even recommendations of who
> might
> be good at solving this particular problem for us.
>
> In the context of a longish multi-page web form we have a need for the
> user
> to tell us at a fairly granular level what their occupation is. the
> total
> length of the list is long, more than 1,000 choices. The accuracy of
> the
> answer is pretty important to our business as is our desire not to stop
> the
> users flow through the application because of either the difficulty or
> perceived intrusiveness of the question. I should add that most users
> don't
> view the question as being necessary based on their understanding of
> what
> they are filling out.
>
> Currently we use an introductory question (labeled currently 'industry',
> but
> in the past 'line of work' - the better version) to narrow down the list
> of
> occupations that are presented to the individual. This approach may
> well be
> the best solution to a difficult problem, but it brings a little
> emotional
> and cognitive overhead with it. Regularly when we observe users they
> will
> grumble that we are asking the same question twice, less so with the
> 'line
> of work' label I believe.
>
> Part of the problem with we have with this approach is that the choices
> in
> the industry list are not very good. The selections for industry are
> confusing and users don't always grasp that if their occupation is not
> showing up as a choice the solution to the problem might be to choose a
> different industry. The actual list has some regulatory constraints and
> a
> fair amount of internal political baggage.
>
> We are looking for a way to develop a new taxonomy that might make the
> process more understandable to the user, while preserving the level of
> detail the business requires. Card-sorting doesn't seem like a good
> tool
> here since ultimately we need to know how individual users categorize
> their
> own occupation, not how they classify a list of occupations with which
> they
> have varying degrees of familiarity. Because of the regulatory
> constraints
> we can't experiment with different versions of the list at any given
> time.
> We've used surveys to test particular taxonomies in the past. Generally
> surveys have proved a good way to rule things out, not develop something
> that works well.
>
> What thoughts do you all have on this? I really haven't found any
> examples
> of folks who do something similar well. I'm interested in advice, or if
> you
> know someone (or if you are someone) who could do a good job of putting
> a
> new taxonomy together that would be good too. We may well have to bring
> in
> the magic aura of expertise that only consultants possess in order to
> sell
> any changes.
>
> You can email me directly or reply to the list. Thanks in advance.
>
> Paul Trumble
>
> --
> Time flies like an arrow. Fruit flies like a banana. - Groucho Marx
>
> http://www.trumbling.com/
> http://www.flickr.com/photos/paultrumble/
> http://www.twitter.com/trumbling
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

2 Sep 2009 - 9:17am
Juan Lanus
2005

Oops! After having sent my posting I noticed the other part of this thread
... and the list narrowing issues.
I have something to add ...
What I did was to make my autocompleter search not only the displayed item
descriptions but also related keywords associated with the "official" names,
that were not displayed. Including colloquial terms that the user never sees
in the page.
Thus, one looking for "accelerator" would also find items containing
"autocomplete".
In your case it might be useful to gather information on what the user
searched for and what occupation finally chose and build an associations map
from eral life data.
--
Juan Lanus

On Wed, Sep 2, 2009 at 10:59, Juan Lanus <juan.lanus at gmail.com> wrote:

> Hi Paul,
>
> Firstly, I would look at the existing information (industry, occupation) to
> detect strange numbers: too high or too low. These might be indicating a
> misplaced occupation or a wrong labeled industry. this could be useful for
> fixing classification error or a complete waste of time, but I could not
> resist doing the analysis.
>
> Secondly, I would make the selection process as transparent to the user as
> possible, so they clearly realize that the "industry" question is a means to
> help them by narrowing the choices in the second question.
> I would consider adding an "all of them" option in the industry question.
> I would make the reaction to an industry selection change highly
> performant, sub-second, through the use of some simple javascript, so the
> user can experiment without the pain of annoying waits, and without losing
> the focus.
> This is to help a user that does not share the mental model of those who
> made the taxonomy. For example, I work for a software factory and might find
> things under consulting, software or outsourcing. Because "industry" is not
> a parent classification for "occupation", think of the guy who works as a
> gardener in a pharmaceutical laboratory. Or an accountant in a casino.
>
> Additionally, depending on the characteristics of the audience, I could set
> an accelerator so they can type words or parts of words and get the list
> shortened by means of al AJAX script. Like thus:
> http://www.tecnosol.com.ar/ui/CG012.htm
> Notice that for the accelerator to be useful it must find occurences of the
> typed text not only at the beginning of the list items but anywhere,
> including inside words. It is very important for this feature to be useful
> that it beared zero time learning, and also that it could be bypassed
> without noticing it.
>
> If applicable I would also implement a multihierarchy in order to be able
> to add an occupation into more then one industry.
> --
> Juan Lanus
>
>
>
> On Tue, Sep 1, 2009 at 12:05, Amy Jones <ajones at convio.com> wrote:
>
>> Have you thought of doing a survey and having people self-identify their
>> occupation? No constraints, just a text field (or two- one for industry
>> and one for occupation). You could even do it as a fill in the blank,
>> eg, "I work as a _____ in the ______ industry."
>> You'd need a fairly large sample size, but it sounds like you may be
>> working with a well-defined audience.
>>
>> Once you have a large list of how people think of their industry and
>> occupation, you can normalize it and that becomes your list.
>>
>> You're never going to have a truly exhaustive list, though (unless
>> you're dealing with a very constrained system), so the choice becomes
>> having people not answer or answer incorrectly, vs having an optional
>> "not listed" selection and having them write-in their occupation if they
>> don't see it on the list. The first means you'll get less accurate
>> data, the second means you'll get more data that will be hard to do
>> anything with, so there are trade-offs either way.
>>
>> Good luck!
>>
>> --Amy Jones
>>
>>
>> -----Original Message-----
>> From: discuss-bounces at lists.interactiondesigners.com
>> [mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
>> Paul Trumble
>> Sent: Tuesday, September 01, 2009 9:34 AM
>> To: discuss at ixda.org
>> Subject: [IxDA Discuss] Designing a long list of items that people
>> mustchoose from.
>>
>> All:
>>
>> I'm looking for some advice, examples or even recommendations of who
>> might
>> be good at solving this particular problem for us.
>>
>> In the context of a longish multi-page web form we have a need for the
>> user
>> to tell us at a fairly granular level what their occupation is. the
>> total
>> length of the list is long, more than 1,000 choices. The accuracy of
>> the
>> answer is pretty important to our business as is our desire not to stop
>> the
>> users flow through the application because of either the difficulty or
>> perceived intrusiveness of the question. I should add that most users
>> don't
>> view the question as being necessary based on their understanding of
>> what
>> they are filling out.
>>
>> Currently we use an introductory question (labeled currently 'industry',
>> but
>> in the past 'line of work' - the better version) to narrow down the list
>> of
>> occupations that are presented to the individual. This approach may
>> well be
>> the best solution to a difficult problem, but it brings a little
>> emotional
>> and cognitive overhead with it. Regularly when we observe users they
>> will
>> grumble that we are asking the same question twice, less so with the
>> 'line
>> of work' label I believe.
>>
>> Part of the problem with we have with this approach is that the choices
>> in
>> the industry list are not very good. The selections for industry are
>> confusing and users don't always grasp that if their occupation is not
>> showing up as a choice the solution to the problem might be to choose a
>> different industry. The actual list has some regulatory constraints and
>> a
>> fair amount of internal political baggage.
>>
>> We are looking for a way to develop a new taxonomy that might make the
>> process more understandable to the user, while preserving the level of
>> detail the business requires. Card-sorting doesn't seem like a good
>> tool
>> here since ultimately we need to know how individual users categorize
>> their
>> own occupation, not how they classify a list of occupations with which
>> they
>> have varying degrees of familiarity. Because of the regulatory
>> constraints
>> we can't experiment with different versions of the list at any given
>> time.
>> We've used surveys to test particular taxonomies in the past. Generally
>> surveys have proved a good way to rule things out, not develop something
>> that works well.
>>
>> What thoughts do you all have on this? I really haven't found any
>> examples
>> of folks who do something similar well. I'm interested in advice, or if
>> you
>> know someone (or if you are someone) who could do a good job of putting
>> a
>> new taxonomy together that would be good too. We may well have to bring
>> in
>> the magic aura of expertise that only consultants possess in order to
>> sell
>> any changes.
>>
>> You can email me directly or reply to the list. Thanks in advance.
>>
>> Paul Trumble
>>
>> --
>> Time flies like an arrow. Fruit flies like a banana. - Groucho Marx
>>
>> http://www.trumbling.com/
>> http://www.flickr.com/photos/paultrumble/
>> http://www.twitter.com/trumbling
>> ________________________________________________________________
>> Welcome to the Interaction Design Association (IxDA)!
>> To post to this list ....... discuss at ixda.org
>> Unsubscribe ................ http://www.ixda.org/unsubscribe
>> List Guidelines ............ http://www.ixda.org/guidelines
>> List Help .................. http://www.ixda.org/help
>> ________________________________________________________________
>> Welcome to the Interaction Design Association (IxDA)!
>> To post to this list ....... discuss at ixda.org
>> Unsubscribe ................ http://www.ixda.org/unsubscribe
>> List Guidelines ............ http://www.ixda.org/guidelines
>> List Help .................. http://www.ixda.org/help
>>
>
>

4 Sep 2009 - 4:17am
Sascha Brossmann
2008

On Wed, Sep 2, 2009 at 16:17, Juan Lanus<juan.lanus at gmail.com> wrote:
> What I did was to make my autocompleter search not only the displayed item
> descriptions but also related keywords associated with the "official" names,
> that were not displayed.

That's what I meant by ‘search within semantic fields’ ;-)

Cheers,

S.

PS: Some threads on this list seem to get split/disconnected, somehow
(including this one, obviously). How come?

Syndicate content Get the feed