Value of utesting a client driven marketing site.

15 Sep 2006 - 1:24pm
7 years ago
9 replies
505 reads
achong
2006

I'm currently working at an agency and am trying to figure out the
value of utesting in this environment. Our current process does not
include utesting of any sort. It's mostly client-driven design through
back and forth conversations.

For the client I'm working with, the site basically markets a product.
The creative thinking revolves around the marketing objective of
making their products appear to be top quality. A lot of times the
results are mini applications, games, or experience pieces that
augment the knowledge that a general user would have.

In this circumstance, is it worthwhile to conduct usability testing?
There is no real data to tie the website to direct sales as the
product's main sale channel exists in brick and mortar stores. The
objective is general awareness for feature sets of the product and
it's superiority over previous iterations. We could potentially test
how much information is retained or if users can access that
information easily but is their value in doing this?

I guess I'm struggling with finding a concrete process to attach my
work to. Currently I feel that my solutions are operating in a void.
They may be valid or witty solutions but I have an unsettling feeling
of not knowing if they are having the intended impact. Maybe it's
something that I need to let go of and embrace a different way of
doing things for the type of work I'm doing.

Is there a valid argument I can make for utesting?

--
Adrian Chong
www.adrianchong.com/blog

Comments

15 Sep 2006 - 2:27pm
achong
2006

Sorry should have labelled this appropriately.

On 9/15/06, Adrian Chong <chongadrian at gmail.com> wrote:
> I'm currently working at an agency and am trying to figure out the
> value of utesting in this environment. Our current process does not
> include utesting of any sort. It's mostly client-driven design through
> back and forth conversations.
>
> For the client I'm working with, the site basically markets a product.
> The creative thinking revolves around the marketing objective of
> making their products appear to be top quality. A lot of times the
> results are mini applications, games, or experience pieces that
> augment the knowledge that a general user would have.
>
> In this circumstance, is it worthwhile to conduct usability testing?
> There is no real data to tie the website to direct sales as the
> product's main sale channel exists in brick and mortar stores. The
> objective is general awareness for feature sets of the product and
> it's superiority over previous iterations. We could potentially test
> how much information is retained or if users can access that
> information easily but is their value in doing this?
>
> I guess I'm struggling with finding a concrete process to attach my
> work to. Currently I feel that my solutions are operating in a void.
> They may be valid or witty solutions but I have an unsettling feeling
> of not knowing if they are having the intended impact. Maybe it's
> something that I need to let go of and embrace a different way of
> doing things for the type of work I'm doing.
>
> Is there a valid argument I can make for utesting?
>
>
> --
> Adrian Chong
> www.adrianchong.com/blog
>

--
Adrian Chong
www.adrianchong.com/blog

17 Sep 2006 - 4:34pm
Jared M. Spool
2003

At 02:24 PM 9/15/2006, Adrian Chong wrote:
>In this circumstance, is it worthwhile to conduct usability testing?
>There is no real data to tie the website to direct sales as the
>product's main sale channel exists in brick and mortar stores. The
>objective is general awareness for feature sets of the product and
>it's superiority over previous iterations. We could potentially test
>how much information is retained or if users can access that
>information easily but is their value in doing this?
Adrian,

Usability testing is a tool. It tells you information about how people
interact with the design. If there is information you'd like to learn about
how people interact with your site, then it's a valuable tool.

That said, I think the bigger concern is that you don't have any measures
of success. If the site went away tomorrow, would anybody in your
organization notice? Would it be reflected anywhere in the bottom line?

Before I'd spend a lot of time designing usability tests, I'd think about
how you'll know if the site is actually helping the organization or not. My
guess is, once you have an understanding of the site's value, you'll have
no trouble seeing how testing could help you refine and improve that value.

Jared

Jared M. Spool, Founding Principal, User Interface Engineering
510 Turnpike Street, Suite 102, North Andover, MA 01845
978 327-5561 jspool at uie.com http://www.uie.com
Blog: http://www.uie.com/brainsparks

18 Sep 2006 - 11:11am
achong
2006

Thanks for the response. That's probably the more accurate issue,
there is no measure of sucess for the site. If the stakeholders thinks
it's "cool" then it's a success for a while until they realize they
spent 500k on something that just looks neat but serves an ambiguous
purpose. Isn't that the nature of marketing though? Unfortunately, I
don't think I can really change a culture as one man :/

I guess I mentioned Usability testing as an issue because I use to do
a wide range of UX duties and now I'm responsible for a very narrow
process that seems to be missing the greater context that usually
informs my work.

On 9/17/06, Jared M. Spool <jspool at uie.com> wrote:
> At 02:24 PM 9/15/2006, Adrian Chong wrote:
> >In this circumstance, is it worthwhile to conduct usability testing?
> >There is no real data to tie the website to direct sales as the
> >product's main sale channel exists in brick and mortar stores. The
> >objective is general awareness for feature sets of the product and
> >it's superiority over previous iterations. We could potentially test
> >how much information is retained or if users can access that
> >information easily but is their value in doing this?
> Adrian,
>
> Usability testing is a tool. It tells you information about how people
> interact with the design. If there is information you'd like to learn about
> how people interact with your site, then it's a valuable tool.
>
> That said, I think the bigger concern is that you don't have any measures
> of success. If the site went away tomorrow, would anybody in your
> organization notice? Would it be reflected anywhere in the bottom line?
>
> Before I'd spend a lot of time designing usability tests, I'd think about
> how you'll know if the site is actually helping the organization or not. My
> guess is, once you have an understanding of the site's value, you'll have
> no trouble seeing how testing could help you refine and improve that value.
>
> Jared
>
>
> Jared M. Spool, Founding Principal, User Interface Engineering
> 510 Turnpike Street, Suite 102, North Andover, MA 01845
> 978 327-5561 jspool at uie.com http://www.uie.com
> Blog: http://www.uie.com/brainsparks
>
>
>

--
Adrian Chong
www.adrianchong.com/blog

18 Sep 2006 - 2:38pm
Jared M. Spool
2003

At 12:11 PM 9/18/2006, Adrian Chong wrote:
>Thanks for the response. That's probably the more accurate issue,
>there is no measure of sucess for the site. If the stakeholders thinks
>it's "cool" then it's a success for a while until they realize they
>spent 500k on something that just looks neat but serves an ambiguous
>purpose. Isn't that the nature of marketing though? Unfortunately, I
>don't think I can really change a culture as one man :/

<Flame>
I have yet to meet a designer who doesn't get inflamed when a non-designer
suggests that solving some complex problem is "just a simple matter of design."

I think it's an unfair statement, even if made in jest, to suggest the
"nature of marketing" entails spending money without caring of
justification. I can assure you that marketers (a generic term, no less
non-descript than "designer") who do their job well are keenly aware of the
the results they are trying to attain. (And before you make some crass
remark about how "those are rare", I'm sure there are many who would argue
that smart, talented designers are equally as "rare.")

Not to pick on Adrian, I don't think it benefits us, as practitioners in a
burgeoning discipline to be so quick to dismiss the talents of other
disciplines, especially those that have been around a lot longer than ours has.
</Flame>

In terms of change, I don't know enough about your situation to know
whether you can affect change or not. However, I can tell you that many
before you have, in situations that sound much more "dire" than what you
might be facing.

As I've said before on this list, senior executives care about 5 things.
Translate what you're trying to do into one of those 5 things and you'll
have no trouble getting their attention.

I learned a long time ago that senior executives don't care about usability
testing. But they do care about:

1) Increasing Revenue
2) Reducing Expenses
3) Increasing Sales from New Customers (aka increasing marketshare)
4) Increasing Sales from Existing Customers (aka leveraging customer base)
5) Increasing Shareholder Value

Ask yourself how conducting a usability test (or a field study, card sort,
prototype, or any other UX activity,) can achieve one of those 5 things and
you'll have your answer as to how and when to use the test.

Hope this helps.

(Sorry for the flame, but you hit one of my pet peeves - showing disrespect
for the other hard-working members on *our* team.)

Jared

Jared M. Spool, Founding Principal, User Interface Engineering
510 Turnpike Street, Suite 102, North Andover, MA 01845
978 327-5561 jspool at uie.com http://www.uie.com
Blog: http://www.uie.com/brainsparks

18 Sep 2006 - 5:17pm
achong
2006

Not to worry. It's my personal hate on for specific type of
marketer/marketing which I should have clarified. I am aware of the 5
points you listed but I think internally it's more of the "if it ain't
broke don't fix it" mentality. Why should they take on more risk by
adopting and integrating new tools into their process if they are
making money with their previous process? There is the potential for
more but why risk it... that's what i feel i'm up against. It kind of
leaves few options out there.

On 9/18/06, Jared M. Spool <jspool at uie.com> wrote:
> At 12:11 PM 9/18/2006, Adrian Chong wrote:
> >Thanks for the response. That's probably the more accurate issue,
> >there is no measure of sucess for the site. If the stakeholders thinks
> >it's "cool" then it's a success for a while until they realize they
> >spent 500k on something that just looks neat but serves an ambiguous
> >purpose. Isn't that the nature of marketing though? Unfortunately, I
> >don't think I can really change a culture as one man :/
>
> <Flame>
> I have yet to meet a designer who doesn't get inflamed when a non-designer
> suggests that solving some complex problem is "just a simple matter of design."
>
> I think it's an unfair statement, even if made in jest, to suggest the
> "nature of marketing" entails spending money without caring of
> justification. I can assure you that marketers (a generic term, no less
> non-descript than "designer") who do their job well are keenly aware of the
> the results they are trying to attain. (And before you make some crass
> remark about how "those are rare", I'm sure there are many who would argue
> that smart, talented designers are equally as "rare.")
>
> Not to pick on Adrian, I don't think it benefits us, as practitioners in a
> burgeoning discipline to be so quick to dismiss the talents of other
> disciplines, especially those that have been around a lot longer than ours has.
> </Flame>
>
> In terms of change, I don't know enough about your situation to know
> whether you can affect change or not. However, I can tell you that many
> before you have, in situations that sound much more "dire" than what you
> might be facing.
>
> As I've said before on this list, senior executives care about 5 things.
> Translate what you're trying to do into one of those 5 things and you'll
> have no trouble getting their attention.
>
> I learned a long time ago that senior executives don't care about usability
> testing. But they do care about:
>
> 1) Increasing Revenue
> 2) Reducing Expenses
> 3) Increasing Sales from New Customers (aka increasing marketshare)
> 4) Increasing Sales from Existing Customers (aka leveraging customer base)
> 5) Increasing Shareholder Value
>
> Ask yourself how conducting a usability test (or a field study, card sort,
> prototype, or any other UX activity,) can achieve one of those 5 things and
> you'll have your answer as to how and when to use the test.
>
> Hope this helps.
>
> (Sorry for the flame, but you hit one of my pet peeves - showing disrespect
> for the other hard-working members on *our* team.)
>
> Jared
>
>
> Jared M. Spool, Founding Principal, User Interface Engineering
> 510 Turnpike Street, Suite 102, North Andover, MA 01845
> 978 327-5561 jspool at uie.com http://www.uie.com
> Blog: http://www.uie.com/brainsparks
>
>
>

--
Adrian Chong
www.adrianchong.com/blog

18 Sep 2006 - 5:33pm
Mark Schraad
2006

The two things, to which you elude, are most devastating to a
company... ...arrogance and complacency. These will most often lead
to not only unrest amongst customer base, but amongst employees. That
is not to say that all companies need be "cutting edge" or "breaking
new ground". If that is what you are after as an employee, and the
current customer base is not - you have a problem. Do you have the
ability to be a change agent. Power and authority are two totally
different things. You do not have to be granted power. And being
granted authority will not make you capable of change. These are
cultural issues and of critical importance. Culture beats strategy 9
times out of 10. Risk and change are a lot to ask of most people. But
a lot of it depends upon how you ask.

Mark

On Sep 18, 2006, at 6:17 PM, Adrian Chong wrote:

> [Please voluntarily trim replies to include only relevant quoted
> material.]
>
> Not to worry. It's my personal hate on for specific type of
> marketer/marketing which I should have clarified. I am aware of the 5
> points you listed but I think internally it's more of the "if it ain't
> broke don't fix it" mentality. Why should they take on more risk by
> adopting and integrating new tools into their process if they are
> making money with their previous process? There is the potential for
> more but why risk it... that's what i feel i'm up against. It kind of
> leaves few options out there.
>
> On 9/18/06, Jared M. Spool <jspool at uie.com> wrote:
>> At 12:11 PM 9/18/2006, Adrian Chong wrote:
>>> Thanks for the response. That's probably the more accurate issue,
>>> there is no measure of sucess for the site. If the stakeholders
>>> thinks
>>> it's "cool" then it's a success for a while until they realize they
>>> spent 500k on something that just looks neat but serves an ambiguous
>>> purpose. Isn't that the nature of marketing though? Unfortunately, I
>>> don't think I can really change a culture as one man :/
>>
>> <Flame>
>> I have yet to meet a designer who doesn't get inflamed when a non-
>> designer
>> suggests that solving some complex problem is "just a simple
>> matter of design."
>>
>> I think it's an unfair statement, even if made in jest, to suggest
>> the
>> "nature of marketing" entails spending money without caring of
>> justification. I can assure you that marketers (a generic term, no
>> less
>> non-descript than "designer") who do their job well are keenly
>> aware of the
>> the results they are trying to attain. (And before you make some
>> crass
>> remark about how "those are rare", I'm sure there are many who
>> would argue
>> that smart, talented designers are equally as "rare.")
>>
>> Not to pick on Adrian, I don't think it benefits us, as
>> practitioners in a
>> burgeoning discipline to be so quick to dismiss the talents of other
>> disciplines, especially those that have been around a lot longer
>> than ours has.
>> </Flame>
>>
>> In terms of change, I don't know enough about your situation to know
>> whether you can affect change or not. However, I can tell you that
>> many
>> before you have, in situations that sound much more "dire" than
>> what you
>> might be facing.
>>
>> As I've said before on this list, senior executives care about 5
>> things.
>> Translate what you're trying to do into one of those 5 things and
>> you'll
>> have no trouble getting their attention.
>>
>> I learned a long time ago that senior executives don't care about
>> usability
>> testing. But they do care about:
>>
>> 1) Increasing Revenue
>> 2) Reducing Expenses
>> 3) Increasing Sales from New Customers (aka increasing marketshare)
>> 4) Increasing Sales from Existing Customers (aka leveraging
>> customer base)
>> 5) Increasing Shareholder Value
>>
>> Ask yourself how conducting a usability test (or a field study,
>> card sort,
>> prototype, or any other UX activity,) can achieve one of those 5
>> things and
>> you'll have your answer as to how and when to use the test.
>>
>> Hope this helps.
>>
>> (Sorry for the flame, but you hit one of my pet peeves - showing
>> disrespect
>> for the other hard-working members on *our* team.)
>>
>> Jared
>>
>>
>> Jared M. Spool, Founding Principal, User Interface Engineering
>> 510 Turnpike Street, Suite 102, North Andover, MA 01845
>> 978 327-5561 jspool at uie.com http://www.uie.com
>> Blog: http://www.uie.com/brainsparks
>>
>>
>>
>
>
> --
> Adrian Chong
> www.adrianchong.com/blog
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org

18 Sep 2006 - 5:42pm
Dave Malouf
2005

Adrian, I'm totally there with you on your problem, but the answer to your question of "why risk it?" needs to fall into the 5 itemms that Jared listed, no?

I feel in my current situation I was in a really similar boat. What I had to do to get out of it was 2 things:

1. Do research - I had to get numbers that defended my point of view.

2. Build it - pictures and prototypes help decision makers envision value for themselves and others (I.e. Their customers whom they want to validate your ideas with, but can only do with examples.)

3. Find examples of success using similar solutions that your stakeholders can respect (that WAs #3. I know. )

There also comes a time to admit that maybe you just aren't right and that could sadly mean that the company really doesn't need you (right now).

Dave

Sent from my Verizon Wireless BlackBerry

19 Sep 2006 - 11:48am
achong
2006

Mark,

I think you hit it spot on. "You do not have to be granted power. And
being granted authority will not make you capable of change." To be
honest I don't think I'm the right person to lead the change although
I know the change is necessary. It's definitely a cultural issue and
there isn't buy in at the most important levels of the organization.
Because nobody has concretely laid out the value of UCD to the
company, in turn the company cannot communicate UCD's value to our
clients.

--
Adrian Chong
www.adrianchong.com/blog

On 9/18/06, Mark Schraad <mschraad at mac.com> wrote:
> The two things, to which you elude, are most devastating to a
> company... ...arrogance and complacency. These will most often lead
> to not only unrest amongst customer base, but amongst employees. That
> is not to say that all companies need be "cutting edge" or "breaking
> new ground". If that is what you are after as an employee, and the
> current customer base is not - you have a problem. Do you have the
> ability to be a change agent. Power and authority are two totally
> different things. You do not have to be granted power. And being
> granted authority will not make you capable of change. These are
> cultural issues and of critical importance. Culture beats strategy 9
> times out of 10. Risk and change are a lot to ask of most people. But
> a lot of it depends upon how you ask.
>
> Mark
>
>
> On Sep 18, 2006, at 6:17 PM, Adrian Chong wrote:
>
> > [Please voluntarily trim replies to include only relevant quoted
> > material.]
> >
> > Not to worry. It's my personal hate on for specific type of
> > marketer/marketing which I should have clarified. I am aware of the 5
> > points you listed but I think internally it's more of the "if it ain't
> > broke don't fix it" mentality. Why should they take on more risk by
> > adopting and integrating new tools into their process if they are
> > making money with their previous process? There is the potential for
> > more but why risk it... that's what i feel i'm up against. It kind of
> > leaves few options out there.
> >
> > On 9/18/06, Jared M. Spool <jspool at uie.com> wrote:
> >> At 12:11 PM 9/18/2006, Adrian Chong wrote:
> >>> Thanks for the response. That's probably the more accurate issue,
> >>> there is no measure of sucess for the site. If the stakeholders
> >>> thinks
> >>> it's "cool" then it's a success for a while until they realize they
> >>> spent 500k on something that just looks neat but serves an ambiguous
> >>> purpose. Isn't that the nature of marketing though? Unfortunately, I
> >>> don't think I can really change a culture as one man :/
> >>
> >> <Flame>
> >> I have yet to meet a designer who doesn't get inflamed when a non-
> >> designer
> >> suggests that solving some complex problem is "just a simple
> >> matter of design."
> >>
> >> I think it's an unfair statement, even if made in jest, to suggest
> >> the
> >> "nature of marketing" entails spending money without caring of
> >> justification. I can assure you that marketers (a generic term, no
> >> less
> >> non-descript than "designer") who do their job well are keenly
> >> aware of the
> >> the results they are trying to attain. (And before you make some
> >> crass
> >> remark about how "those are rare", I'm sure there are many who
> >> would argue
> >> that smart, talented designers are equally as "rare.")
> >>
> >> Not to pick on Adrian, I don't think it benefits us, as
> >> practitioners in a
> >> burgeoning discipline to be so quick to dismiss the talents of other
> >> disciplines, especially those that have been around a lot longer
> >> than ours has.
> >> </Flame>
> >>
> >> In terms of change, I don't know enough about your situation to know
> >> whether you can affect change or not. However, I can tell you that
> >> many
> >> before you have, in situations that sound much more "dire" than
> >> what you
> >> might be facing.
> >>
> >> As I've said before on this list, senior executives care about 5
> >> things.
> >> Translate what you're trying to do into one of those 5 things and
> >> you'll
> >> have no trouble getting their attention.
> >>
> >> I learned a long time ago that senior executives don't care about
> >> usability
> >> testing. But they do care about:
> >>
> >> 1) Increasing Revenue
> >> 2) Reducing Expenses
> >> 3) Increasing Sales from New Customers (aka increasing marketshare)
> >> 4) Increasing Sales from Existing Customers (aka leveraging
> >> customer base)
> >> 5) Increasing Shareholder Value
> >>
> >> Ask yourself how conducting a usability test (or a field study,
> >> card sort,
> >> prototype, or any other UX activity,) can achieve one of those 5
> >> things and
> >> you'll have your answer as to how and when to use the test.
> >>
> >> Hope this helps.
> >>
> >> (Sorry for the flame, but you hit one of my pet peeves - showing
> >> disrespect
> >> for the other hard-working members on *our* team.)
> >>
> >> Jared
> >>
> >>
> >> Jared M. Spool, Founding Principal, User Interface Engineering
> >> 510 Turnpike Street, Suite 102, North Andover, MA 01845
> >> 978 327-5561 jspool at uie.com http://www.uie.com
> >> Blog: http://www.uie.com/brainsparks
> >>
> >>
> >>
> >
> >
> > --
> > Adrian Chong
> > www.adrianchong.com/blog
> > ________________________________________________________________
> > Welcome to the Interaction Design Association (IxDA)!
> > To post to this list ....... discuss at ixda.org
> > List Guidelines ............ http://listguide.ixda.org/
> > List Help .................. http://listhelp.ixda.org/
> > (Un)Subscription Options ... http://subscription-options.ixda.org/
> > Announcements List ......... http://subscribe-announce.ixda.org/
> > Questions .................. lists at ixda.org
> > Home ....................... http://ixda.org/
> > Resource Library ........... http://resources.ixda.org
>
>

19 Sep 2006 - 11:52am
achong
2006

Dave,

Thanks for the support. For myself, I think I'm at a point of whether
there is value for myself to invest so much energy for what may end up
being a lost cause. It seems where ever I go I am an agitator of the
current culture. It leads me to believe I may be leading myself into
the wrong cultural fits and need to reinvestigate a role that would
suit me.

--
Adrian Chong
www.adrianchong.com/blog

On 9/18/06, Dave at ixda.org <Dave at ixda.org> wrote:
> Adrian, I'm totally there with you on your problem, but the answer to your question of "why risk it?" needs to fall into the 5 itemms that Jared listed, no?
>
> I feel in my current situation I was in a really similar boat. What I had to do to get out of it was 2 things:
>
> 1. Do research - I had to get numbers that defended my point of view.
>
> 2. Build it - pictures and prototypes help decision makers envision value for themselves and others (I.e. Their customers whom they want to validate your ideas with, but can only do with examples.)
>
> 3. Find examples of success using similar solutions that your stakeholders can respect (that WAs #3. I know. )
>
> There also comes a time to admit that maybe you just aren't right and that could sadly mean that the company really doesn't need you (right now).
>
> Dave
>
>
> Sent from my Verizon Wireless BlackBerry
>

Syndicate content Get the feed