User reaction to a lack of ratings and reviews

14 Jul 2009 - 11:32am
5 years ago
10 replies
878 reads
Michael Etgen
2008

Does anyone know of any academic or industry research that addresses
how users react-to/perceive products on a site when there is a
function displayed for ratings/reviews, but none (or maybe just 1)
have been completed?

I've looked through a number of papers/articles and have yet to see
that particular issue addressed.

Comments

15 Jul 2009 - 7:08am
jasonrobb
2009

Micheal,

That's a good question. As far as solid research goes, I haven't
come across it, though I'll keep my eyes open.

Wearing my user hat, if I saw 0 stars/0 reviews everywhere, I would
probably get the impression that this site doesn't have a
community.

I'm having a similar challenge at my company
(http://languageinternational.com). We don't have reviews or ratings
yet, but we do have a few testimonials.

One possible solution could be to only show the stars or reviews when
you have 1 or more. As in, don't even show the header "Reviews"
unless you actually have some. That way you can provide value when
you have it, and avoid giving the impression that nobody is home when
you don't have ratings/reviews.

Good luck,

Jason R.

--
Jason Robb
http://jasonrobb.com
http://uxboston.com
http://uiscraps.tumblr.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from ixda.org (via iPhone)
http://www.ixda.org/discuss?post=43730

15 Jul 2009 - 8:48am
Ian Chan
2005

Another option is to have somebody go through and review each item
once. I know it seems disingenuous, but frankly it's the solution most
often used. Hiding the ratings until you have one results in UI
inconsistency. The problem you're having has to do with user
participation -- sometimes you just need to seed the site and userbase.

adrian

> One possible solution could be to only show the stars or reviews when
> you have 1 or more. As in, don't even show the header "Reviews"
> unless you actually have some. That way you can provide value when
> you have it, and avoid giving the impression that nobody is home when
> you don't have ratings/reviews.
>

15 Jul 2009 - 9:08am
Joshua Porter
2007

As Adrian mentions, seeding is a bit unethical, as the reviews aren't
authentic. (keeping reviews authentic is an ongoing problem all the
time, however)

Another idea is to reward first reviewing behavior.

So, give people a special badge on their profile when they review an
item first.

Yelp does this for business reviews. You can see an example here:

http://www.yelp.com/user_details?userid=AgzHzIzC6ZHTKgker0egAg

Notice that this person has 50 "firsts". (in the left-hand column)

This method has several positive effects.

1) It gets the item in the database (hugely important for user-
generated content sites like Yelp)
2) It gets a review active for that item (thus getting over the hurdle
in question)
3) It rewards users in a unique way (people recognize this distinction
as different than a simple review).

This method is a win-win for both the site and users.

Josh

On Jul 15, 2009, at 9:48 AM, adrian chan wrote:

> Another option is to have somebody go through and review each item
> once. I know it seems disingenuous, but frankly it's the solution
> most often used. Hiding the ratings until you have one results in UI
> inconsistency. The problem you're having has to do with user
> participation -- sometimes you just need to seed the site and
> userbase.
>
> adrian
>
>> One possible solution could be to only show the stars or reviews when
>> you have 1 or more. As in, don't even show the header "Reviews"
>> unless you actually have some. That way you can provide value when
>> you have it, and avoid giving the impression that nobody is home when
>> you don't have ratings/reviews.
>>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

15 Jul 2009 - 8:15am
C K Vijay Bhaskar
2009

As a user, I typically dont go and start rating - depending on my
mood, time available, usefulness of the content etc on product pages.

Also I have seen many users who do a "random" rating just to
complete a process without giving away the actual feel/usefulness of
the product.
The other way I can think of is to display how many people have
visited the page and the time that they have spent on the page. Even
though they may not have rated the product, this measure could be a
good indicator on how many people visit and time that they may have
spent on the page.
This data can be tinkered to assess the content etc and decide if the
product page, its description etc was in fact readable or useful.
Hope this helps.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=43730

15 Jul 2009 - 11:00am
jasonrobb
2009

Adrian,

With regards to hiding ratings:

Zappos, for instance, hides the ratings on the search results page,
until the shoe has been rated.

http://www.quicksnapper.com/files/1946/5248341084A5DFBFF1CD89_m.png

Being consistent for the sake of consistency isn't a good reason to
be consistent. Hiding the ratings in this case for unrated shoes
reduces the noise on the page. This is helpful, and well done, I
think.

Cheers,

Jason R.

--
Jason Robb
http://jasonrobb.com
http://uxboston.com
http://uiscraps.tumblr.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=43730

15 Jul 2009 - 11:31am
Ian Chan
2005

Jason,

Not sure I like the zappos approach. Visually, i would prefer to have
the ratings even if they're "blank." For consistency's sake ;-) (If
consistency isn't desired for its own sake, then for what else?) Now
that's just a visual argument. In terms of what it indicates, the lack
of ratings to me indicates the same thing as an unrated rating. So I
dont see how they've solved that in any way: no ratings here, next to
all the other shoes that do have ratings, just says "no rating" ... Or
so it seems to me. To me absence of the ratings can be noise -- when
something seems to be missing, isnt that the same kind of noise as
something that's not yet filled out?

We're splitting hairs, but that's what we do well.

On a side note, this opens a back door to social interaction design
and social usability matters: A ratings system has two social
functions: to encourage the act of rating by user; to display average
ratings. Interestingly, my suggestion favors the former; yours I think
favors the latter. My suggestion is to leave ratings in there -- we
want user to rate -- and if needs be then have one user rate just to
seed the activity. Your suggestion is to remove it because it doesnt
show anything, which is totally valid and true.

How would we design a principle here? If the input element also
provides a social connotation, which function prevails? The call to
action or the display of data?

Personally this is why I think a lot of social design elements
introduce social bias and distortion : input mechanisms are the
display mechanisms also. But that's another topic....

thoughts?

adrian

>
> With regards to hiding ratings:
>
> Zappos, for instance, hides the ratings on the search results page,
> until the shoe has been rated.
>
> http://www.quicksnapper.com/files/1946/5248341084A5DFBFF1CD89_m.png
>
> Being consistent for the sake of consistency isn't a good reason to
> be consistent. Hiding the ratings in this case for unrated shoes
> reduces the noise on the page. This is helpful, and well done, I
> think.
>
> Cheers,
>
> Jason R.

15 Jul 2009 - 3:52pm
Diana Wynne
2008

I'd rather see a blank than read a review that was only written to solve a
UI problem.

Yelp's "firsts" do reward a certain kind of competitive behavior. It doesn't
mean those people's opinions are particularly reliable.

Diana

On Wed, Jul 15, 2009 at 9:31 AM, adrian chan <adrian at gravity7.com> wrote:

> Jason,
>
> Not sure I like the zappos approach. Visually, i would prefer to have the
> ratings even if they're "blank." For consistency's sake ;-) (If
> consistency isn't desired for its own sake, then for what else?) Now that's
> just a visual argument. In terms of what it indicates, the lack of ratings
> to me indicates the same thing as an unrated rating. So I dont see how
> they've solved that in any way: no ratings here, next to all the other shoes
> that do have ratings, just says "no rating" ... Or so it seems to me. To me
> absence of the ratings can be noise -- when something seems to be missing,
> isnt that the same kind of noise as something that's not yet filled out?
>
> We're splitting hairs, but that's what we do well.
>
> On a side note, this opens a back door to social interaction design and
> social usability matters: A ratings system has two social functions: to
> encourage the act of rating by user; to display average ratings.
> Interestingly, my suggestion favors the former; yours I think favors the
> latter. My suggestion is to leave ratings in there -- we want user to rate
> -- and if needs be then have one user rate just to seed the activity. Your
> suggestion is to remove it because it doesnt show anything, which is totally
> valid and true.
>
> How would we design a principle here? If the input element also provides a
> social connotation, which function prevails? The call to action or the
> display of data?
>
> Personally this is why I think a lot of social design elements introduce
> social bias and distortion : input mechanisms are the display mechanisms
> also. But that's another topic....
>
> thoughts?
>
> adrian
>
>
>> With regards to hiding ratings:
>>
>> Zappos, for instance, hides the ratings on the search results page,
>> until the shoe has been rated.
>>
>> http://www.quicksnapper.com/files/1946/5248341084A5DFBFF1CD89_m.png
>>
>> Being consistent for the sake of consistency isn't a good reason to
>> be consistent. Hiding the ratings in this case for unrated shoes
>> reduces the noise on the page. This is helpful, and well done, I
>> think.
>>
>> Cheers,
>>
>> Jason R.
>>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

15 Jul 2009 - 2:48pm
Michael Etgen
2008

Well, the particular angle I'm thinking about here is more of the
shopper evaluating products versus a customer coming back to
rate/review.

Are there particular behavior patterns we see when a shopper finds a
product with no reviews/ratings that "looks good"?

If the same product has ample ratings/reviews elsewhere but not here,
what is the user's perception (this site stinks or is untrustworthy?)
and how does it affect their behavior? If the same product has few or
no ratings/reviews at all anywhere, then what's my
perception/behavior?

- Michael

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=43730

16 Jul 2009 - 2:08pm
Jared M. Spool
2003

On Jul 15, 2009, at 3:52 PM, Diana Wynne wrote:

> Yelp's "firsts" do reward a certain kind of competitive behavior. It
> doesn't
> mean those people's opinions are particularly reliable.

You're right that individual people's opinions aren't particularly
reliable, but the system tends to self correct.

In our work, we've noticed that when people see a review they don't
agree with, they are more motivated to review than if they see reviews
they do agree with.

So, if the first reviewers skew away from the average beliefs, it will
eventually populate in the direction that matches the average.

Jared

16 Jul 2009 - 1:41pm
Jared M. Spool
2003

On Jul 14, 2009, at 8:32 AM, Michael Etgen wrote:

> Does anyone know of any academic or industry research that addresses
> how users react-to/perceive products on a site when there is a
> function displayed for ratings/reviews, but none (or maybe just 1)
> have been completed?

We've been talking about it. I mentioned it in this presentation: http://is.gd/1wndV
(The specifics you're interested start about slide 26.)

Basically, we've found there's a negative reaction that users have
when the review or rating system isn't populated or is only filled
with negative reviews or ratings.

Our current thinking is that you only want to have a review/rating
system if you are sure folks will participate.

When we studied Amazon's reviews, we estimated that for the average
product, it takes 1300 purchases to generate a review. If you want 20
reviews for your most popular products and you run at an average
conversion rate of 2%, we estimate you need 1,300,000 visitors to the
product to succeed.

On low volume sites, you have to resort to other strategies to
encourage reviewing & rating.

Hope that helps,

Jared

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: @jmspool

Syndicate content Get the feed