Help us spread the news!

Rating Systems

A rating system is a great way to create a dialogue between you and your users. You receive feedback on your service and your users feel like their opinion is heard.

When choosing a rating system you want to think about what kind of data will be the most useful to you as well as to your users. Some common reasons to use a rating system are:

1. To improve the quality of your product or service

2. Recommend the best quality product to your users

3. Target products to your users based on their tastes

 

There are a variety of rating systems out there from the facebook ‘like’, to the 5 star system used by top websites such as Netflix, Itunes, and Amazon.

With each rating system model, there is a give and take between the complexity of the rating system and its usability. While you might get better data from a 10 star rating system, your users will be more likely to actually use the simpler 5 star system. The best rating system models are simple enough to be intuitive and engaging for your users, but complex enough to provide usable data.

Problem with the 5 star system

(From XKCD)

Although 5 star rating is one of the most prevalent rating systems, it suffers from a few major flaws:

○ Averaged Ratings: this means that 1) a bad review can skew the data, especially if there are only a few ratings, and 2) It doesn’t take into account the number of positive or negative ratings. A product, for instance, with a single 5 star rating (average = 5) will be rated higher than a product with eighty 5 star ratings and one 1 star rating (average = 4.95).

○ Polarized Ratings: most users tend to review either when they are very happy or very upset. A study, using Amazon data, found that the majority of reviews were bimodal, receiving either 1 or 5 stars, with very little 2-4 star ratings.

○ J Shape Ratings: The same Amazon study found that while almost all ratings were 1 or 5 stars, the majority fell into the 5 star category, giving the distribution a “J” shape.
Take Uber, for instance; in a Wired review of Uber ratings, the author criticizes how most riders give 5 star driver ratings across the board, “I’ve ridden with 4.7 star drivers who wear gloves and open passenger doors and 4.7 star drivers  who couldn’t pass a road test”.
The XKCD comic (above) also does a good job of illustrating this problem.

Alternative Rating Systems

Here, I have compiled a list of better rating alternatives to the 5 Star System, that should help lead to more responses, better feedback for your company, and more useful information for your users.

 

Thumbs up and down Model

In this model, users are given two options: like or dislike. This rating system is very easy to use and the information it conveys is very intuitive: this number of people liked the video/ this number of people did not. Since the majority of ratings in a 5 star model are either 1 or 5 stars, it makes some sense to simplify the ratings to a “like” or “dislike”.

 


Youtube uses this particular model effectively. It allows users to rate others’ videos with a simple like or dislike, and then sort videos by most “likes”. However, depending on your particular product or service, this binary model can feel limiting regarding the feedback you can receive from your audience.

 

Sliding Scale

The sliding scale allows for a wider range of responses, creating a finer rating system for more exact data on user responses. In this model, users are given a slider from negative to positive and asked to place themselves on the scale.

A good example of this is Steepster, which allows users to rate their teas on a sliding scale:

 

 

The Steepster scale also includes markers on the slider with the ratings of other teas to help compare products. You can read more about it here.

 

Comparison Rating

In certain instances, numbered scales can seem arbitrary and unhelpful for users. Which product should I buy: the one with a 4.3 star average or the one with 4.4? Instead of a scale rating system, the comparison rating system gives users the choice of two options and has them choose which they prefer.
College Humor, an online comedy website, uses this model to help choose the “Top Horror Movie” or the “Best Fictional Band“.
For the top horror movie list, users are given two movies and asked to vote for one.

All the responses are then compiled to create a ordered ranking of the “top horror movies”.


This comparison model can be used to help customers directly compare two products they may be considering buying. You can also use the comparison data to create a rank-ordered list, so users can see the “top products” or “top content” based on other users’ preferences. This system, however, requires that you have like products that can be compared and a set “pool” of choices for users to vote on.

 

Emotion Rating

The Emotion Rating system does not rely on any particular scale (e.g. bad to good), but instead asks users to choose their emotional reaction.  A good example of this rating system is the Vicomi Feelbacks, which ask users to choose an emotional reaction from the 5 choices: Happy, Amused, Angry, Indifferent, Excited.


This type of rating system works especially well for reviewing content on blogs, online magazines, and newspapers. Users read the article and then select how they feel from the different emotion options.

This model provides you with insights into how users are reacting on an emotional level to your content, and can be used to tailor related content based on emotion rather than just by similar topics. You can learn more about it here.

 

Color Rating

The Color Rating system, like the one created by Lidoli, uses a model which visually displays ratings based on a color-coded scale. First, users choose a rating that is associated with a specific color:

 

The ratings for each product are then compiled into a color matrix and ordered with positive feelings on the top and negative feelings on the bottom:

 

 

The color rating system visually displays how various users feel about the different product rather than collapsing it into an average. Learn more about Lidoli’s model here.

Conclusion

Whether you are using a rating system to sell more products, write better content, or improve recommendations, make sure that you system is beneficial for both you and your users. Remember, feedback is the key to improvement so invest some time in finding the best rating system for you.

Have a favorite rating system? Let us know in the comments below.