When do reviews adequately portray a product or service?

It’s not always about quantity

Generally the more reviews you have, the more they converge on a consensus assessment of the experience provided (Figure 1). That suggests to the prospective customer that the experience is very predictable and low risk.

Image

Figure 1. Only 35 reviews, but the rating distribution is starting to converge on 4 stars.

But, that is not always true. Some experiences are polarizing: some people love them while others hate them. Sometimes random chance will bring these two sides together in near equal numbers, with results confusing to the prospective customer (Figure 2).

Figure2

Figure 2. Many reviews, but consensus is polarized.

Helpfulness

There is another dimension to many review sites (Figure 3), where readers assess the helpfulness of reviews. We found that this information tends to bring clarity to even polarized reviews, however the review sites do not present this information in a helpful way.

Figure3

Figure 3. Review sites have different ways of indicating the helpfulness of reviews, but while they display this information for each review and sometimes the object of the review it is uncertain what the implications of this information is.

We found that readers’ assessments of helpfulness gradually peak and then decline over time (Figure 4). It seems that the peak is the point where the reviews finally begin to capture all the information necessary for consumers to make an informed prediction of whether the reviewed object or entity is right for them. We call this point information sufficiency. Since matching expectations with outcomes is the key to customer satisfaction, we recommend that consumers be told when the available reviews might not allow them to make an informed prediction of their outcome. This signal may be best sent with a simple set of icons that indicate whether information sufficiency has been reached (Figure 5).

Figure4

Figure 4. The helpfulness of a set of reviews was observed to peak over time. The time at which the peak occurs is denoted the point of information sufficiency. The vertical dotted line matches one in Figure 6.

Figure5

Figure 5. Since matching expectations with outcomes is the key to customer satisfaction, we recommend that consumers be told when the available reviews might not yet allow them to make an informed prediction of their outcome.

Helpfulness and satisfaction interact

Our final observation on this issue is that information sufficiency seems to mark the minima of overall customer satisfaction (Figure 6). We believe this occurs because after information sufficiency is reached prospective customers are better able to predict if they will be satisfied with the experience by reading the reviews. Therefore, customers that expect to be unhappy stay away and those that participate are a self-selected better match for the product or service being offered.

Figure6

Figure 6. The vertical dotted line matches one in Figure 4 that marks the point of information sufficiency. Customer satisfaction steadily decreases until this point, and increases thereafter as customers who are a good fit for the value offering are better able to self-select into the experience while those who are a poor fit stay away.

Advertisements

One thought on “When do reviews adequately portray a product or service?

  1. Pingback: Revisiting McPhee’s Theory of Exposure and the Long Tail | Artistic Analytics

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s