What Do Amazon’s Star Ratings Really Mean?

Nothing in the maze that is online shopping is ever what it seems. Unless it’s a product you already know, buying something is always a bit of a gamble. The way the fabric feels, whether the cream will give you a rash—these things remain impossible to assess through a screen. To make a decision, buyers often rely on those who purchased before them: They read the reviews.

On Amazon, like many ecommerce sites, buyers also leave star ratings for items, from one to five. They’re important shorthand, symbolizing at a glance how people felt about a certain item. They also serve as a convenient way to filter, rank, and manage the overwhelming amount of choice that is a hallmark of 21st-century living. (How else can someone decide between, say, over 40,000 shower curtain options?) Higher scores are crowdsourced seals of approval; Amazon even built brick-and-mortar stores devoted entirely to goods with four or more stars.

But Amazon’s little yellow luminaries are less straightforward than they appear. Why does one product get 4.7 stars and another get 4.8? I know what you’re thinking: Louise, clearly these are just averages—take the total number of stars and divide that by the number of reviews, and there you go. Just simple math! But reader, that is not the case. While Amazon says these stars reflect the “average customer review” of an item, the calculation behind them is more complicated—and mysterious—than that.

Starting in 2015, Amazon began weighting stars using a proprietary machine-learning model. Some reviews now count more than others in the total average, based on factors like how recent they are and whether they come from “verified” purchasers (meaning Amazon could confirm the reviewer actually bought the item they claimed to love or hate). David Bryant, an Amazon seller who also blogs about the company, believes Amazon may also take into consideration factors like the age of the reviewer’s account and the average star rating they usually leave. “There appears to be some discount applied to reviewers who predominantly leave negative reviews,” he says.

The new system is largely a good thing, at least from the customer’s perspective. Five-star ratings from three years ago probably shouldn’t count as much as three-star ratings left just last week. But it also represents an additional layer of algorithmic secrecy, which can be frustrating for third-party merchants on the ecommerce site. Amazon’s seller forums are full of merchants struggling to understand the black box of how their products are ranked and rated. “Why would a 5-star review make the average drop a full star?” asked one poster last year.

Sellers have a lot to lose if their product rating goes down. Good word of mouth can turn a relatively unknown item into a huge success. “Reviews are more important than a brand,” says Fred Dimyan, CEO of Potoo Solutions, a firm that consults with ecommerce companies. “There’s major brands that are being crushed by small direct-to-Amazon or direct-to-consumer brands.” He points to Cali White activated charcoal and coconut toothpaste for example, the number-one best-selling toothpaste on Amazon, which beat out famous companies like Crest and Colgate.

Star ratings aren’t only influenced by Amazon’s algorithms. They’re also sometimes manipulated by sellers who pay for glowing reviews in order to raise the rank of their products in Amazon’s search results. “Amazon very quickly removes positive reviews because they find them sometimes to be fake,” Dimyan says. “They more quickly remove positive reviews than they remove negative reviews.”

Even “verified” Amazon reviews may not always be what they appear. “There are plenty of verified reviews that need to come down and should be deleted,” says Chris McCabe, a former Amazon employee who now runs a consultancy for Amazon sellers. “All of them were from buyer accounts that had a stake in what they were buying.” For example, the seller may have reimbursed them for the item via PayPal after they bought it. Amazon prohibits posting reviews “in exchange for compensation of any kind (including free or discounted products, refunds, or reimbursements) or on behalf of anyone else.”

Courting reviewers is just as important to Amazon as the reviews they write are to sellers. The company relies on a sprawling network of contributors—who mostly do this work for free—to get customers to trust its products, many of which aren’t sold by household names. This setup has spawned an elaborate ecosystem that goes beyond just distinguishing real reviews from fake. First, there’s the Vine program, where sellers can have new products reviewed by Amazon’s “Vine Voices” in exchange for giving them free merchandise. (If you’re thinking, “Can I get free stuff?” the answer is probably no. The Vine program is unfortunately invite-only.)

Amazon also reviews its own reviewers and ranks the Top 10,000 Contributors on a constantly shifting leaderboard. Members of this elite crew are given badges to display on their profiles, like “Top 500 Reviewer.” The rankings aren’t influenced just by the number of reviews a person has but also by how many customers found their feedback to be helpful. On the webpage where Amazon lists these esteemed tastemakers, it invites you to “Take a minute to explore the reviews written by these customers. They will inspire you.” The company also honors top reviewers from previous years in its Hall of Fame. But the system is also competitive—reviewers who enjoy being in the top 10,000 can’t simply rest on their laurels. A handy Reddit bot monitors who gets cut from the list each day.

Have a tip about Amazon? Contact the author at louise_matsakis@wired.com or via Signal at 347-966-3806.

When you buy something using the retail links in our stories, we may earn a small affiliate commission. Read more about how this works.