First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.
The users’ guide to user reviews
- — 18 July, 2003 08:08
Mike Woznicki has been reviewing computer hardware, software, books, and videos on Amazon.com for the past five years. Woznicki, an IT manager and technical certification instructor, certainly knows his stuff. He is currently ranked number 11 among Amazon.com Top Reviewers, based on visitors’ assessments of the helpfulness of his reviews.
But Woznicki isn’t your typical reviewer. For one thing, he gets most of the products he reviews free from manufacturers. For another, he has reviewed more than 1500 products. Woznicki’s situation brings up a couple of questions. Can receiving freebies influence his reviews? And can anyone really know that many different products well?
Woznicki’s circumstances are unusual, but questions about user reviews are not. Does the reviewer have any real knowledge of the area he or she is writing about? Could a sly company submit top reviews about itself to drive up its own score? Finding reviews you can trust is always an issue.
We went behind the scenes to find out how much you can depend on user reviews.
Our findings? In general, you can trust a lot of user reviews, though it might take some work — and a dose of scepticism — to ferret out the reliable ones. Of course, the great thing about user reviews is that the people who write them live with the products in the real world. By contrast, editorial reviews in PC World, for example, are usually restricted by time and limited testing scenarios. Real customers often encounter problems after several months, or even a year — something that an editorial review can’t capture. In fact, many PC World editors are big fans of user reviews for these reasons.
A few user-review Web sites reported that some funny business does go on, however. Sometimes, a merchant’s employees will try to post phoney reviews either to bump up their own company’s rating or to drive down the competition’s score. The sites do their best to stop such tampering before the reviews get posted. Beyond that, some sites offer tools to help you filter what you see so you can get more out of the user reviews.
Our two favourites: Epinions.com and PCPhotoReview.com. Epinions.com displays reviews in a digestible format and provides useful filtering tools. And for shutterbugs, PCPhotoReview.com’s write-ups are usually packed with valuable details on a wide range of digital (and film) cameras (see the table on page 42 for a list of all the sites we examined).
Generally, posting a review is straightforward. Most sites require that you sign up with an e-mail address before you submit your review. Then you write the review, select a rating (or whatever the site requires), and click Post Review (or something similar). Most sites examine postings before allowing them to appear; sites take a few hours or even a couple of days to post reviews. Epinions.com is the exception; it posts reviews immediately.
Sites like PriceGrabber.com and Amazon.com monitor submissions for swear words and nonsense text. Most sites will also exclude a review if it’s clear that the user either does not own the product or has never interacted with the merchant. Sites will take down offensive postings, but they usually won’t edit the text or correct typos.
Web sites that specialise in merchant ratings sometimes run into fraudulent activity. “Many unscrupulous companies do try to submit reviews for themselves on ResellerRatings.com,” Scott Wainner, the site’s chief executive officer, acknowledges. Such companies are usually merchants with poor ratings trying to boost their overall score. But some companies also bad-mouth competing merchants to force their ratings to drop.
To combat spurious reviews, PriceGrabber.com and ResellerRatings.com employ a variety of antifraud measures. Both sites indicated that they use specific tools to investigate suspicious postings, but they wouldn’t disclose additional details. If the sites find that a merchant has acted improperly, they remove the guilty party from their listings.
Questionable postings may also come from people who are compensated for writing reviews of products or services. Such compensation can come in the form of free merchandise or cash.
Mike Woznicki gets to keep the products that manufacturers send him. He says that this arrangement doesn’t affect what he writes because he chooses the products he wants to review. “The idea that I would favour one [company] over the other doesn’t happen,” he says. (Note: manufacturers provide products for PC World to evaluate. Hardware is sent back when testing is completed. Other media outlets handle reviews differently.)
Woznicki’s situation seems to be the exception rather than the rule, however. Other reviewers said that they don’t receive free products. Epinions.com is the only site we looked at that pays money — albeit a nominal sum — for reviews.
In light of the potential for conflicts of interest and fraud on review sites, you do need to exercise your judgement of whose opinions you can trust. You must also look out for reviews from writers who may not know what they’re talking about. For example, a negative review may stem from a problem involving the user and not the product itself. Perhaps the reviewer had used the product for just one day or didn’t know how to operate it.
Work the tools
Trusting a site’s reviews is one side of the equation; ultimately, it’s what you get out of a site that counts. Some make it easy to find the reviews you want. Epinions.com, CNet Download.com, and PCPhotoReview.com offer great search engines and link you to the reviews you’re after quickly. On Photo.net, however, the user reviews are buried several clicks into the EzShop section of the site, and its search engine leaves much to be desired. When we searched Photo.net for the Canon PowerShot G2, for example, the site didn’t return results of any kind, even though the product was listed with 22 reviews.
When we looked at the sites, we didn’t find many reviews of newer products. Plus, some sites have pages of reviews for one product but a handful of posts for others. NewEgg.com, a shopping site, could boast 112 reviews for the ATI Radeon 9700 Pro graphics board, but had just six Windows XP Home Edition reviews.
Different sites attract users with different interests, so it’s worth your while to hunt around until you find sites that match your interests and expertise level.
Though reviews on most sites have undergone at least minimal inspection, it doesn’t necessarily mean they’re worth reading. That’s where community feedback — if any — comes into play. Sites like Amazon.com, Epinions.com, and PCPhotoReview.com ask you to vote on how helpful you found a review. Based on those votes, the sites will automatically push the highest-rated reviews to the front so you see them first, which saves you from wading through the muck.
Most sites let you sort reviews by date or rating. Epinions.com offers even greater customisation: you can create a ‘Web of Trust’, which lets you pick reviewers whose opinions you feel are in line with yours. After that, any relevant reviews of theirs will bubble up to the top of the list.
Even after you’ve used filtering and other tools on the sites, you still have to decide whether the review says anything worthwhile. “The ideal review is detailed, compares a product to similar products, and is written after someone has used the product for long enough to really know it well,” says John Shafer, channel manager for PCPhotoReview.com. So delve into the details, especially the negative comments, and look beyond the overall rating (see ‘Make the most of user reviews’ on page 40 for additional tips).
With practice, you’ll become very adept at scanning reviews and zooming in on the informative ones. Remember that most people are motivated to write reviews either when they’re thrilled to bits or when they’re just ready to vent. So expect to see reviews that are either very favourable or really negative (one good reason to focus on the details and ignore the overall ratings). And make sure you read as many reviews as you can handle. That way, the useless or fishy ones will be balanced out by the real gems.
Make the most of user reviews
As with any source of information, you must know what to look for. Keep these tips in mind the next time you peruse the postings.
- Skim past the rating: one person’s five may be another’s three, so there’s often little value in the overall score. You have to focus on the details to understand the user’s point of view.
- The more reviews, the merrier: “You should never read just one or two reviews… Any one person may be off their rocker, so to speak, whereas a dozen or two dozen people probably aren’t,” says Scott Wainner of ResellerRatings.com.
- Add up the numbers: if you do like to consider the overall cumulative rating of a product, make sure you factor in the total number of reviews. The more reviews a product gets, the better the chance of a representative rating.
- Use the tools at hand: take advantage of filters and other features to make your search more efficient and refine what you see.
- Focus on the negatives: as bad as it sounds, you can save a lot of time if you learn about the problems people have had with a product or a merchant. Plus, a single problem may be unique to a reviewer’s complicated PC configuration, but if you see a recurring theme in a dozen reviews, there’s probably something to it.
- Locate the experts: if you want a digital camera, visit a site that specialises in digital camera user reviews, like PCPhotoReview.com. People who post reviews on these sites tend to have a higher level of expertise and provide more in-depth details.
- Dip into discussions: if all else fails, go to Google Groups (http://groups.google.com) and search for the product or merchant in the Usenet archives. It’s often a great source for advice, if you’re willing to wade through many posts and the off-topic turns they can take.
|Web sites||Site monitors reviews||Users can sort ratings||Advanced customisation features 1||Antifraud measures||Comments||URL|
|Amazon.com (2)||Yes||Yes||Yes||No||Reading reviews while you shop is easy. Certain categories like digital cameras often have overly positive reviews.||www.amazon.com|
|ComputingReview.com||Yes||Yes||No||Yes||The site covers a surprisingly limited selection of computer products, and offers few reviews for each item.||www.computingreview.com|
|CNet Download.com||Yes||Yes||No||Yes||The site is easy to navigate, its users love to speak up, and you can expect to find a lot of geeky comments.||www.download.com|
|Epinions.com (3)||No||Yes||Yes||Yes||It’s easy to find product reviews in a flash, and the site provides useful ways to customise review preferences.||www.epinions.com|
|NewEgg.com (2)||Yes||Yes||No||No||Covers a wide range of tech products, and the users share a high level of expertise. Site can be hard to navigate.||www.newegg.com|
|PCPhotoReview.com||Yes||Yes||Yes||Yes||This specialised community offers an impressive number of camera-related products and reviews to go with them.||www.pcphotoreview.com|
|Photo.net||Yes||Yes||No||No||The reviews are often by experts, but the site’s interface makes it hard to get to them. Selection of cameras is small.||www.photo.net|
|PriceGrabber.com||Yes||Yes||No||Yes||Nicely combines ratings of merchants and products.||www.pricegrabber.com|
|ResellerRatings.com||Yes||No||No||Yes||Provides easy at-a-glance info.||www.resellerratings.com|
|Footnotes: 1 Options let users sort reviews in ways other than by date or rating — by particular reviewers or different aspects of transactions, for example; 2 Primarily a shopping site; 3 DealTime has announced plans to acquire Epinions.com.|