Sturgeon’s law: Ninety percent of everything is crap.1
Rankings & reviews online
The internet is full of websites that ostensibly rank, rate, and/or review companies within a given industry. Most of these websites are crappy. Generally, these ranking websites cover industries where affiliate programs offering website owners large commissions are common.
Here are a few examples of industries and product categories where useless review websites are especially common:
- Credit cards
- Web hosting services
- Online fax services
- VoIP services
- VPN services
- Foam mattresses
If you Google a query along the lines of “Best [item from the list above]” you’ll likely receive a page of search results with a number of “top 10 list” type sites. At the top of your search results you will probably see ads like these:
Lack of in-depth evaluation methodologies
Generally, these “review” sites don’t go into any kind of depth to assess companies. As far as I can tell, rankings tend to be driven primarily by a combination of randomness and the size of commissions offered.
Admittedly, it’s silly to think that the evaluation websites found via Google’s ads would be reliable. Unfortunately, the regular (non-ad) search results often include a lot of garbage “review” websites. From the same query above:
Most of these websites don’t offer evaluation methodologies that deserve to be taken seriously.
Even the somewhat reputable names on the list (i.e. CNET & PCMag) don’t offer a whole lot. Neither CNET nor PCMag clearly explain their methodologies, and the written content doesn’t lead me to believe either entity went in depth to evaluate the services considered.2
Fooling consumers
If consumers easily recognized these bogus evaluation websites for what they are, the websites would just be annoyances. Unfortunately, it looks like a substantial portion of consumers don’t realize these websites lack legitimacy.
Google offers a tool that presents prices that “advertisers have historically paid for a keyword’s top of page bid.” According to this tool, advertisers are frequently paying several dollars per click on the kind of queries that return ads for bogus evaluation websites:3
We should expect that advertisers will only be willing to pay for ads when the expected revenue per click is greater than the cost per click. The significant costs paid per click suggest that a non-trivial portion of visitors to bogus ranking websites end up purchasing from one of the suggested companies.
How biased are evaluation websites found via Google?
Let’s turn to another industry. The VPN industry shares a lot of features with the web hosting industry. Both VPN and web hosting services tend to be sold online with reasonably low prices and reoccurring billing cycles. Affiliate programs are very common in both industries.
There’s an awesome third-party website, ThatOnePrivacySite.net, that assesses VPN services and refuses to accept commissions.4 ThatOnePrivacySite has reviewed over thirty VPN services. At the time of writing, only one, Mullvad, has received a “TOPG Choice” award,5 indicating an excellent review.6
Interestingly, Mullvad doesn’t have an affiliate program. That allowed me to perform a little experiment. I Googled the query “Best VPN service”. I received 15 results directing to websites that ranked VPN services.
Six of the results came from paid ads. None of those six websites listed Mullvad.
Of the nine websites in the organic results, only three listed Mullvad:7
- Tom’s Guide
- TheBestVPN.com
- PCWorld
