Tag: Wireless market

Photo of a frustrated person with a broken phone

Consumer Reports’ Broken Cell Service Rankings

Christian

Several months ago, I published a blog post arguing that Consumer Reports’ cell phone rankings were broken. This month, Consumer Reports updated those rankings with data from another round of surveying its subscribers. The rankings are still broken.

Consumer Reports slightly changed its approach this round. While Consumer Reports used to share results on 7 metrics, it now uses 5 metrics:

  1. Value
  2. Customer support
  3. Data
  4. Reception
  5. Telemarketing call frequency

Of the 19 carriers Consumer Reports’ assesses, only 5 operate their own network hardware.1 The other 14 carriers resell access to other companies’ networks while maintaining their own customer support teams and retail presences.2

Several of the carriers that don’t run their own network offer service over only one host network:

  • Cricket Wireless – AT&T’s network
  • Page Plus Cellular – Verizon’s network
  • MetroPCS – T-Mobile’s network
  • CREDO Mobile – Verizon’s network
  • Boost Mobile – Sprint’s network
  • GreatCall – Verizon’s network
  • Virgin Mobile – Sprint’s network

To test the validity of Consumer Reports’ methodology, we can compare scores on metrics assessing network quality between each of these carriers and their host network. At first glance, it looks like the reception and data metrics should both be exclusively about network quality. However, the scores for data account for value as well as quality:3

Data service indicates overall experience (e.g., cost, speed, reliability) with the data service.
I think it was a methodological mistake to account for value within the data metric then account for value again in the value metric. That leaves us with only the reception scores.4 Here are the scores the four host operators get for reception:

  • Verizon – Good
  • T-Mobile – Fair
  • AT&T – Poor
  • Sprint – Poor

How do those companies’ scores compare to scores earned by carriers that piggyback on their networks?

  • Cricket Wireless has good reception while AT&T has poor reception.
  • Page Plus and Verizon both have good reception.
  • MetroPCS has good reception while T-Mobile has fair reception.
  • CREDO and Verizon both have good reception.
  • Boost has very good reception while Sprint has poor reception.
  • GreatCall and Verizon both have good reception.
  • Virgin has good reception while Sprint has poor reception.

In the majority of cases, carriers beat their host networks. The massive differences between Cricket/AT&T and Boost/Sprint are especially concerning. In no cases do host operators beat the carriers that piggyback on their networks. I would have expected the opposite outcome. Host networks generally give higher priority to their direct subscribers when networks are busy.

The rankings are broken.

What’s the problem?

I see two especially plausible explanations for why the survey results aren’t valid for comparison purposes:

  • Non-independent scoring – Respondents may take prices into account when assessing metrics other than value. If that happens, scores won’t be valid for comparisons across carriers.
  • Selection bias – Respondents were not randomly selected to try certain carriers. Accordingly, respondents who use a given carrier probably differ systematically from respondents that use another carrier. Differences in scores between two carriers could reflect either (a) genuine differences in service quality or (b) differences in the type of people who use each service.

Consumer Reports, please do better!

My earlier blog post about Consumer Reports’ methodology is one of the most popular articles I’ve written. I’m nearly certain staff at Consumer Reports have read it. I’ve tried to reach out to Consumer Reports through two different channels. First, I was ignored. Later, I got a response indicating that an editor might reach out to me. So far, that hasn’t happened.

I see three reasonable ways for Consumer Reports’ to respond to the issues I’ve raised:

  • Adjust the survey methodology.
  • Cease ranking cell phone carriers.
  • Continue with the existing methodology, but mention its serious problems prominently when discussing results.

Continuing to publishing rankings based on a broken methodology without disclosing problems is irresponsible.

Markets Are Honest

Christian

I’ve been reading a ton of articles with commentators’ takes on whether a merger between Sprint and T-Mobile will be good or bad for consumers. Almost everything I’ve read has taken a strong position one way or the other. I don’t think I’ve seen a single article that expressed substantial uncertainty about whether a merger would be good or bad.

It could be that everyone is hugely biased on both sides of the argument. Or maybe the deal is so bad that only incredibly biased people would consider making an argument that the merger will be good for consumers. I’m not sure.

I like to look at how markets handle situations I’m uncertain about. In the last few years, I’ve regularly seen liberal politicians and liberal news agencies arguing that we’re about to see the end of Trump’s presidency because of some supposedly impeachable action that just came to light. I’m not Trump’s biggest fan, but I’ve found a lot of arguments about how he’s about to be impeached too far-fetched. I have a habit of going to the political betting market PredictIt when I see new arguments of this sort. PredictIt has markets on lots of topics, including whether or not Trump will be impeached.

Politicians and newspapers have an incentive to say things that will generate attention. A lot of the time, doing what gets attention is at odds with saying what’s true. People putting money in markets have incentives that are better aligned with truth.

Most of the time I’ve seen articles about Trump’s impending impeachment, political betting markets haven’t moved much. In rare occasions where markets moved significantly, I’ve had a good indication that something major actually happened.

Wall Street investors have a strong incentive to understand how the merger will actually affect network operators’ success. Unsurprisingly, T-Mobile’s stock increased substantially when key information indicating likely approval of a merger came out. Sprint’s stock also increased in value.

What’s much weirder is that neither Verizon’s stock nor AT&T’s stock seemed to take a negative hit on the days when important information about the merger’s likelihood came out. In fact, it actually looks like the stocks may have increased slightly in value.1

You could tell complicated stories to explain why a merger could be good for competing companies’ stock prices and also good for consumers. I think the simpler story is much more plausible: Wall Street is betting the merger will be bad for consumers.

Maybe none of this should be surprising. There were other honest signals earlier on in the approval process. As far as I can tell, neither Verizon nor AT&T seriously resisted the merger:2

Disclosure: At the time of writing, I have financial relationships with a bunch of telecommunications companies, including all of the major U.S. network operators except T-Mobile.

Abstract photo representing wireless technology

New RootMetrics Report – Verizon Wins Again

Christian

Yesterday, RootMetrics released its report on mobile network performance in the first half of 2019. Here are the overall, national scores for each network:1

  • Verizon – 94.8 points
  • AT&T – 93.2 points
  • T-Mobile – 86.9 points
  • Sprint – 86.7 points

While Verizon was the overall winner, AT&T wasn’t too far behind. T-Mobile came in a distant third with Sprint just behind it.

RootMetrics also reports which carriers scored the best on each of its metrics within individual metro areas. Here’s how many metro area awards each carrier won along with the change in the number of rewards received since the last report:2

  • Verizon – 672 awards (+5)
  • AT&T – 380 (+31)
  • T-Mobile – 237 (-86)
  • Sprint – 89 (+9)

My thoughts

Overall this report wasn’t too surprising since the overall results were so similar to those from the previous report. The decline in the number of metro area awards T-Mobile won is large, but I’m not sure I should take the change too seriously. There may have been a big change in T-Mobile’s quality relative to other networks, but I think it’s also possible the change can be explained by noise or a change in methodology. In its report, RootMetrics notes the following:3

T-Mobile’s performance didn’t necessarily get worse. Rather, AT&T, Sprint, and Verizon each made award gains in the test period, which corresponded with T-Mobile’s decreased award count.

I continue to believe RootMetrics’ data collection methodology is far better than Opensignal’s methodology for assessing networks at the national level. I take this latest set of results more seriously than I take the Opensignal results I discussed yesterday. That said, I continue to be worried about a lack of transparency in how RootMetrics aggregates its underlying data to arrive at final results. Doing that aggregation well is hard.

A final note for RootMetrics:
PLEASE DISCLOSE FINANCIAL RELATIONSHIPS WITH COMPANIES YOU EVALUATE!

Is Google Fi Worth It?

Christian

Google Fi uses an admirably simple pricing structure. A base rate of $20 per month offers subscribers unlimited talk and text. Beyond that, users are charged $10 per gigabyte of data. Single-line plans are capped at a monthly charge of $80, so subscribers that use 6GB of data will pay the same monthly price as subscribers that use 10GB of data.1 While I like the simplicity of the pricing structure, plans end up being fairly expensive. It’s my impression that Google Fi has had its current pricing structure in place for several years despite the cost per byte of data dropping in the industry at large.

Fi-enabled devices have technology that allows them to switch between T-Mobile, U.S. Cellular, and Sprint’s networks. While the technology is cool, I’m not sure I’d choose seamless switching between three networks with mediocre coverage over exclusive access to Verizon’s more reliable network.2

Fi now officially supports devices that are not Fi-enabled. When these devices are used with Fi, they’ll only have access to T-Mobile’s network. Many mobile virtual network operators use T-Mobile’s network and offer far better prices than Fi. For example, Mint Mobile’s plans blow Fi’s prices out of the water.3 Even with a Fi-enabled device, I think most people can find a better deal. A light user would pay $30 per month before taxes and fees for texts, talk, and 1GB of data on Fi’s network. You could get the same unlimited texting, unlimited talk, and 1GB of data with Verizon’s prepaid service for $30.4 RedPocket can offer those resources on any of the major networks for $19 per month.5

For heavy data users, the case against Fi is even clearer. Using 6+ gigabytes of data brings the Fi monthly bill to $80 before taxes and fees. At that cost, I expect you could purchase an unlimited, postpaid plan with any of the Big Four carriers.

Despite my negativity, I’m still a huge fan of Fi’s simplicity and remarkable international roaming policies. Hopeful Fi will revamp its prices in the near future to become more competitive with the other options on the market.

Magical Growth in Subscriber Numbers

Christian

Yesterday, one of my favorite journalists covering wireless, Mike Dano, published an article with the title “Why Wireless Carriers Magically Keep Growing Every Quarter.”

Dano notes that there’s been a roughly 2.5% growth in wireless subscribers for each of the past several quarters. This growth rate is tricky to make sense of:

[MoffettNathanson analysts] noted that the industry’s growth rate appears to be outstripping population growth rates and the growing number of teenagers getting phones, and isn’t attributable to other factors like the growing number of secondary phone-type devices like the Apple Watch. ‘The most likely answer appears to be the simplest,’ wrote the MoffettNathanson analysts. ‘Carriers are offering free or partially subsidized phones in return for adding additional lines.’

They continued: ‘It is all but certain that some customers have taken advantage of these offers even if it means adding a line they don’t need, and won’t use. The customer would simply reassign the new BOGO handset to an existing (used) line, moving an old unwanted handset to the new (unused) line.’

I’m not convinced this is the simplest explanation. A lot of consumers would find the process of adding a line to take advantage of a buy-one-get-one (BOGO) offer then switching service between devices complicated or sketchy. Around 2012, massive phone subsidies on post-paid plans were extremely common. At the time, I was involved in the cell phone resale business. I noticed that a surprising number of people were eligible for subsidized upgrades but not using them. In these scenarios, a subscriber could upgrade to a new ~$400 device for free, switch back to an old device, and quickly resell the new device. Even though the opportunity was relatively simple, I got the impression that people rarely took advantage of it.

The other problem I have with the explanation is that if consumers are taking advantage of BOGO offers in large numbers, carriers ought to notice what’s going on. Perhaps some carriers want to pad their subscriber numbers, but I find it unlikely that there’s an industry-wide willingness to pad subscriber numbers today since that will lead to higher churn in a year or two. I would guess that some carriers seeing consumers regularly add new lines to get free devices would be inclined to promote device-financing options. I expect financing options would often be simpler for consumers and more profitable for carriers.

That said, it’s pretty clear that at least some new lines come from those taking advantage of BOGO offers. A recent FCC filing stated the following (emphasis mine):

Sprint’s postpaid net additions recently have been driven by ‘free lines’ offered to Sprint customers and the inclusion of less valuable tablet and other non-phone devices, as well as pre to post migrations that do not represent ‘new’ Sprint customers.1
What I wonder is whether BOGO offers are the primary driver of unexpected growth. Dano mentions a claim found in a recent Wall Street Journal article (paywalled) that’s based on work from New Street Research:
Telecom consultant New Street Research estimated that customers signing unneeded wireless contracts to pocket more valuable smartphones added 1.7 million ‘fake’ lines to cellphone carriers’ tallies in 2018.
If we take that number at face value, that’s roughly 400,000 lines per quarter.2 However, BOGO offers are not new. They reached their peak several years ago. For “fake” BOGO lines to be driving growth, there must be more “fake” lines getting activated now than there are “fake” lines falling off.3 From my vantage point, it looks like BOGO offers might be less appealing than they were in the past. It used to be the case that devices that had been out for a few years were substantially worse than recently-released devices. That seems less true today. Recent declines in iPhone sales may indicate the other people feel the way I do.

What else might explain the large growth in subscriber numbers? On Twitter, industry-analyst Roger Entner mentioned that the growth could be due to subscribers transferring off of the Lifeline subsidization program.

It’s an interesting puzzle, and I might just be missing something. Despite my skepticism, I still don’t think it’s implausible that BOGO promotions really are driving lots of growth in subscriber numbers.