Blog

Photo of a frustrated person with a broken phone

Consumer Reports’ Broken Cell Service Rankings

Christian

Several months ago, I published a blog post arguing that Consumer Reports’ cell phone rankings were broken. This month, Consumer Reports updated those rankings with data from another round of surveying its subscribers. The rankings are still broken.

Consumer Reports slightly changed its approach this round. While Consumer Reports used to share results on 7 metrics, it now uses 5 metrics:

  1. Value
  2. Customer support
  3. Data
  4. Reception
  5. Telemarketing call frequency

Of the 19 carriers Consumer Reports’ assesses, only 5 operate their own network hardware.1 The other 14 carriers resell access to other companies’ networks while maintaining their own customer support teams and retail presences.2

Several of the carriers that don’t run their own network offer service over only one host network:

  • Cricket Wireless – AT&T’s network
  • Page Plus Cellular – Verizon’s network
  • MetroPCS – T-Mobile’s network
  • CREDO Mobile – Verizon’s network
  • Boost Mobile – Sprint’s network
  • GreatCall – Verizon’s network
  • Virgin Mobile – Sprint’s network

To test the validity of Consumer Reports’ methodology, we can compare scores on metrics assessing network quality between each of these carriers and their host network. At first glance, it looks like the reception and data metrics should both be exclusively about network quality. However, the scores for data account for value as well as quality:3

Data service indicates overall experience (e.g., cost, speed, reliability) with the data service.
I think it was a methodological mistake to account for value within the data metric then account for value again in the value metric. That leaves us with only the reception scores.4 Here are the scores the four host operators get for reception:

  • Verizon – Good
  • T-Mobile – Fair
  • AT&T – Poor
  • Sprint – Poor

How do those companies’ scores compare to scores earned by carriers that piggyback on their networks?

  • Cricket Wireless has good reception while AT&T has poor reception.
  • Page Plus and Verizon both have good reception.
  • MetroPCS has good reception while T-Mobile has fair reception.
  • CREDO and Verizon both have good reception.
  • Boost has very good reception while Sprint has poor reception.
  • GreatCall and Verizon both have good reception.
  • Virgin has good reception while Sprint has poor reception.

In the majority of cases, carriers beat their host networks. The massive differences between Cricket/AT&T and Boost/Sprint are especially concerning. In no cases do host operators beat the carriers that piggyback on their networks. I would have expected the opposite outcome. Host networks generally give higher priority to their direct subscribers when networks are busy.

The rankings are broken.

What’s the problem?

I see two especially plausible explanations for why the survey results aren’t valid for comparison purposes:

  • Non-independent scoring – Respondents may take prices into account when assessing metrics other than value. If that happens, scores won’t be valid for comparisons across carriers.
  • Selection bias – Respondents were not randomly selected to try certain carriers. Accordingly, respondents who use a given carrier probably differ systematically from respondents that use another carrier. Differences in scores between two carriers could reflect either (a) genuine differences in service quality or (b) differences in the type of people who use each service.

Consumer Reports, please do better!

My earlier blog post about Consumer Reports’ methodology is one of the most popular articles I’ve written. I’m nearly certain staff at Consumer Reports have read it. I’ve tried to reach out to Consumer Reports through two different channels. First, I was ignored. Later, I got a response indicating that an editor might reach out to me. So far, that hasn’t happened.

I see three reasonable ways for Consumer Reports’ to respond to the issues I’ve raised:

  • Adjust the survey methodology.
  • Cease ranking cell phone carriers.
  • Continue with the existing methodology, but mention its serious problems prominently when discussing results.

Continuing to publishing rankings based on a broken methodology without disclosing problems is irresponsible.

Markets Are Honest

Christian

I’ve been reading a ton of articles with commentators’ takes on whether a merger between Sprint and T-Mobile will be good or bad for consumers. Almost everything I’ve read has taken a strong position one way or the other. I don’t think I’ve seen a single article that expressed substantial uncertainty about whether a merger would be good or bad.

It could be that everyone is hugely biased on both sides of the argument. Or maybe the deal is so bad that only incredibly biased people would consider making an argument that the merger will be good for consumers. I’m not sure.

I like to look at how markets handle situations I’m uncertain about. In the last few years, I’ve regularly seen liberal politicians and liberal news agencies arguing that we’re about to see the end of Trump’s presidency because of some supposedly impeachable action that just came to light. I’m not Trump’s biggest fan, but I’ve found a lot of arguments about how he’s about to be impeached too far-fetched. I have a habit of going to the political betting market PredictIt when I see new arguments of this sort. PredictIt has markets on lots of topics, including whether or not Trump will be impeached.

Politicians and newspapers have an incentive to say things that will generate attention. A lot of the time, doing what gets attention is at odds with saying what’s true. People putting money in markets have incentives that are better aligned with truth.

Most of the time I’ve seen articles about Trump’s impending impeachment, political betting markets haven’t moved much. In rare occasions where markets moved significantly, I’ve had a good indication that something major actually happened.

Wall Street investors have a strong incentive to understand how the merger will actually affect network operators’ success. Unsurprisingly, T-Mobile’s stock increased substantially when key information indicating likely approval of a merger came out. Sprint’s stock also increased in value.

What’s much weirder is that neither Verizon’s stock nor AT&T’s stock seemed to take a negative hit on the days when important information about the merger’s likelihood came out. In fact, it actually looks like the stocks may have increased slightly in value.1

You could tell complicated stories to explain why a merger could be good for competing companies’ stock prices and also good for consumers. I think the simpler story is much more plausible: Wall Street is betting the merger will be bad for consumers.

Maybe none of this should be surprising. There were other honest signals earlier on in the approval process. As far as I can tell, neither Verizon nor AT&T seriously resisted the merger:2

Disclosure: At the time of writing, I have financial relationships with a bunch of telecommunications companies, including all of the major U.S. network operators except T-Mobile.

Motorola G7 Play Box

Moto G7 Play – The Ultimate Budget Phone

Christian

Earlier this year, Motorola released its G7 Play. Despite a full price of only $199.99, it’s an amazing phone.

If you’re interested in diving deeply into the phone’s technical specs, I recommend Digital Trends’ review. The phone’s hardware isn’t as impressive as what’s in today’s $1,000 flagship devices, but the phone still packs plenty of power. I haven’t had any trouble with the G7 Play’s performance in a month or two of use. Even the battery life is good. I expect the majority of smartphone users would be highly satisfied with the phone’s performance. That said, those looking for optimal performance on high-end mobile games or the best camera possible should consider other devices.

The G7 Play is one of a limited number of phones that qualifies to be on my list of universal unlocked phones. When the phone is purchased directly from Motorola, it should have the radio hardware and whitelisting necessary for compatibility and solid performance on all four major networks in the U.S. Versions of the G7 Play purchased from carriers and third-party retailers may have less extensive compatibility than devices purchased directly from Motorola.

The phone runs Android 9 and can do nearly everything I expect higher-end Android phones to be capable of. It even has a handful of clever features Motorola added—e.g., I’ve enjoyed the convenience of being able to toggle the flashlight on and off by shaking the phone side to side.

Here are the most meaningful negative aspects of the phone I can come up with:

  • There’s a notch on the front of the phone that houses a camera and a microphone. The phone would be more aesthetically appealing without the notch.
  • The camera isn’t as good as many of the cameras found on high-end devices.
  • NFC is not supported.
  • The phone only has 2GB of RAM. This may limit the phone’s performance when multitasking, but I haven’t had any problems yet.

These limitations don’t really bother me. I don’t think they’ll bother most other people either.

Motorola offers two other models in the same series of phones that cost slightly more but come with more powerful hardware: the G7 Power and G7.

Verizon Pushes Back Deadline For 3G Retirement

Christian

Verizon has updated a web page about the company’s plans for retiring its 3G network. Previously, the web page indicated that (a) Verizon planned to retire its 3G network by the end of 2019 and (b) Verizon would no longer activate devices that were CDMA-only or did not support HD Voice:

Verizon Wireless is retiring its CDMA (3G) network at the end of 2019. As a result, we are no longer allowing activation of CDMA-only devices, including CDMA-only basic phones and smartphones, or 4G LTE smartphones that do not support HD Voice service.

The updated web page suggests Verizon plans to keep its 3G network available to customers until the end of 2020. It also looks like some CDMA-only phones and phones without HD Voice may be eligible for activation until the end of the year:

Starting January 1, 2020, Verizon will no longer allow any CDMA (3G and 4G Non-HD Voice) ‘Like-for-Like’ device changes.

The page also indicates that bringing your own CDMA device to activate on an existing line will be prohibited starting 1/1/2020.

As networks’ change their deadlines, I plan to update my earlier blog post covering each major networks’ plans for phasing out 3G networks.

Location, Location, Location

Christian

In my opinion, major wireless networks can be ranked pretty clearly in terms of their current, nationwide reliability:

  1. Verizon (best)
  2. AT&T
  3. T-Mobile
  4. Sprint (worst)

I get frustrated when network operators make misleading statements about nationwide quality, and I sometimes write articles calling out bullshit claims. That said, a network’s typical reliability throughout the U.S. may be very different from that network’s quality in a given area. When deciding which carrier you should use, it only matters how carriers perform where you want to use your phone.

In the last year, I’ve run speed tests in Boulder, Colorado with a bunch of carriers (using all four of the major U.S. networks). A few days ago, I ran a speed test on a phone with service from Tello, a carrier that runs over Sprint’s network. While Sprint has the worst nationwide network, the speed test found a download speed far faster than I’ve seen in Boulder with any other carrier:

129 Mbps speed test result

As a general rule, service is more expensive on networks with better nationwide performance. If you live where an underdog network performs well, you might be able to get great service at a bargain price.

Sprint – Now Offering Nationwide 5G!??

Christian

Today I was looking at Sprint’s coverage map. By default, the map appears to be displaying Sprint’s coverage profile for 5G data:

Sprint barely has any 5G coverage, so the map surprised me.

If you change the selection in the dropdown menu, you’ll see that the area shaded for “Data coverage” is identical whether the 4G or the 5G option is selected. However, the shaded area changes when “Non-LTE” is selected on the dropdown. I think Sprint may have made an honest mistake, but it has the potential to confuse consumers.

As you scroll in on specific areas, the “Data coverage” entry in the legend disappears and more finely grained categories appear:

It’s odd that users can select specific types of coverage but still see a map that differentiates between multiple types of coverage.

We know AT&T is willing to mislead its customers into believing their 4G service is 5G. At the moment, I’m going to give Sprint the benefit of the doubt. After all, the legend doesn’t even appear until a user toggles its visibility.

DOJ Clears T-Mobile’s Merger With Sprint

Christian

As expected, the Department of Justice made an announcement today approving a merger between Sprint and T-Mobile. While the merger isn’t officially closed, DOJ approval was the largest hurdle T-Mobile and Sprint needed to jump before making their merger a reality.

As far as I can tell, the terms of the merger were consistent with what most commentators were expecting:

  • Most of Sprint’s prepaid business will be divested to DISH1
  • DISH will get Sprint’s 800 MHz spectrum
  • DISH will receive access to the New T-Mobile’s network for at least 7 years2
  • DISH will have the option to take over leases on some retail stores and cell sites

I don’t think mergers between telecom companies have a good track record of benefiting consumers. I hope this merger will be different, but I’m not betting on it. As many others have pointed out, something is odd about the whole arrangement. The divestitures to DISH are ostensibly intended to allow DISH to create a viable, facilities-based carrier (i.e., a carrier that has its own hardware and doesn’t just piggyback off other companies’ networks). If DISH is likely to succeed, it’s hard to explain why Sprint couldn’t remain a viable force. Maybe I’m misunderstanding something important.

I expect the merger-related transitions to take a few years, and I plan to write about new developments as they occur. Should be interesting.

For those interested, here are a few excerpts from T-Mobile’s announcement:

The proposed New T-Mobile, will divest Sprint’s prepaid businesses and Sprint’s 800 MHz spectrum assets to DISH. Additionally, upon the closing of the divestiture transaction, the companies will provide DISH wireless customers access to the New T-Mobile network for seven years and offer standard transition services arrangements to DISH during a transition period of up to three years. DISH will also have an option to take on leases for certain cell sites and retail locations that are decommissioned by the New T-Mobile, subject to any assignment restrictions.
The New T-Mobile will be committed to divest Sprint’s entire prepaid businesses including Boost Mobile, Virgin Mobile and Sprint-branded prepaid customers (excluding the Assurance brand Lifeline customers and the prepaid wireless customers of Shenandoah Telecommunications Company and Swiftel Communications, Inc.), to DISH for approximately $1.4 billion. These brands serve approximately 9.3 million customers in total.
With this agreement, Boost Mobile, Virgin Mobile, and Sprint-branded prepaid customers, as well as new DISH wireless customers, will have full access to the legacy Sprint network and the New T-Mobile network in a phased approach. Access to the New T-Mobile network will be through an MVNO arrangement, as well as through an Infrastructure MNO arrangement enabling roaming in certain areas until DISH’s 5G network is built out.
The companies have also committed to engage in good faith negotiations regarding the leasing of some or all of DISH’s 600 MHz spectrum to T-Mobile.
Evolution of cell phones

Phones & Keeping Up With The Joneses

Christian

People buy a lot of shit they don’t need to impress other people. The phrase “keeping up with the Joneses” often has a negative connotation. I’m not sure that’s entirely fair.

People care a lot about their social status. Status is helpful for everything from getting jobs to finding romantic partners. Like it or not, buying fancy things can improve people’s social status.

Products do a better job signaling social status when they are conspicuous. Product designers and marketers know this. You don’t see sports cars with fancy engines and subtle, Honda Civic-like exteriors. Sports cars are flashy.

Electric cars tend to look like vehicles Martians might drive. There’s no engineering reason why electric cars need to look goofy. However, carmakers know that electric car owners want other people to know which cars are electric. Unique aesthetics send signals.

Recently, there’s been indications that high-end phones aren’t selling as well as they used to. I’ve seen a lot of plausible explanations: innovations have been limited, cheap phones are awfully good these days, and carriers don’t subsidize devices the way they used to. I want to throw out another possibility: fancy phones are way less conspicuous than they used to be.

The first time one of my friends got a cell phone, I was in fifth grade. At that time, just having a cell phone was cool. But my friend didn’t just have a phone. You see, his phone could flip.

Flip phone photo

Even from a distance, you could tell my buddy’s phone wasn’t just any old phone. It was a flip phone.

When flip phones advanced, the fancier ones tended to look cooler. Remember the Razr?

For the next several years, top-tier phones continued to have unique aesthetics. In 2007, the first iPhone was released. At the time, you knew an iPhone when you saw one. Only the iPhone had a screen almost as large as the phone itself.1

A few iPhone generations later, Apple managed to keep its iPhone 4 conspicuous with a sleeker appearance than earlier models.

In the last few years, companies have run out of ways to keep fancy phones conspicuous. It seems like the goal has been to develop phones that (a) are thin and (b) have as much of the body devoted to screen space as possible. Almost every phone these days is rectangular, sleek, and almost all screen. The Motorola G6 Play is a budget phone. It’s still thin, sleek, and mostly covered by a screen:

g6 play

It used to be relatively easy to tell what phone someone was using just by glancing at it. Now that most phones look similar, that’s much harder.

Abstract photo representing wireless technology

New RootMetrics Report – Verizon Wins Again

Christian

Yesterday, RootMetrics released its report on mobile network performance in the first half of 2019. Here are the overall, national scores for each network:1

  • Verizon – 94.8 points
  • AT&T – 93.2 points
  • T-Mobile – 86.9 points
  • Sprint – 86.7 points

While Verizon was the overall winner, AT&T wasn’t too far behind. T-Mobile came in a distant third with Sprint just behind it.

RootMetrics also reports which carriers scored the best on each of its metrics within individual metro areas. Here’s how many metro area awards each carrier won along with the change in the number of rewards received since the last report:2

  • Verizon – 672 awards (+5)
  • AT&T – 380 (+31)
  • T-Mobile – 237 (-86)
  • Sprint – 89 (+9)

My thoughts

Overall this report wasn’t too surprising since the overall results were so similar to those from the previous report. The decline in the number of metro area awards T-Mobile won is large, but I’m not sure I should take the change too seriously. There may have been a big change in T-Mobile’s quality relative to other networks, but I think it’s also possible the change can be explained by noise or a change in methodology. In its report, RootMetrics notes the following:3

T-Mobile’s performance didn’t necessarily get worse. Rather, AT&T, Sprint, and Verizon each made award gains in the test period, which corresponded with T-Mobile’s decreased award count.

I continue to believe RootMetrics’ data collection methodology is far better than Opensignal’s methodology for assessing networks at the national level. I take this latest set of results more seriously than I take the Opensignal results I discussed yesterday. That said, I continue to be worried about a lack of transparency in how RootMetrics aggregates its underlying data to arrive at final results. Doing that aggregation well is hard.

A final note for RootMetrics:
PLEASE DISCLOSE FINANCIAL RELATIONSHIPS WITH COMPANIES YOU EVALUATE!

Woman making a skeptical face

Opensignal Released a New Report – I’m Skeptical

Christian

Opensignal just released a new report on the performance of U.S. wireless networks. The report ranks major U.S. networks in five categories based on crowdsourced data:

  • 4G availability
  • Video experience
  • Download speed experience
  • Upload speed experience
  • Latency experience

Verizon took the top spot for 4G availability and video experience. T-Mobile came out on top for both of the speed metrics. T-Mobile and AT&T shared the top placement for the latency experience metric.

Selection bias

I’ve previously raised concerns about selection bias in Opensignal’s data collection methodology. Opensignal crowdsources data from typical users. Crowdsourcing introduces issues since there are systematic differences between the typical users of different networks. Imagine that Network A has far more extensive coverage in rural areas than Network B. It stands to reason that Network A likely has more subscribers in rural areas than Network B. Lots of attributes of subscribers vary in similar ways between networks. E.g., expensive networks likely have subscribers that are wealthier.

Analyses of crowdsourced data can capture both (a) genuine differences in network performance and (b) differences in how subscribers on each network use their devices. Opensignal’s national results shouldn’t be taken too seriously unless Opensignal can make a compelling argument that either (a) its methodology doesn’t lead to serious selection bias or (b) it’s able to adequately adjust for the bias.

Speed metrics

Opensignal ranks carriers based on average download and upload speeds. In my opinion, average speeds are overrated. The portion of time where speeds are good enough is much more important than the average speed a service offers.

Opensignal’s average download speed results are awfully similar between carriers:

  • Verizon – 22.9 Mbps
  • T-Mobile – 23.6 Mbps
  • AT&T – 22.5 Mbps
  • Sprint – 19.2 Mbps

Service at any of those speeds would be sufficient for almost any activities people typically use their phones for. Without information about how often speeds were especially low on each network, it’s hard to come to conclusions about differences in the actual experience on each network.