Blog

RootMetrics’ Report For Late 2019

Yesterday, RootMetrics released its latest report on the performance of U.S. wireless networks. I’d been looking forward to this report. RootMetrics’ drive testing methodology has some advantages over the approaches used by other companies that evaluate network performance.

Results

RootMetrics’ results were generally unsurprising. As with the last report, Verizon was the big winner, followed by AT&T in second place, T-Mobile in third, and Sprint in fourth.

Here are the overall, national scores out of 100 for each of the major networks:

  • Verizon – 94.6 points
  • AT&T – 93.2 points
  • T-Mobile – 86.5 points
  • Sprint – 83.2 points

RootMetrics also reports which carriers scored the best on each of its metrics within individual metro areas. Here’s how many metro area awards each carrier won (along with the change in the number of rewards received since the last report):

  • Verizon – 660 awards (-12)
  • AT&T – 401 awards (+21)
  • T-Mobile – 217 awards (-20)
  • Sprint – 80 awards (-9)

AT&T’s improvements

RootMetrics’ results align with the results of other recent evaluations suggesting aspects of AT&T’s network are becoming more competitive. AT&T fared particularly well in RootMetrics’ latest speed metrics. While Verizon narrowly beat AT&T in the final speed score out of 100 (90.7/100 for Verizon vs. 90.2/100 for AT&T), AT&T narrowly beat Verizon in aggregate median download speed (33.1 Mbps for AT&T vs. 32.7 Mbps for Verizon).

It appears that RootMetrics’ final speed scores are based on something more than median download speed. That may be a good thing: having consistent speeds is arguably much more important than having high average or median speeds. Still, I’m frustrated that I can’t figure out exactly how the final speed scores are derived. RootMetrics continues to be non-transparent about the math underlying its analyses.

A section of the latest report suggests that Verizon may do a particularly good job of avoiding sluggish speeds:

Verizon’s ‘slowest’ median download speed of 17.9 Mbps, recorded in Fresno, CA, was still quite strong and would allow end users to complete the majority of data tasks with ease. In fact, Fresno was the only market in which Verizon registered a median download speed below 20 Mbps. No other carrier came close to matching Verizon’s consistency of delivering fast speeds in metros across the US.

5G performance

The new report includes details about RootMetrics’ recent tests on 5G networks. I found the 5G results unsurprising, and I’m not going to comment on them further at this time. I think 5G deployments are still in too early a stage for the results to be of much interest.

Link Roundup – 1/25/20

This post is the first of hopefully many link roundup posts.

  • Masnick of TechDirt explains how Apple dropped plans to encrypt its customers’ backups after the FBI pushed back on the company’s plans. It seems like there’s a lot of misinformation going on in this situation.
  • Physicist Casey Handmer is extremely optimistic about SpaceX’s Starlink. If you’re not already familiar, Starlink is a satellite constellation SpaceX is working on to provide global internet connectivity. At the moment, the company plans to have over 10,000 satellites deployed in the constellation.
  • Tim Farrar is very critical of Handmer’s post. In my opinion, Farrar is way too negative on Starlink, but many of his criticisms seem correct.
  • The U.K. government may place some limitations on Huawei, but it looks like the company may have an easier time in the U.K. than in the U.S.
  • There seems to be ongoing confusion about how TracFone’s SmartSIM product will work.

MVNOs Hiding Their Host Operators

Mobile virtual network operators (MVNOs) often appear to be prohibited (or at least discouraged) from explicitly acknowledging which networks they run over. Still, it seems that most MVNOs don’t have to keep their host networks entirely secret. The effects of these policies can be kind of funny.

Red Pocket

The MVNO Red Pocket operates over all of the major U.S. networks, but it only mentions Sprint by name. The following screenshot comes from a part of Red Pocket’s website that lists the networks the company offers service over:



The unnamed networks are color-coded to match the colors used in the host networks’ branding: AT&T in blue, T-Mobile in pink, and Verizon in red.

Ting

The MVNO Ting currently operates over Sprint and T-Mobile’s networks. Verizon will be added to the list soon. Today, Ting published a blog post about the upcoming addition. Ting still isn’t naming the networks it works with, but the company is making easy-to-interpret statements like: “In 2020, Ting will be on every major coast-to-coast network except AT&T.” The blog post included a video where the names of each network the company works with were censored out:

Xfinity Mobile Growing Quickly

An SEC filing Comcast published today showed that Xfinity Mobile has been adding subscribers quickly. In the last quarter, Xfinity Mobile added 261,000 new lines.[1] Overall, Xfinity Mobile had a net increase of 816,000 lines in 2019, bringing its total subscriber count at the end of the year to slightly over two million lines.[2] It’s now fair to say that Xfinity Mobile is one of the largest MVNOs in the U.S.

By my math, Xfinity Mobile’s subscriber base grew by about 66% in 2019.[3] It will be interesting to see whether Comcast can keep that kind of growth rate going forward.

By one method of accounting, Xfinity Mobile looks unprofitable. However, the brand seems to be moving in the right direction towards profitability:[4]

Cable Communications results include a loss of $116 million from our wireless business, compared to a loss of $191 million in the prior period.
One could argue that way of looking at the financials leaves out something important. Subscribing to Xfinity Mobile makes it harder for a Comcast customer to cancel their internet service. Looking exclusively at the balance sheet of Comcast’s wireless segment leaves out the benefits Comcast receives as Xfinity Mobile reduces churn in the company’s other product segments.

At the moment, Xfinity Mobile presents a pretty good value proposition for current Xfinity Internet customers. Going forward, I’m not sure whether Comcast will keep Xfinity Mobile’s prices competitive to encourage growth or increase prices to make each subscriber more profitable.

Opensignal’s 2020 U.S. Mobile Performance Report

Today, Opensignal released a new report on the performance of U.S. wireless networks. The report details results on seven different metrics.

Here are the networks that took the top spot for each metric at the national level:

  • Video Experience – Verizon
  • Voice App Experience – T-Mobile/AT&T (draw)
  • Download Speed Experience – AT&T
  • Upload Speed Experience – T-Mobile
  • Latency Experience – AT&T
  • 4G Availability – Verizon
  • 4G Coverage Experience – Verizon

It’s important to interpret these results cautiously due to limitations in Opensignal’s crowdsourcing approach. Since performance data is collected from actual subscribers’ devices, factors not directly related to underlying network quality may impact the organization’s results. For example, if subscribers on a network are unusually likely to use low-end devices or live in rural areas, that will affect the outcome of Opensignal’s analyses. Still, Opensignal’s results are interesting; they’re drawn from a huge data set involving primarily automated performance tests.

Download speed findings

The most notable result in the latest report might be AT&T’s first-place finish on the download speed metric. In the previous Opensignal report, T-Mobile won first place for download speeds, and AT&T took third place. I’ve recently been critical of the methodologies used in some other evaluations that suggested AT&T had the nation’s fastest network. While many of those methodological criticisms still stand, the fact that Opensignal’s arguably more reliable methodology also found AT&T to have the fastest network leads me to believe I was too harsh. I’ll be interested to see whether AT&T also takes the top spot for speeds in RootMetrics’ upcoming report.

New metrics

Two new metrics were introduced in this report: Voice App Experience and 4G Coverage Experience. The Voice App Experience metric assesses the quality of voice calls via apps like Skype and Facebook Messenger. I’m not exactly sure how the metric works, but it looks like all four networks received similar scores. Opensignal deemed all these scores as indicative of “acceptable” quality.

The 4G Coverage Experience metric adds a bit of complexity to the previously existing 4G Availability metric. The coverage metric assesses 4G availability across areas all Opensignal’s users find themselves in, regardless of their network.

AT&T’s Misuse Of Ookla’s Speedtest Data

Earlier this month, AT&T published a news release claiming: “AT&T is the ‘Nation’s Fastest Wireless Network’ in 2019”. The claim is based on data collected by Ookla, the company behind Speedtest.net.

In the piece, AT&T writes:

With over 10 million consumer-initiated tests taken daily with Speedtest, Ookla provides invaluable insight into the performance, quality and accessibility of networks worldwide.

Ookla’s reliance on consumer-initiated tests has serious downsides. For example, consumers on different networks use different kinds of phones. Premium phones tend to have hardware that supports faster connections than cheap phones. If subscribers on Network A tend to run tests using premium phones while subscribers on Network B tend to run tests from budget phones, Network A is going to have an advantage in consumer-initiated tests that’s unrelated to underlying network performance.

Ookla’s methodology is also prone to selection bias. Consumer-initiated tests don’t occur among a randomly selected sample of subscribers on each network. Consequently, test results likely aren’t representative of typical network performance. There was a clear case of this problem when AT&T began misleadingly labeling some of its 4G service “5GE.” Here’s an excerpt from a Speedtest.net blog post:[1]

In the final week of Q1, we also observed an increase in faster tests taken on AT&T’s network. Upon investigation, we discovered that this correlated with the release of iOS 12.2 and the roll out of AT&T’s 5G E icon. We also found that the increase in tests was coming from device models that would have started to display the 5G E icon, such as the newer generations of iPhone (XR, XS Max, XS, X, 8, 8 Plus), indicating that consumers were seeing the new icon and taking a test to see what speeds they were getting.

While no method for evaluating mobile network performance is perfect, I tend to think Opensignal and RootMetrics use methodologies that are more reliable than Ookla’s. AT&T didn’t take the top spot for speeds in RootMetrics’ most recent report or Opensignal’s most recent report.

Silly criticisms

AT&T’s news release includes ridiculous digs at competitors:

Speedtest results show we increased our speeds by 45.7% year-over-year, while one of our competitors never ‘checked the box’ on speed with only a 16.5% increase year over year and the other defined what it meant to be ‘Un-speedy’ with only a 9.4% increase year over year.

Year-over-year changes in average speeds don’t on their own indicate whether networks are fast. A visual in the news release is illuminating:

Chart

AT&T performed poorly relative to T-Mobile and Verizon in early 2018. In a sense, AT&T was able to get a 45.7% year-over-year increase in speeds because it performed so poorly in 2018.

Consistency

I’d argue that AT&T’s news release ignores the most important part of Speedtest’s 2019 report. In my opinion, average speed is overrated. For most consumers, it’s far more important to consistently have decent speeds than to have high average speeds.

Ookla reports a consistency metric based on the proportion of tests that exceed a threshold of 5Mbps. Verizon takes the top spot on this metric, followed by T-Mobile, with AT&T coming in third. Verizon also beats out AT&T for coverage availability, another metric that can act as a proxy for consistency.[2]

Bias against Verizon

In Ookla’s main analyses, data is only included from “competitive geographies.” Competitive geographies only include areas where Ookla has a substantial number of test results from at least three major networks.[3] There are defensible reasons for Ookla to use the competitive geographies filter. However, it should be acknowledged that Verizon has the nation’s most extensive network and likely outperforms AT&T and other networks in non-competitive geographies.[4]

SIM Cards

TracFone Teases SmartSIM

TracFone, the company behind several large MVNOs, appears to be working on a new product called SmartSIM. Apparently, some TracFone customers recently received a marketing email that mentioned SmartSIM.[1] I was briefly able to access NoDeadZone.com, a website that shared some basic information about SmartSIM. Oddly, the website now automatically redirects to locations.totalwireless.com (Total Wireless is a brand owned by TracFone).

While NoDeadZone.com was accessible, it offered a short video explaining SmartSIM. Apparently, the technology will allow subscribers to switch rapidly between multiple networks based on which network offers the best signal. There’s been some speculation about how up-and-coming eSim technology may enable more people to take advantage of dynamic network switching of this sort. However, the video I gave me the impression that SmartSIM would involve a conventional, removable SIM card rather than an eSIM. At the moment, I’m unsure if TracFone is licensing switching technology Google Fi built, introducing new technology, or something else.

NoDeadZones.com allowed visitors to enter their zip codes to see if SmartSIM was available where they lived. I tried several zip codes, and all were ineligible. It seems that other people had the same experience. I’m not sure whether any zip codes were really eligible for the service.

So far, I haven’t heard of TracFone responding to any requests for more information about SmartSIM. I’m curious about the lack of communication along with the decision to redirect NoDeadZone.com to the main Total Wireless site. It’s enough to make me wonder whether a mistake was made that led SmartSIM to become public knowledge before TracFone intended.

I’ll be keeping close tabs on how the story develops. Dynamic network switching has the potential to improve wireless service and change how it’s priced. With switching technology, it may be possible to charge different rates to different subscribers based on factors like a subscriber’s location, the extent of network congestion, or the quality of service a subscriber receives. Dynamic pricing could potentially lead to far more efficient network usage than conventional pricing—which might ultimately lead to a decrease in how much consumers pay for wireless service.

Infinity symbol

Tello Launches “Unlimited Everything” Plan

Today, Tello launched its “unlimited everything” plan for $39 per month. I’m frustrated by how Tello named its new plan. I say that as a fan of the company; Tello has some of the best options on the market for budget-sensitive consumers who don’t use a lot of data.

If subscribers on Tello’s unlimited everything plan use 25GB of data in a billing period, they will be throttled to sluggish, 2G speeds. As I’ve previously argued, unlimited plans at 2G speeds are bogus. Once the throttle kicks in, subscribers will find that data is unusable or barely usable for many purposes. While I’ve argued that many so-called “unlimited” plans are misnamed, Tello seems to have doubled down on its misnomer. Both “unlimited” and “everything” do a poor job of describing Tello’s new plan.

When carriers throttle data to 2G speeds, that usually means speeds are capped at 128Kbps. Imposing a maximum speed of 128Kbps puts a theoretical limit on total data use of about 65GB per month.[1] In practice, few subscribers will use more than 26GB per month because the internet will be sluggish and frustrating use after the 25GB threshold is reached.

To Tello’s credit, the company does an unusually good job of disclosing the throttle. Here’s an example from the banner on Tello’s homepage this morning:

Tello Homepage Banner

Virgin Mobile USA Is Shutting Down: Subscribers To Be Transferred To Boost Mobile

Virgin Mobile USA, a flanker brand owned by Sprint, is shutting down. Today, reports surfaced from many Virgin Mobile customers who received texts that started like this:

Virgin Mobile USA is discontinuing and your account will auto-transfer to Boost Mobile.

Boost Mobile is another flanker brand owned by Sprint, and it should offer almost all customers service that is quite similar to the service Virgin Mobile USA has been offering.

Interestingly, I can’t find a press release from Virgin Mobile about the shutdown. However, the company has published a web page with an FAQ about the upcoming changes. The page illuminates several details about how the automated transfer of subscribers from Virgin to Boost will be handled:

  • Subscribers will keep their existing phone numbers.
  • Subscribers should be able to keep their current phones.
  • The transfers will begin in February.
  • Boost Mobile will not accept Paypal. Subscribers that paid for Virgin service with PayPal will need to choose a new payment method.
  • Payment dates will usually be unaffected by the transfer from Virgin to Boost.
  • Subscribers automatically paying for Virgin service via credit card or debit card will have their payment methods transferred automatically.
  • Device insurance purchased from Virgin should carry over to Boost.
  • Devices using Virgin’s Mobile Broadband service will not automatically be transferred to Boost. Subscribers using these devices will need to find new carriers.[1]

Virgin Mobile USA suggests that most subscribers will receive pricing with Boost that is the same or better than existing pricing with Virgin:

In most instances, your existing account will be transferred to Boost Mobile with your device, and a comparable or better Boost Mobile service plan at no extra cost to you… In fact, since Boost Mobile accounts have taxes and fees included, customers will end up paying less than you do now on similar plans.

While Virgin Mobile USA is suggesting customers will not face increased prices, I suggest that subscribers pay attention to any changes in their bills and plans over time. While I expect most subscribers will not be affected adversely in the short-term, it doesn’t look like Boost has committed not to raising prices or forcing plan changes in the future. I’d be especially vigilant if you currently have a grandfathered plan that Virgin no longer offers to new customers.

In most cases, I think Virgin subscribers should anticipate a smooth transition to Boost. Still, the transition may present a good moment for subscribers to consider other options on the market. Mint Mobile, a relatively new, low-cost carrier, may offer many people better coverage and lower prices than Virgin or Boost.

Why is Virgin Mobile USA shutting down?

On the FAQ page about Virgin’s shut down, one of the questions listed is: “I have been a Virgin Mobile customer for a long time, why is my account being transferred to Boost Mobile?” Virgin responds to the question with a non-answer:

We appreciate your loyalty. To ensure that we offer the best service to our customers, we regularly examine our plans. At this time a decision to discontinue the Virgin Mobile USA service has been made. As we are committed to providing you with great service, we will transfer your account to our sister brand Boost Mobile.

While I’m unsure exactly what’s going on, I expect there’s an effort underway to consolidate Sprint’s flanker brands in advance of news about whether a merger between Sprint and T-Mobile will go through.

AT&T’s Claim To Being America’s Best Network

AT&T has been running an ad campaign with commercials where the company claims to offer the best network.

These commercials start with a funny skit that leads to the line, “just ok is not ok.” The commercials’ narrator then says something along the lines of: “AT&T is America’s best wireless network according to America’s biggest test.”

Here’s an example:



Alternate versions of the commercial involve ok babysitters, ok sushi, ok surgeons, and more.

AT&T bases its “best network” claim on the results of Global Wireless Solutions’s (GWS) 2018 tests. The claim is at odds with the results of many other companies’ evaluations and my own view.

The meaning of the word “best” is ambiguous, but I’d guess that a survey of professionals in the wireless industry would find that most people consider RootMetrics to be the best evaluation firm in the wireless industry. Verizon fared far better than AT&T in RootMetrics’s most recent evaluation.

It’s unclear to me what AT&T is claiming when it calls GWS’s test, “America’s biggest test.” Is it the biggest test in terms of miles driven, data points collected, area covered, or something else? GWS may have the biggest test according to one metric, but it’s not unambiguously the biggest test in the nation.