Proof & Network Evaluations

RootMetrics recently tweeted about how its latest analyses prove which networks are the fastest and most reliable:

RootMetrics' Tweet

I’m a big fan of RootMetrics, but the tweet annoyed me. There’s a ton of flexibility in how network evaluators can approach their work. Will performance data be crowdsourced from consumers or collected via in-house testing? How will data be cleaned and aggregated? What regions will be included? Etc.

Phrases like “fastest network” and “most reliable network” are ambiguous. Do you determine the fastest network based on average download speeds, median download speeds, or something else?

RootMetrics’ tweet is especially odd in light of their latest report. Depending on which speed metric you choose to look at, you could argue that either Verizon or AT&T has the fastest network. AT&T narrowly beats Verizon in terms of median download speed. Verizon narrowly beats AT&T in RootMetrics’ overall speed scores.1

RootMetrics’ Results In Dense City Centers

RootMetrics’ most recent report includes information about network speeds in city centers. Here’s how RootMetrics’ describes the tests:1

Capacity is critical for a good mobile experience in highly congested areas. To show you how the carriers performed in high-traffic areas of select cities, we measured each carrier’s median download speed outdoors in the dense urban cores of 13 major cities across the country.

The results are presented with the following graphic that lists the median download speed in megabits per second from the fastest and slowest network in each city:2

RootMetrics' median download speeds in city centers

Despite Sprint’s lousy performance in RootMetrics’ overall, national-level results, Sprint still managed to offer the fastest speeds in 3 of the 13 city centers RootMetrics considered. The results are consistent with a point I’ve tried to emphasize in the past: if you tend to stay in one area, you shouldn’t worry too much about national-level network performance.

Prioritization

I haven’t seen information about which plans RootMetrics’ test devices use on each network. In fact, I’m not entirely sure RootMetrics uses service plans that are available to regular consumers. My guess is that RootMetrics’ test devices have prioritization and quality of service levels at the high-end of what’s available to regular consumers. If my guess is correct, RootMetrics’ test devices have higher priority than many budget-friendly services that run over the Big Four networks. A few examples of those services:

  • AT&T: Cricket Wireless unlimited plans
  • T-Mobile: Metro by T-Mobile plans
  • Sprint: Mobile hotspot plans and Boost Mobile plans
  • Verizon: Verizon Prepaid plans, the Start Unlimited plan, and Visible plans

Subscribers using low-priority services are especially likely to experience slow speeds in congested areas. I’d be interested in seeing RootMetrics rerun its city-center tests with low-priority services.

Mint Mobile Offering 3 Months Free During The Superbowl – No Serious Catches

Mint Mobile is going to run a huge promotion this Sunday during the Superbowl. Once the game starts (around 3:15PM PST), new customers can go to mintmobile.com/free/ to get 3 months of service for free on Mint’s plan that includes unlimited minutes, unlimited texts, and 3GB of data per month. The promotion doesn’t appear to involve any serious catches. There’s no long-term commitment or device purchase required.

Those who take advantage of the promotion will need to give Mint a credit card number and temporarily pay a $0.01 charge. Here’s how Mint explains it:

This 3-month 3GB plan is 100% on us. There is just a $0.01 temporary payment verification to verify the credit card used is valid. And even that will be refunded within 5-7 business days.

It sounds like the promotion will last for something like 3-5 hours. Mint is suggesting that the promotion will end when the final whistle blows at the end of the Superbowl (though I won’t be shocked if the promotion stays available for a bit longer).

This promotion looks like an awesome deal for people who want to try Mint. I’ll be surprised if there end up being any unexpected catches or gotchas.

Terms

Here are some more details about the terms of the promotion.

  • Only one person per household may take advantage of the promotion.
  • Service must be activated within 45 days.
  • As usual, Mint will allow subscribers to port existing numbers or receive new numbers.
  • As with Mint’s other plans, if you use up your 3GB data allotment, you can continue using additional data at very slow speeds.
  • If you want more than 3GB of high-speed data, you can upgrade by paying the prorated difference between the 3GB plan’s price and the price of a plan with a larger data allowance.
  • After the 3 free months are over, you can purchase further service according to Mint’s standard plans and prices.

Reynolds’ tweet

Actor Ryan Reynolds, an owner of Mint Mobile, tweeted about the promotion earlier today:

Ryan Reynolds tweet about Mint Mobile's 3 months free promotion

RootMetrics’ Report For Late 2019

Yesterday, RootMetrics released its latest report on the performance of U.S. wireless networks. I’d been looking forward to this report. RootMetrics’ drive testing methodology has some advantages over the approaches used by other companies that evaluate network performance.

Results

RootMetrics’ results were generally unsurprising. As with the last report, Verizon was the big winner, followed by AT&T in second place, T-Mobile in third, and Sprint in fourth.

Here are the overall, national scores out of 100 for each of the major networks:

  • Verizon – 94.6 points
  • AT&T – 93.2 points
  • T-Mobile – 86.5 points
  • Sprint – 83.2 points

RootMetrics also reports which carriers scored the best on each of its metrics within individual metro areas. Here’s how many metro area awards each carrier won (along with the change in the number of rewards received since the last report):

  • Verizon – 660 awards (-12)
  • AT&T – 401 awards (+21)
  • T-Mobile – 217 awards (-20)
  • Sprint – 80 awards (-9)

AT&T’s improvements

RootMetrics’ results align with the results of other recent evaluations suggesting aspects of AT&T’s network are becoming more competitive. AT&T fared particularly well in RootMetrics’ latest speed metrics. While Verizon narrowly beat AT&T in the final speed score out of 100 (90.7/100 for Verizon vs. 90.2/100 for AT&T), AT&T narrowly beat Verizon in aggregate median download speed (33.1 Mbps for AT&T vs. 32.7 Mbps for Verizon).

It appears that RootMetrics’ final speed scores are based on something more than median download speed. That may be a good thing: having consistent speeds is arguably much more important than having high average or median speeds. Still, I’m frustrated that I can’t figure out exactly how the final speed scores are derived. RootMetrics continues to be non-transparent about the math underlying its analyses.

A section of the latest report suggests that Verizon may do a particularly good job of avoiding sluggish speeds:

Verizon’s ‘slowest’ median download speed of 17.9 Mbps, recorded in Fresno, CA, was still quite strong and would allow end users to complete the majority of data tasks with ease. In fact, Fresno was the only market in which Verizon registered a median download speed below 20 Mbps. No other carrier came close to matching Verizon’s consistency of delivering fast speeds in metros across the US.

5G performance

The new report includes details about RootMetrics’ recent tests on 5G networks. I found the 5G results unsurprising, and I’m not going to comment on them further at this time. I think 5G deployments are still in too early a stage for the results to be of much interest.

Link Roundup – 1/25/20

This post is the first of hopefully many link roundup posts.

  • Masnick of TechDirt explains how Apple dropped plans to encrypt its customers’ backups after the FBI pushed back on the company’s plans. It seems like there’s a lot of misinformation going on in this situation.
  • Physicist Casey Handmer is extremely optimistic about SpaceX’s Starlink. If you’re not already familiar, Starlink is a satellite constellation SpaceX is working on to provide global internet connectivity. At the moment, the company plans to have over 10,000 satellites deployed in the constellation.
  • Tim Farrar is very critical of Handmer’s post. In my opinion, Farrar is way too negative on Starlink, but many of his criticisms seem correct.
  • The U.K. government may place some limitations on Huawei, but it looks like the company may have an easier time in the U.K. than in the U.S.
  • There seems to be ongoing confusion about how TracFone’s SmartSIM product will work.

MVNOs Hiding Their Host Operators

Mobile virtual network operators (MVNOs) often appear to be prohibited (or at least discouraged) from explicitly acknowledging which networks they run over. Still, it seems that most MVNOs don’t have to keep their host networks entirely secret. The effects of these policies can be kind of funny.

Red Pocket

The MVNO Red Pocket operates over all of the major U.S. networks, but it only mentions Sprint by name. The following screenshot comes from a part of Red Pocket’s website that lists the networks the company offers service over:



The unnamed networks are color-coded to match the colors used in the host networks’ branding: AT&T in blue, T-Mobile in pink, and Verizon in red.

Ting

The MVNO Ting currently operates over Sprint and T-Mobile’s networks. Verizon will be added to the list soon. Today, Ting published a blog post about the upcoming addition. Ting still isn’t naming the networks it works with, but the company is making easy-to-interpret statements like: “In 2020, Ting will be on every major coast-to-coast network except AT&T.” The blog post included a video where the names of each network the company works with were censored out:

Xfinity Mobile Growing Quickly

An SEC filing Comcast published today showed that Xfinity Mobile has been adding subscribers quickly. In the last quarter, Xfinity Mobile added 261,000 new lines.1 Overall, Xfinity Mobile had a net increase of 816,000 lines in 2019, bringing its total subscriber count at the end of the year to slightly over two million lines.2 It’s now fair to say that Xfinity Mobile is one of the largest MVNOs in the U.S.

By my math, Xfinity Mobile’s subscriber base grew by about 66% in 2019.3 It will be interesting to see whether Comcast can keep that kind of growth rate going forward.

By one method of accounting, Xfinity Mobile looks unprofitable. However, the brand seems to be moving in the right direction towards profitability:4

Cable Communications results include a loss of $116 million from our wireless business, compared to a loss of $191 million in the prior period.
One could argue that way of looking at the financials leaves out something important. Subscribing to Xfinity Mobile makes it harder for a Comcast customer to cancel their internet service. Looking exclusively at the balance sheet of Comcast’s wireless segment leaves out the benefits Comcast receives as Xfinity Mobile reduces churn in the company’s other product segments.

At the moment, Xfinity Mobile presents a pretty good value proposition for current Xfinity Internet customers. Going forward, I’m not sure whether Comcast will keep Xfinity Mobile’s prices competitive to encourage growth or increase prices to make each subscriber more profitable.

Opensignal’s 2020 U.S. Mobile Performance Report

Today, Opensignal released a new report on the performance of U.S. wireless networks. The report details results on seven different metrics.

Here are the networks that took the top spot for each metric at the national level:

  • Video Experience – Verizon
  • Voice App Experience – T-Mobile/AT&T (draw)
  • Download Speed Experience – AT&T
  • Upload Speed Experience – T-Mobile
  • Latency Experience – AT&T
  • 4G Availability – Verizon
  • 4G Coverage Experience – Verizon

It’s important to interpret these results cautiously due to limitations in Opensignal’s crowdsourcing approach. Since performance data is collected from actual subscribers’ devices, factors not directly related to underlying network quality may impact the organization’s results. For example, if subscribers on a network are unusually likely to use low-end devices or live in rural areas, that will affect the outcome of Opensignal’s analyses. Still, Opensignal’s results are interesting; they’re drawn from a huge data set involving primarily automated performance tests.

Download speed findings

The most notable result in the latest report might be AT&T’s first-place finish on the download speed metric. In the previous Opensignal report, T-Mobile won first place for download speeds, and AT&T took third place. I’ve recently been critical of the methodologies used in some other evaluations that suggested AT&T had the nation’s fastest network. While many of those methodological criticisms still stand, the fact that Opensignal’s arguably more reliable methodology also found AT&T to have the fastest network leads me to believe I was too harsh. I’ll be interested to see whether AT&T also takes the top spot for speeds in RootMetrics’ upcoming report.

New metrics

Two new metrics were introduced in this report: Voice App Experience and 4G Coverage Experience. The Voice App Experience metric assesses the quality of voice calls via apps like Skype and Facebook Messenger. I’m not exactly sure how the metric works, but it looks like all four networks received similar scores. Opensignal deemed all these scores as indicative of “acceptable” quality.

The 4G Coverage Experience metric adds a bit of complexity to the previously existing 4G Availability metric. The coverage metric assesses 4G availability across areas all Opensignal’s users find themselves in, regardless of their network.

SIM Cards

TracFone Teases SmartSIM

TracFone, the company behind several large MVNOs, appears to be working on a new product called SmartSIM. Apparently, some TracFone customers recently received a marketing email that mentioned SmartSIM.1 I was briefly able to access NoDeadZone.com, a website that shared some basic information about SmartSIM. Oddly, the website now automatically redirects to locations.totalwireless.com (Total Wireless is a brand owned by TracFone).

While NoDeadZone.com was accessible, it offered a short video explaining SmartSIM. Apparently, the technology will allow subscribers to switch rapidly between multiple networks based on which network offers the best signal. There’s been some speculation about how up-and-coming eSim technology may enable more people to take advantage of dynamic network switching of this sort. However, the video I gave me the impression that SmartSIM would involve a conventional, removable SIM card rather than an eSIM. At the moment, I’m unsure if TracFone is licensing switching technology Google Fi built, introducing new technology, or something else.

NoDeadZones.com allowed visitors to enter their zip codes to see if SmartSIM was available where they lived. I tried several zip codes, and all were ineligible. It seems that other people had the same experience. I’m not sure whether any zip codes were really eligible for the service.

So far, I haven’t heard of TracFone responding to any requests for more information about SmartSIM. I’m curious about the lack of communication along with the decision to redirect NoDeadZone.com to the main Total Wireless site. It’s enough to make me wonder whether a mistake was made that led SmartSIM to become public knowledge before TracFone intended.

I’ll be keeping close tabs on how the story develops. Dynamic network switching has the potential to improve wireless service and change how it’s priced. With switching technology, it may be possible to charge different rates to different subscribers based on factors like a subscriber’s location, the extent of network congestion, or the quality of service a subscriber receives. Dynamic pricing could potentially lead to far more efficient network usage than conventional pricing—which might ultimately lead to a decrease in how much consumers pay for wireless service.

Infinity symbol

Tello Launches “Unlimited Everything” Plan

Today, Tello launched its “unlimited everything” plan for $39 per month. I’m frustrated by how Tello named its new plan. I say that as a fan of the company; Tello has some of the best options on the market for budget-sensitive consumers who don’t use a lot of data.

If subscribers on Tello’s unlimited everything plan use 25GB of data in a billing period, they will be throttled to sluggish, 2G speeds. As I’ve previously argued, unlimited plans at 2G speeds are bogus. Once the throttle kicks in, subscribers will find that data is unusable or barely usable for many purposes. While I’ve argued that many so-called “unlimited” plans are misnamed, Tello seems to have doubled down on its misnomer. Both “unlimited” and “everything” do a poor job of describing Tello’s new plan.

When carriers throttle data to 2G speeds, that usually means speeds are capped at 128Kbps. Imposing a maximum speed of 128Kbps puts a theoretical limit on total data use of about 65GB per month.1 In practice, few subscribers will use more than 26GB per month because the internet will be sluggish and frustrating use after the 25GB threshold is reached.

To Tello’s credit, the company does an unusually good job of disclosing the throttle. Here’s an example from the banner on Tello’s homepage this morning:

Tello Homepage Banner