Don’t Take Ookla Too Seriously

Ookla recently published its Q2 report on the performance of U.S. wireless networks. As I’ve discussed before, I’m not a fan of Ookla’s methodology.[1] Because of my qualms, I’m not going to bother summarizing Ookla’s latest results. However, I do want to draw attention to a part of the recent report.

Ookla’s competitive geographies filter

In the last year or two, Ookla has restricted its main analyses to only account for data from “competitive geographies.” Here’s how Ookla explains competitive geographies:

To meet the definition of ‘competitive’ in the U.S., a zip code must contain samples from at least three top national competitors…but no competitor can have more than 2/3 of the samples in that zip code.

The competitive geographies filter mitigates some of the problems with Ookla’s methodology but also introduces a bunch of new issues.

Availability

Ookla’s latest results for 4G availability illustrate the issues:

Ookla 4G availability scores

Sprint unambiguously has the smallest coverage profile of the four nationwide networks.[2] The competitive geographies filter makes Ookla’s availability metric so meaningless that Sprint can nevertheless tie for the best availability.

Lots of regions only have coverage from Verizon. All those data points get thrown away because they come from non-competitive geographies. Other areas have coverage from only Verizon and AT&T. Again, those data points get thrown out because they’re not from competitive geographies. What’s the point of measuring availability while ignoring the areas where differences in network availability are most substantial?

Giving Ookla credit

Despite my criticisms, I want to give Ookla some credit. Many evaluators develop complicated, poorly thought-out metrics and only share those metrics when the results seem reasonable. I appreciate that Ookla didn’t hide its latest availability information because the results looked silly.

Google Fi After The T-Mobile & Sprint Merger

Google Fi brought a lot of innovations and customer-friendly features to the wireless market. I’d argue that Fi’s biggest innovations have been in network switching. Subscribers using “Designed for Fi” phones can automatically switch between coverage from T-Mobile, Sprint, and U.S. Cellular’s networks.

Losing Sprint

Fi’s network switching is about to become a lot less interesting. Sprint’s network will disappear. U.S. Cellular doesn’t have a nationwide network.

The darker shade in the map below shows where U.S. Cellular’s network is available:[1]

Map of licensed U.S. Cellular markets

U.S. Cellular’s network does not cover the majority of the U.S. Once Sprint’s network is gone, Google Fi will be a T-Mobile-based carrier in many places.[2]

T-Mobile’s network will get better as it integrates Sprint’s assets, so I don’t expect Fi to decrease substantially in quality. However, Fi may become a much less competitive option in comparison to other carriers. There are a lot of carriers that run over T-Mobile’s network. These carriers will also offer better performance as T-Mobile improves its network. Some carriers using T-Mobile’s network are priced much better than Fi. For example, Mint Mobile sells a plan with 8GB of data, unlimited minutes, and unlimited texts for as low as $20 per month. Fi would charge at least $70 per month for the same level of usage.[3]

I don’t mean to imply Fi will be left in the dust. The carrier offers high priority data, amazing international roaming options, and a user-friendly experience. Many low-cost, T-Mobile-based carriers don’t have those elements. Can Fi convince subscribers that Fi’s premium features justify the service’s price tag?

Will MVNOs get squeezed?

Low-cost carriers may get squeezed by T-Mobile. When Sprint goes offline, MVNOs will have fewer networks they can offer service over. The reduction in options may allow T-Mobile to increase the rates it charges carriers that use T-Mobile’s network.[4] While low-cost carriers may have no option but to raise the prices charged to consumers, Fi may be better positioned. Fi is fairly expensive. It’s unlikely T-Mobile would charge Fi so much that Google would struggle to stay in the market.

Piggy bank

AT&T’s Killer Deals On Prepaid Plans

AT&T tweaked its prepaid plans a little while back. I think some of the current offerings are awfully good for people who want a balance between cost and performance.

AT&T’s prepaid plan with unlimited minutes, unlimited texts, and 8GB of data is a particularly good option. This plan has enough data for most people, and as best as I can tell comes with the same level of priority during network congestion as most of AT&T’s postpaid plans. The plan has a base price of $50 per month, but there are several ways it can be purchased at a significant discount.

  • $40 per month on a month-to-month plan with automatic payments enabled
  • $33 per month with three months paid upfront
  • $25 per month with 12 months paid upfront

The plan is a solid competitor to Mint Mobile’s popular 8GB plan. Mint’s plan is priced a bit differently. New customers or those purchasing a year of service upfront can get Mint’s plan for as low as $20 per month.[1] While Mint’s plan can be a bit cheaper, the carrier runs over T-Mobile’s network, which has a smaller coverage profile than AT&T. Mint subscribers will also have low priority data access, which could lead to slow speeds during periods of network congestion.

AT&T is also offering a pretty great deal on one of its prepaid unlimited plans. With automatic payments enabled, AT&T’s Unlimited Data Plus plan is only $50 per month. Unlike a lot of prepaid unlimited plans, subscribers on this plan will have high-priority data for the first 22GB of data use each month.

Verizon’s 5G Mobile Hotspot & Tethering Policies

Verizon’s higher-end unlimited plans come with monthly allotments of full-speed 4G hotspot data.

  • Play More Unlimited – 15GB
  • Do More Unlimited – 15GB
  • Get More Unlimited – 30GB

Subscribers that burn through all of their full-speed 4G data can continue to use more hotspot data with a 4G connection throttled to a sluggish 600Kbps speed.

Verizon doesn’t prominently advertise it, but the company has separate policies for 5G hotspot data. Here’s a bit from a section of Verizon’s website that gives additional details about one of the carrier’s unlimited plans:[1]

Unlimited 5G Ultra Wideband Mobile Hotspot

An unlimited 5G Ultra Wideband mobile hotspot means your smartphone becomes an ultra-fast and reliable Wi-Fi connection for your other devices. Service available in select locations and requires a 5G capable device.

Yesterday, I was exploring my Verizon online account for the Play More Unlimited plan and found this:

Screenshot of Verizon account interface showing a 5G hotspot allotment

Verizon is currently allowing 50GB of full-speed 5G hotspot use each month on the Play More plan. After 50GB of use, speeds are throttled to 3Mbps. While 3Mbps isn’t a great speed for hotspot use, it’s still fast enough that most people could browse the internet and do work on a laptop without overwhelming frustration.

I’m guessing the Do More and Get More plans have the same 50GB allotment, but I’m not sure. It’s odd that Verizon’s 5G hotspot allotment on the Play More plan is so much larger than the regular (4G) allotment.

I wonder if the large allotment is a promotional thing. Will we see the allotment shrink as more subscribers get 5G-compatible phones?

Opensignal’s Report on U.S. 5G Performance – No Big Surprises

Earlier this week, Opensignal released a report on the performance of 5G networks in the United States. Opensignal’s report puts some numbers and data behind two things that were already clear:

  • T-Mobile is destroying the competition in terms of 5G coverage, but T-Mobile’s 5G isn’t very fast
  • Verizon’s 5G is outrageously fast, but the coverage profile is terrible.

Opensignal primarily collects its performance data by crowdsourcing data from tests that run in the background on regular people’s phones. It looks like the company restricted the data underlying this report to include tests run from 5G-compatible phones.

Speeds

Verizon destroyed the competition with an average 5G download speed of 495Mbps. The other major networks in the U.S. had 5G download speeds averaging around 50Mbps. Verizon’s dominance in download speeds is due to the company’s focus on rolling out millimeter wave 5G.

Coverage

Unlike Verizon, T-Mobile has focused on deploying sub-6 5G. This type of 5G is great for covering large areas, but less impressive for delivering high speeds. Unsurprisingly, T-Mobile dominated in terms of 5G availability. According to Opensignal’s data, T-Mobile subscribers were able to access 5G 22.5% of the time. Verizon did about fifty times worse with an availability score of 0.4%.

While 0.4% is low, it’s still a better availability score than I would have predicted for Verizon. I wonder if Opensignal’s crowdsourcing approach might lend Verizon’s availability scores a leg up. If living near a Verizon 5G deployment makes a Verizon customer more likely to purchase a 5G phone, selection bias can creep in and cause Opensignal to overestimate Verizon’s actual 5G availability.

Silly press releases

Following Opensignal’s release of its report, T-Mobile published a press release. The company bragged about the network’s excellent 5G coverage without mentioning that the network got demolished in the download speed results.

Verizon published its own press release bragging about ludicrous download speeds. Verizon’s awful 5G availability score was not mentioned.

Download Speed Experience – 5G Users

Opensignal’s report included a metric called Download Speed Experience – 5G Users. The results for this metric were calculated by looking at users with 5G-compatible phones and tracking their average download speed even at times where they did not have 5G connections. In some sense, this single metric does some accounting for both 5G speeds and 5G availability.

Verizon and AT&T tied for the top spot:
Results graph showing AT&T and Verizon tied for the top spot

The metric is interesting, but I don’t think it quite captures how users will feel about the quality of their download speed experiences. The marginal value of a 10Mbps boost in download speeds that moves a subscriber from 5Mbps to 15Mbps is much greater than the marginal value of a 10Mbps boost that moves a subscriber from 500Mbps to 510Mbps. Collapsing a distribution of download speeds into a single, average download speed masks this reality.

Verizon’s Revamped Shared Data Plans

Verizon used to offer several different plans with shared pools of data. The carrier has now simplified its offerings with only two shared data plans. Both plans include unlimited minutes and texts. The cheaper plan offers 5GB of shared data. The more expensive plan offers 10GB of shared data.

For customers who enroll in paperless billing and Auto Pay, The 5GB plan costs $30 per month plus $25 for each line. The 10GB plan costs $40 per month plus $25 for each line.

Customers that don’t enroll in Auto Pay and paperless billing will be charged $10 more every month on each line.

Examples

  • Two lines on the 5GB plan would cost $80 per month with Auto Pay and paperless billing. That would include the $30 base charge for the plan and two $25 line-access fees.
  • Three lines on the $10 GB plan would cost $105 per month with Auto Pay and paperless billing (a $40 base charge plus $75 in line-access fees).
Abstract art about the idea of updates

Verizon’s Updates To Prepaid Plans

Verizon recently updated the structure of its prepaid plans. This update was needed. As I mentioned in a previous blog post, Verizon’s old structure for prepaid plans was confusing and difficult for consumers to make sense of.

Main new phone plans

Verizon now offers three primary prepaid phone plans. All three plans come with unlimited minutes and texts. The plans vary in their monthly data allotments:

  • Unlimited – $65 base price
  • 15GB – $50 base price
  • 5GB – $40 base price

Several discounts are available. Customers that enroll in automatic payments can save $5 per line each month after the first month of service. Verizon has also introduced loyalty discounts on these plans. After three months of service, a $5 per line discount kicks in each month. The loyalty discount jumps to $10 per line each month after nine months of service.

Here are the monthly prices a long-term customer eligible for all the discounts would end up paying (before taxes and fees):

  • Unlimited – $50
  • 15GB – $35
  • 5GB – $25

Verizon has dropped the multi-line discounts it used to offer on prepaid accounts with more than one line.

Other plans

Verizon offers a somewhat-hidden talk and text plan (no high-speed data) with a base price of $35 per month. The plan is eligible for the automatic payments discount but is not eligible for loyalty discounts.

Three data plans are available for tablet and hotspot devices. Customers on these plans are eligible for a $5 per month autopay discount.

  • 6GB – $40 base price
  • 16GB – $50 base price
  • 30GB – $70 base price

Switching promo

Verizon is offering an online-only promo on the 15GB plan. Customers that switch to the plan can get a $60 bill credit. Here are the terms:[1]

Requires new port in phone activation. Must be active on a 15 GB or Unlimited Verizon Prepaid plan for the first 2 months. $60 service credit applied to Account Owner immediately after 2nd monthly plan payment. Offer not available for tablets or Jetpacks.
5G abstract

5G+ Brilliant Marketing From AT&T?

A while back, AT&T began calling some its 4G service 5GE. It tricked consumers into thinking they were getting 5G when they weren’t. It was bullshit.

Now, AT&T is calling its millimeter wave (mmWave) service 5G+. In some ways, the 5G+ label could confuse consumers. Verizon’s 5G service uses pretty much exclusively mmWave right now. Verizon doesn’t call its mmWave service “5G+.” To Verizon, mmWave is just “5G.”

Still, I’m not annoyed by AT&T’s 5G+ branding. Not all 5G is the same. Lower frequency, sub-6 5G, is typically way slower than mmWave 5G. Laypeople don’t realize how different sub-6 5G is from mmWave 5G. If we want consumers to understand the difference, carriers need to help make that happen. Calling sub-6 service “5G” and mmWave service “5G+” makes a clear distinction for consumers.

Neville Ray’s Statement on T-Mobile’s Outage

Yesterday, Neville Ray, President of Technology at T-Mobile, shared more information about the cause of T-Mobile’s outage on Monday:

Many of our customers experienced a voice and text issue yesterday, specifically with VoLTE (Voice over LTE) calling…The trigger event is known to be a leased fiber circuit failure from a third party provider in the Southeast. This is something that happens on every mobile network, so we’ve worked with our vendors to build redundancy and resiliency to make sure that these types of circuit failures don’t affect customers. This redundancy failed us and resulted in an overload situation that was then compounded by other factors. This overload resulted in an IP traffic storm that spread from the Southeast to create significant capacity issues across the IMS (IP multimedia Subsystem) core network that supports VoLTE calls.

We have worked with our IMS (IP Multimedia Subsystem) and IP vendors to add permanent additional safeguards to prevent this from happening again and we’re continuing to work on determining the cause of the initial overload failure.

So, I want to personally apologize for any inconvenience that we created yesterday and thank you for your patience as we worked through the situation toward resolution.

In Hindsight: T-Mobile’s Network Outage

Yesterday, T-Mobile experienced a serious network outage beginning around 10am MT. The outage persisted through most of the day and primarily affected voice and text services.

Around 11pm MT, T-Mobile announced that outage was over:

These issues are now resolved. We again apologize for any inconvenience and thank you for your patience.

What caused the outage?

A lot of speculation was floating around yesterday. Some Twitter users, including a congressman, suggested there was a huge distributed denial-of-service attack causing trouble for U.S. networks. This story never made much sense.

Others speculated that a massive network failure was causing issues for all the major cellular operators. This idea was based on information from the website Downdetector, an entity that aggregates user complaints in real-time. Downdetector suggested that customer complaints about Verizon and AT&T were increasing at the same time that T-Mobile was experiencing issues.

Here’s how Downdectector describes its methodology:

Downdetector collects status reports from a series of sources, including Twitter and reports submitted on our websites and mobile apps. Our system validates and analyzes these reports in real-time, allowing us to automatically detect outages and service disruptions in their very early stages.

I expect the customer complaints about Verizon and AT&T were ultimately caused by T-Mobile’s network issues. Verizon and AT&T customers may have tried to call T-Mobile customers, then thought their own carriers (rather than T-Mobile) were at fault when calls failed.

Both AT&T and Verizon made statements suggesting that their networks were operating normally.

Updates from T-Mobile & the FCC

Yesterday evening, T-Mobile’s CEO, Mike Sievert, gave a vague explanation of what caused the outage:

T-Mobile has been experiencing a voice and text issue that has intermittently impacted customers in markets across the U.S…This is an IP traffic related issue that has created significant capacity issues in the network core throughout the day. Data services have been working throughout the day.

The outage was likely triggered as T-Mobile took steps to merge Sprint’s technology and/or customer base into the New T-Mobile. I haven’t heard any further information about the outage from a plausible-looking source, but details may come out soon. Ajit Pai, the FCC Chairman, shared the following tweet last night: