FCC Reveals Misleading Coverage Claims

On Wednesday, the FCC released a fascinating report related to the Mobility Fund Phase II (MF-II). The MF-II is a planned program to provide federal funding for network build-outs in rural areas that are underserved by 4G coverage.

To determine which geographic areas were underserved, the FCC requested coverage maps and data from network operators. After reviewing the data and allowing outside entities to challenge the datas’ reliability, the FCC became concerned about the accuracy of the information shared by T-Mobile, U.S. Cellular, and Verizon. The FCC decided to conduct its own performance tests and compare the results of its tests to the information the network operators provided. Here’s what the agency found:1

Through the investigation, staff discovered that the MF-II coverage maps submitted by Verizon, U.S. Cellular, and T-Mobile likely overstated each provider’s actual coverage and did not reflect on-the-ground performance in many instances. Only 62.3% of staff drive tests achieved at least the minimum download speed predicted by the coverage maps—with U.S. Cellular achieving that speed in only 45.0% of such tests, T-Mobile in 63.2% of tests, and Verizon in 64.3% of tests…In addition, staff was unable to obtain any 4G LTE signal for 38% of drive tests on U.S. Cellular’s network, 21.3% of drive tests on T-Mobile’s network, and 16.2% of drive tests on Verizon’s network, despite each provider reporting coverage in the relevant area.

Incentives

When considering the accuracy of coverage maps, I try to think about the incentives network operators face. When advertising to consumers, network operators often have an incentive to overstate the extent of their coverage. However, incentives can run in the opposite direction in other situations. For example, when trying to get approval for a merger between Sprint and T-Mobile, Sprint had incentives to make its 4G coverage profile look limited and inferior to the coverage profiles of other nationwide networks.2

I’m not well-informed about the MF-II, so I don’t feel like I have a good grasp of all the incentives at play. That said, it’s not clear that all network operators would have an incentive to overstate their coverage. A network operator that claimed to offer coverage in an area it didn’t cover may limit competitors’ access to subsidies in that area. However, a network operator erroneously claiming to cover an area may prevent itself from receiving subsidies in that area.

Challenges

After network operators submitted coverage information to the FCC, a number of entities, including both governments and network operators, were allowed to challenge the validity of coverage information submitted by others. Here’s a bit more detail about the challenge process:3

After release of the map of presumptively eligible areas, mobile service providers, state, local, and Tribal government entities, and other interested parties granted a waiver were eligible to submit challenges in the challenge process via an online system operated by USAC. Challengers that requested access to the USAC MF-II Challenge Portal were able to access the provider-specific coverage maps, after agreeing to keep the coverage data confidential, and to file challenges to providers’ coverage claims by submitting speed test data. Challengers were required to conduct speed tests pursuant to a number of standard parameters using specific testing methods on the providers’ pre-approved handset models. The Commission adopted the requirement that challengers use one of the handsets specified by the provider primarily to avoid inaccurate measurements resulting from the use of an unsupported or outdated device—e.g., a device that does not support all of the spectrum bands for which the provider has deployed 4G LTE…During the eight-month challenge window, 106 entities were granted access to the MF-II Challenge Portal. Of the 106 entities granted access to the MF-II Challenge Portal, 38 were mobile service providers required to file Form 477 data, 19 were state government entities, 27 were local government entities, 16 were Tribal government entities, and six were other entities that filed petitions requesting, and were each granted, a waiver to participate.

About a fifth of the participating entities went on to submit challenges:4

21 challengers submitted 20.8 million speed tests across 37 states.

The challenge data often showed failed tests and lackluster speeds in areas where network operators claimed to offer coverage:5

During the challenge process, some parties entered specific concerns into the record. For example:6

Smith Bagley (d/b/a Cellular One) submitted maps of its service area in Arizona overlaid with Verizon’s publicly-stated 4G LTE coverage and the preliminary results of drive tests that Smith Bagley had conducted. Smith Bagley asserted that, for large stretches of road in areas where Verizon reported coverage, its drive testers recorded no 4G LTE signal on Verizon’s network. Smith Bagley argued that the ‘apparent scope of Verizon’s inaccurate data and overstated coverage claims is so extensive that, as a practical matter, the challenge process will not and cannot produce the necessary corrections.’
As part of a public report detailing its experience, Vermont published a map showing its speed test results which contradicted the coverage maps in Vermont of U.S. Cellular, T-Mobile, and Verizon, among others. This map included information on the approximately 187,000 speed tests submitted by Vermont, including download speed, latency, and signal strength. In the report, Vermont detailed that 96% of speed tests for U.S. Cellular, 77% for T-Mobile, and 55% for Verizon failed to receive download speeds of at least 5 Mbps.

After reviewing the challenges, the FCC requested additional information from the five largest network operators (AT&T, T-Mobile, Verizon, Sprint, and U.S. Cellular) to understand the assumptions involved in the networks’ coverage models.

FCC tests

Around the same time the FCC was requesting additional information from network operators, the agency also began its own testing of Verizon, U.S. Cellular, and T-Mobile’s networks. These speed tests took place in 12 states and primarily made use of a drive-testing methodology. As mentioned earlier, analyses of the FCC’s test data suggested that the on-the-ground experience with Verizon, T-Mobile, and U.S. Cellular’s network was much different than the experience that would be expected based on the information the networks provided to the FCC.

What happened?

A lot of the commentary and news articles I’ve seen in response to the FCC’s report seem to conclude that network operators are bullshitters that intentionally lied about the extent of their coverage. I have reservations about fully accepting that conclusion. Accurately modeling coverage is difficult. Lots of factors affect the on-the-ground experience of wireless subscribers. The FCC largely acknowledges this reality in its report:

Providers were afforded flexibility to use the parameters that they used in their normal course of business when parameters were not specified by the Commission. For example, the Commission did not specify fading statistics or clutter loss values, and providers were required to model these factors as they would in the normal course of business.7
Our speed testing, data analyses, and inquiries, however, suggest that some of these differences may be the result of some providers’ models: (1) using a cell edge RSRP value that was too low, (2) not adequately accounting for network infrastructure constraints, including backhaul type and capacity, or (3) not adequately modeling certain on-the-ground factors—such as the local clutter, terrain, and propagation characteristics by spectrum band for the areas claimed to be covered.8

Further supporting the idea that assessing coverage is difficult, the FCC didn’t just find that its tests contradicted the initial information submitted by network operators. The FCC data also contradicted the data submitted by those who challenged network operators’ data:

The causes of the large differences in measured download speed between staff and challenger speed tests taken within the same geographic areas, as well as the high percentage of tests with a download speed of zero in the challenger data, are difficult to determine. Discrepancies may be attributable to differences in test methodologies, network factors at the time of test, differences in how speed tet apps or drive test software process data, or other factors…Given the large differences between challenger and staff results however, we are not confident that individual challenger speed test results provide an accurate representation of the typical consumer on-the-ground experience.9

While the FCC found some of the information submitted by networks to be misleading about on-the-ground service quality, I don’t believe it ended up penalizing any network operators or accusing them of anything too serious.10 Still, the FCC did suggest that some of the network operators could have done better:

Staff engineers, however, found that AT&T’s adjustments to its model to meet the MF-II requirements may have resulted in a more realistic projection of where consumers could receive mobile broadband. This suggests that standardization of certain specifications across the largest providers could result in coverage maps with improved accuracy. Similarly, the fact that AT&T was able to submit coverage data that appear to more accurately reflect MF-II coverage requirements raises questions about why other providers did not do so. And while it is true that MF-II challengers submitted speed tests contesting AT&T’s coverage data, unlike for other major providers, no parties alleged in the record that AT&T’s MF-II coverage data were significantly overstated.11

FCC response

The FCC concluded that it should make some changes to its processes:12

First, the Commission should terminate the MF-II Challenge Process. The MF-II coverage maps submitted by several providers are not a sufficiently reliable or accurate basis upon which to complete the challenge process as it was designed.
Second, the Commission should release an Enforcement Advisory on broadband deployment data submissions, including a detailing of the penalties associated with filings that violate federal law, both for the continuing FCC Form 477 filings and the new Digital Opportunity Data Collection. Overstating mobile broadband coverage misleads the public and can misallocate our limited universal service funds.
Third, the Commission should analyze and verify the technical mapping data submitted in the most recent Form 477 filings of Verizon, U.S. Cellular, and T-Mobile to determine whether they meet the Form 477 requirements. Staff recommends that the Commission assemble a team with the requisite expertise and resources to audit the accuracy of mobile broadband coverage maps submitted to the Commission. The Commission should further consider seeking appropriations from Congress to carry out drive testing, as appropriate.
Fourth, the Commission should adopt policies, procedures, and standards in the Digital Opportunity Data Collection rulemaking and elsewhere that allow for submission, verification, and timely publication of mobile broadband coverage data. Mobile broadband coverage data specifications should include, among other parameters, minimum reference signal received power (RSRP) and/or minimum downlink and uplink speeds, standard cell loading factors and cell edge coverage probabilities, maximum terrain and clutter bin sizes, and standard fading statistics. Providers should be required to submit actual on-the-ground evidence of network performance (e.g., speed test measurement samplings, including targeted drive test and stationary test data) that validate the propagation model used to generate the coverage maps. The Commission should consider requiring that providers assume the minimum values for any additional parameters that would be necessary to accurately determine the area where a handset should achieve download and upload speeds no less than the minimum throughput requirement for any modeling that includes such a requirement.

Reflections

The FCC’s report illustrates how hard it is to assess network performance. Assumptions must be made in coverage models, and the assumptions analysts choose to make can have substantial effects on the outputs of their models. Similarly, on-the-ground performance tests don’t always give simple-to-interpret results. Two entities can run tests in the same area and find different results. Factors like the time of day a test was conducted or the type of device that was used in a test can have big consequences.

If we want consumers to have better information about the quality of service networks can offer, we need entities involved in modeling and testing coverage to be transparent about their methodologies.

VerHIDEzon – Brought To You By T-Mobile

T-Mobile just started a satirical ad campaign criticizing Verizon. T-Mobile’s CEO, John Legere, kicked the campaign off with this tweet:

Tweet from T-Mobile's CEO

The ad campaign criticizes Verizon for its decision to charge a premium for 5G service without publishing a map of areas where 5G service is available. The website for the campaign, VerHIDEzon.com, has some entertaining content:

We believe in charging a premium for 5G, without telling you where you’ll have coverage.
Why do we do this? Because we’re VerHIDEzon, and we do whatever we want…Every day we wake up with one goal in mind: charge our customers as much as possible.


T-Mobile makes a good point. It’s silly for Verizon to charge for 5G service without publishing information that indicates the extent of Verizon’s 5G coverage. Still, I find the campaign kind of odd. Neither company has much 5G coverage at the moment. Almost no one is using 5G-compatible phones yet. It may make business sense for T-Mobile to run the campaign today, but more time will need to pass before 5G has a lot of relevance for typical consumers.

T-Mobile Launches New Test Drive Program

In the last couple of days, T-Mobile has begun a new trial program. Non-customers can get a mobile hotspot along with 30 days of service and 30GB of data at no cost.

The program provides an easy option for people to test how well T-Mobile’s network could work for them. Once a hotspot is turned on, users can connect their existing phones and devices via Wi-Fi. The hotspots are compatible with LTE band 71, so hotspot users can experience the benefits of T-Mobile’s new Extended Range LTE signal.1

What’s the catch? As far as I can tell, there isn’t one. Signing up is easy, no credit card is required, and it’s not even necessary to return the hotspot at the end of the trial. I’ve joined the program, and I’ll probably post again in about a month with an update on my experience.

You can sign up for the program here. T-Mobile’s CEO, John Legere, discusses the program in more detail in the video below.

DOJ Clears T-Mobile’s Merger With Sprint

As expected, the Department of Justice made an announcement today approving a merger between Sprint and T-Mobile. While the merger isn’t officially closed, DOJ approval was the largest hurdle T-Mobile and Sprint needed to jump before making their merger a reality.

As far as I can tell, the terms of the merger were consistent with what most commentators were expecting:

  • Most of Sprint’s prepaid business will be divested to DISH1
  • DISH will get Sprint’s 800 MHz spectrum
  • DISH will receive access to the New T-Mobile’s network for at least 7 years2
  • DISH will have the option to take over leases on some retail stores and cell sites

I don’t think mergers between telecom companies have a good track record of benefiting consumers. I hope this merger will be different, but I’m not betting on it. As many others have pointed out, something is odd about the whole arrangement. The divestitures to DISH are ostensibly intended to allow DISH to create a viable, facilities-based carrier (i.e., a carrier that has its own hardware and doesn’t just piggyback off other companies’ networks). If DISH is likely to succeed, it’s hard to explain why Sprint couldn’t remain a viable force. Maybe I’m misunderstanding something important.

I expect the merger-related transitions to take a few years, and I plan to write about new developments as they occur. Should be interesting.


For those interested, here are a few excerpts from T-Mobile’s announcement:

The proposed New T-Mobile, will divest Sprint’s prepaid businesses and Sprint’s 800 MHz spectrum assets to DISH. Additionally, upon the closing of the divestiture transaction, the companies will provide DISH wireless customers access to the New T-Mobile network for seven years and offer standard transition services arrangements to DISH during a transition period of up to three years. DISH will also have an option to take on leases for certain cell sites and retail locations that are decommissioned by the New T-Mobile, subject to any assignment restrictions.
The New T-Mobile will be committed to divest Sprint’s entire prepaid businesses including Boost Mobile, Virgin Mobile and Sprint-branded prepaid customers (excluding the Assurance brand Lifeline customers and the prepaid wireless customers of Shenandoah Telecommunications Company and Swiftel Communications, Inc.), to DISH for approximately $1.4 billion. These brands serve approximately 9.3 million customers in total.
With this agreement, Boost Mobile, Virgin Mobile, and Sprint-branded prepaid customers, as well as new DISH wireless customers, will have full access to the legacy Sprint network and the New T-Mobile network in a phased approach. Access to the New T-Mobile network will be through an MVNO arrangement, as well as through an Infrastructure MNO arrangement enabling roaming in certain areas until DISH’s 5G network is built out.
The companies have also committed to engage in good faith negotiations regarding the leasing of some or all of DISH’s 600 MHz spectrum to T-Mobile.
Image depicting the idea of "change."

Ting Plans to Drop T-Mobile and Add Verizon

Added 2/18/2020: Ting has found a way to continue offering service over T-Mobile’s network going forward. The parts of this post about Ting’s plans to migrate subscribers away from T-Mobile’s network are no longer accurate.

Yesterday, it was made public that the mobile virtual network operator Ting will soon cease offering service over T-Mobile’s network and begin offering service over Verizon’s network. It was also announced that Ting had extended its existing agreement with Sprint through September 2020.1

Dropping T-Mobile

Based on my understanding of yesterday’s SEC filing, I expect Ting will continue to offer service over T-Mobile’s network to existing subscriber until at least late 2019 and possibly late 2020.2 I’m less sure whether new customers will be able to sign up for service over T-Mobile’s network for much longer.

Adding Verizon

The new arrangement with Verizon is based on a five-year agreement that Elliot Noss, CEO and president of Ting’s parent company spoke positively about:3

“With Verizon, we will be adding the network that in our opinion has the best coverage and performance ratings in the U.S. Our contract with Verizon is better than that with T-Mobile in terms of rates, guarantees and other financial terms, which had negatively impacted Ting Mobile’s past performance. Finally, our dealings with Verizon to this point have been productive and professional. ​​So long-term, we see this as very positive news.”

Going Forward

I’ve previously raved about Ting’s customer support, but I’ve been reluctant to strongly recommend Ting since its rates have been fairly high for access to mediocre networks (Sprint and T-Mobile). If Ting’s rates don’t increase as a result of moving to Verizon’s higher-quality network, I think the case for recommending Ting becomes a lot stronger.

Ting anticipates some friction migrating its T-Mobile subscribers to new networks:4

“We estimate the costs of migration, primarily in the form of SIMs, shipping and device marketing, to be in the range of $3 million this year, and as much as $12 million over the following years. These variable costs are mostly in the nature of the marketing costs needed to move customers from one network to the other, and will mostly be in the form of inducements, device subsidies and/or a form of service credit. There’s not a lot of precedent to provide guidance on which marketing programs will be most effective, and we are also unsure of how many, if any, of our customers will refuse or fail to move. This makes the total migration cost difficult to estimate. Taking on as much as $12 to $15 million in unplanned, one-time costs, over a few years, is a lot for us. That being said, this move is key to putting the mobile business back on a stronger long-term footing.

When Ting drops T-Mobile, it will no longer offer service over a GSM network. I expect the move from T-Mobile (and GSM) to Verizon makes more financial sense today that it would have a few years ago. As operators gradually replace their 3G networks with LTE networks, support for older GSM and CDMA technologies is becoming less important.


Added 2/26/2020: Ting officially launched service over Verizon’s network in February 2020.