Opensignal just released a new report on the performance of U.S. wireless networks. The report ranks major U.S. networks in five categories based on crowdsourced data:
Download speed experience
Upload speed experience
Verizon took the top spot for 4G availability and video experience. T-Mobile came out on top for both of the speed metrics. T-Mobile and AT&T shared the top placement for the latency experience metric.
I’ve previously raised concerns about selection bias in Opensignal’s data collection methodology. Opensignal crowdsources data from typical users. Crowdsourcing introduces issues since there are systematic differences between the typical users of different networks. Imagine that Network A has far more extensive coverage in rural areas than Network B. It stands to reason that Network A likely has more subscribers in rural areas than Network B. Lots of attributes of subscribers vary in similar ways between networks. E.g., expensive networks likely have subscribers that are wealthier.
Analyses of crowdsourced data can capture both (a) genuine differences in network performance and (b) differences in how subscribers on each network use their devices. Opensignal’s national results shouldn’t be taken too seriously unless Opensignal can make a compelling argument that either (a) its methodology doesn’t lead to serious selection bias or (b) it’s able to adequately adjust for the bias.
Opensignal ranks carriers based on average download and upload speeds. In my opinion, average speeds are overrated. The portion of time where speeds are good enough is much more important than the average speed a service offers.
Opensignal’s average download speed results are awfully similar between carriers:
Verizon – 22.9 Mbps
T-Mobile – 23.6 Mbps
AT&T – 22.5 Mbps
Sprint – 19.2 Mbps
Service at any of those speeds would be sufficient for almost any activities people typically use their phones for. Without information about how often speeds were especially low on each network, it’s hard to come to conclusions about differences in the actual experience on each network.
Added 2/18/2020: Ting has found a way to continue offering service over T-Mobile’s network going forward. The parts of this post about Ting’s plans to migrate subscribers away from T-Mobile’s network are no longer accurate.
Yesterday, it was made public that the mobile virtual network operator Ting will soon cease offering service over T-Mobile’s network and begin offering service over Verizon’s network. It was also announced that Ting had extended its existing agreement with Sprint through September 2020.
Based on my understanding of yesterday’s SEC filing, I expect Ting will continue to offer service over T-Mobile’s network to existing subscriber until at least late 2019 and possibly late 2020. I’m less sure whether new customers will be able to sign up for service over T-Mobile’s network for much longer.
The new arrangement with Verizon is based on a five-year agreement that Elliot Noss, CEO and president of Ting’s parent company spoke positively about:
“With Verizon, we will be adding the network that in our opinion has the best coverage and performance ratings in the U.S. Our contract with Verizon is better than that with T-Mobile in terms of rates, guarantees and other financial terms, which had negatively impacted Ting Mobile’s past performance. Finally, our dealings with Verizon to this point have been productive and professional. So long-term, we see this as very positive news.”
I’ve previously raved about Ting’s customer support, but I’ve been reluctant to strongly recommend Ting since its rates have been fairly high for access to mediocre networks (Sprint and T-Mobile). If Ting’s rates don’t increase as a result of moving to Verizon’s higher-quality network, I think the case for recommending Ting becomes a lot stronger.
Ting anticipates some friction migrating its T-Mobile subscribers to new networks:
“We estimate the costs of migration, primarily in the form of SIMs, shipping and device marketing, to be in the range of $3 million this year, and as much as $12 million over the following years. These variable costs are mostly in the nature of the marketing costs needed to move customers from one network to the other, and will mostly be in the form of inducements, device subsidies and/or a form of service credit. There’s not a lot of precedent to provide guidance on which marketing programs will be most effective, and we are also unsure of how many, if any, of our customers will refuse or fail to move. This makes the total migration cost difficult to estimate. Taking on as much as $12 to $15 million in unplanned, one-time costs, over a few years, is a lot for us. That being said, this move is key to putting the mobile business back on a stronger long-term footing.
When Ting drops T-Mobile, it will no longer offer service over a GSM network. I expect the move from T-Mobile (and GSM) to Verizon makes more financial sense today that it would have a few years ago. As operators gradually replace their 3G networks with LTE networks, support for older GSM and CDMA technologies is becoming less important.
Added 2/26/2020: Ting officially launched service over Verizon’s network in February 2020.
On June 20, PCMag published its latest results from performance testing on the major U.S. wireless networks. Surprisingly, AT&T rather than Verizon took the top spot in the overall rankings. I expect this was because PCMag places far more weight on network performance within cities than performance in less-populated areas.
In my opinion, PCMag’s methodology overweights average upload and download speeds at the expense of network reliability. Despite my qualms, I found the results interesting to dig into. PCMag deserves a lot of credit for its thoroughness and unusual level of transparency.
PCMag claims to be more transparent about its methodology than other entities that evaluate wireless networks. I’ve found this to be true. PCMag’s web page covering its methodology is detailed. Sascha Segan, the individual who leads the testing, quickly responded to my questions with detailed answers. I can’t say anything this positive about transparency demonstrated by RootMetrics or OpenSignal.
To measure network performance, PCMag used custom speed test software developed by Ookla. The software was deployed on Samsung Galaxy S10 phones that were driven to 30 U.S. cities as they collected data. In each city, stops were made in several locations for additional data collection. PCMag only recorded performance on LTE networks. If a phone was connected to a non-LTE network (e.g., a 3G network) during a test, the phone would fail that test. PCMag collected data on six metrics:
Average download speed
Percent of downloads over a 5Mbps speed threshold
Average upload speed
Percent of uploads over a 2Mbps speed threshold
Reliability (percent of the time a connection was available)
The Galaxy S10 is a recent, flagship device and has essentially the best technology available for high-performance on LTE networks. Accordingly, PCMag’s test are likely to show better performance than consumers using lower-end devices will experience. PCMag’s decision to use the same high-performance device on all networks may prevent selection bias that sometimes creeps up in crowdsourced data when subscribers on one network tend to use different devices than subscribers on another network.
In my opinion, PCMag’s decision not to account for performance on non-LTE networks somewhat limits the usefulness of its results. Some network operators still use a lot of non-LTE technologies.
PCMag accounts for networks’ performance on several different metrics. To arrive at overall rankings, PCMag gives networks a score for each metric and assigns specific weights to each metric. Scoring multiple metrics and reasonably assigning weights is far trickier than most people realize. A lot of evaluation methodologies lose their credibility during this process (see Beware of Scoring Systems).
PCMag shares this pie chart when describing the weights assigned to each metric:
The pie chart doesn’t tell the full story. For each metric, PCMag gives the best-performing network all the points available for that metric. Other networks are scored based on how far they are away from the best-performing network. For example, if the best-performing network has an average download speed of 100Mbps (a great speed), it will get 100% of the points available for average download speed. Another network with an average speed of 60Mbps (a good speed) would get 60% of the points available for average download speed.
The importance of a metric is determined not just by the weight it’s assigned. The variance in a metric is also extraordinarily important. PCMag measures reliability in terms of how often a phone has an LTE connection. Reliability has low variance. 100% reliability indicates great coverage (i.e., a connection is always available). 80% reliability is bad. Networks’ reliability barely affects PCMag’s rankings since reliability measures are fairly close to 100% even on unreliable networks.
The scoring system is sensitive to how reliability numbers are presented. Imagine there are only two networks:
Network A with 99% reliability
Network B with 98% reliability
Using PCMag’s approach, both network A and B would get a very similar number of points for reliability. However, it’s easy to change how the same metric is presented:
Network A has no connection 1% of the time
Network B has no connection 2% of the time
If PCMag put the reliability metric in this format, network B would only get half of the points available for reliability.
As a general rule, I think average speed metrics are hugely overrated. It’s important that speeds are good enough for people to do what they want to do on their phones. Having speeds that are way faster than the minimum speed that’s sufficient won’t benefit people much.
I’m glad that PCMag put some weight on reliability and on the proportion of tests that exceeded fairly minimum upload and download speed thresholds. However, these metrics just don’t have nearly as much of an effect on PCMag’s final results as I think they should. The scores for Chicago provide a good illustration:
Despite having the worst reliability score and by far the worst score for downloads above a 5Mbps threshold, T-Mobile still manages to take the top ranking. Without hesitation, I’d choose service with Verizon or AT&T’s performance in Chicago over service with T-Mobile’s performance in Chicago. (If you’d like to get a better sense of how scores for different metrics drove the results in Chicago, see this Google sheet where I’ve reverse engineered the scoring.)
To create rankings for regions and final rankings for the nation, PCMag combines city scores and scores for suburban/rural areas. As I understand it, PCMag mostly collected data in cities, and roughly 20% of the overall weight is placed on data from rural/suburban areas. Since a lot more than 20% of the U.S. population lives in rural or suburban areas, one could argue the national results overrepresent performance in cities. I think this puts Verizon at a serious disadvantage in the rankings. Verizon has more extensive coverage than other networks in sparsely populated areas.
While I’ve been critical in this post, I want to give PCMag the credit it’s due. First, the results for each metric in individual cities are useful and interesting. It’s a shame that many people won’t go that deep into the results and will instead walk away with the less-useful conclusion that AT&T took the top spot in the national rankings.
PCMag also deserves credit for not claiming that its results are the be-all-end-all of network evaluation:
Other studies may focus on downloads, or use a different measurement of latency, or (in Nielsen’s case) attempt to measure the speeds coming into various mobile apps. We think our balance makes the most sense, but we also respect the different decisions others have made.
Understanding networks’ prioritization policies can be tricky. Recently, I’ve spent some time digging through the major U.S. networks’ policies. I’ve created a web page for each network listing which services I expect to be prioritized and which services I expect not to be prioritized. Making sense of each network’s policies isn’t always easy, so it’s possible I’ve misclassified some services. Please contact me if you think any information is incorrect or missing.
A lot of numbers have been thrown out about how many jobs will be created or destroyed if the proposed merger between Sprint and T-Mobile goes through. Here are a few examples I found on the New T-Mobile website:
When dealing with complex systems full of feedback loops, nothing happens in isolation. This fact is often overlooked. I’ll occasionally see an article where a nutrition journalist explains that there are 3,500 calories in a pound. Using some simple math, they’ll explain that since a bag of chips has 200 calories, falling into the temptation to eat a bag of chips every day will lead a person to gain about ten pounds over six months. This is of course silly. By the same logic, eating a bag of chips each day could lead a person to gain 800 pounds over 40 years.
People can’t eat a bag of chips without affecting anything else about their body or their behavior. The body is a complex system. There are lots of feedback loops that regulate things like hunger and energy expenditure.
The economy works in a similar manner. Jobs cannot be created out of thin air. Jobs can’t be eliminated in isolation. If the New T-Mobile hires 10,000 people, it’s not reducing the number of unemployed people by 10,000. The emphasis on job creation is especially strange right now. The U.S. unemployment rate is currently lower than it’s been at any other point in my life.
Google Fi uses an admirably simple pricing structure. A base rate of $20 per month offers subscribers unlimited talk and text. Beyond that, users are charged $10 per gigabyte of data. Single-line plans are capped at a monthly charge of $80, so subscribers that use 6GB of data will pay the same monthly price as subscribers that use 10GB of data. While I like the simplicity of the pricing structure, plans end up being fairly expensive. It’s my impression that Google Fi has had its current pricing structure in place for several years despite the cost per byte of data dropping in the industry at large.
Fi-enabled devices have technology that allows them to switch between T-Mobile, U.S. Cellular, and Sprint’s networks. While the technology is cool, I’m not sure I’d choose seamless switching between three networks with mediocre coverage over exclusive access to Verizon’s more reliable network.
Fi now officially supports devices that are not Fi-enabled. When these devices are used with Fi, they’ll only have access to T-Mobile’s network. Many mobile virtual network operators use T-Mobile’s network and offer far better prices than Fi. For example, Mint Mobile’s plans blow Fi’s prices out of the water. Even with a Fi-enabled device, I think most people can find a better deal. A light user would pay $30 per month before taxes and fees for texts, talk, and 1GB of data on Fi’s network. You could get the same unlimited texting, unlimited talk, and 1GB of data with Verizon’s prepaid service for $30. RedPocket can offer those resources on any of the major networks for $19 per month.
For heavy data users, the case against Fi is even clearer. Using 6+ gigabytes of data brings the Fi monthly bill to $80 before taxes and fees. At that cost, I expect you could purchase an unlimited, postpaid plan with any of the Big Four carriers.
Despite my negativity, I’m still a huge fan of Fi’s simplicity and remarkable international roaming policies. Hopeful Fi will revamp its prices in the near future to become more competitive with the other options on the market.
Earlier this month, Verizon improved its prepaid plan offerings by relaunching the double data promotion it has run in the past. As before, all of prepaid phone plans include unlimited talk and text. Plans differ in the amount of high-speed data allotted each month. Here are the current monthly prices for each plan before taxes, fees, or discounts:
$30 – 1GB high-speed data
$40 – 6GB high-speed data
$50 – 16GB high-speed data
$65 – Unlimited high-speed data
Streaming video will likely be limited to 480p quality on Verizon’s prepaid plans. Unlimited data at low (2G) speeds is available if a customer uses up all of his or her allotted high-speed data.
A $5 per month per line discount is usually available if subscribers pay automatically each month. The discount cannot be applied to the $30 per month plan or the first month of service.
When an initial line is purchased at full price, the following multiline discounts are available:
$10 off — Additional line with 6GB high-speed data
$15 off — Additional line with 16GB high-speed data
$20 off — Additional line with unlimited high-speed data
The current deals appear to be part of a temporary promotion, but I haven’t seen any indication of an end date for the promotion. When the promotion ends, it looks like customers who’ve already taken advantage of the promotion will continue to receive the deal for the lifetime of their plan:
Limited time offer. Available to new activations only. Bonus data added each month as long as your account remains active. Once activated, customers can move between eligible plans anytime and keep the bonus data. Your promotional data allowance will not appear in your shopping cart, but will be seen in your account after activation.
I’m not aware of any mobile virtual network operator (MVNO) operating on Verizon’s network offering better deals to heavy data-users than Verizon is now offering directly. However, consumers that use small amounts of data each month may still be able to find better deals with MVNOs operating over Verizon’s network.
I’ve been trying Verizon’s prepaid service for a few weeks now, and I’m a fan. While Verizon’s prepaid subscribers may be deprioritized in times of congestion, I believe the prepaid service offers nearly the same coverage as Verizon’s postpaid service.
I have a financial relationship with Verizon. As of 5/28/2019, I’m likely to receive a $24 commission when I refer a new prepaid subscriber to Verizon. I share more details about my relationship with Verizon here.
Major U.S. network operators are slowly phasing out their 3G technology. In this post, I share my impressions about the status of the Big Four networks’ 3G phase-outs. For what it’s worth, I’m not confident all of the information is correct, and it’s possible some network operators won’t stick to their current deadlines.
Verizon plans to mostly retire its 3G network by the end of 2020. More details about Verizon’s plans and policies can be found in an entry on Verizon’s knowledgebase
Although the network will largely be retired, FierceWireless explains that some of Verizon’s enterprise customers may continue to have access to 3G service after regular customers lose access.
AT&T appears to have plans to retire its 3G network in early 2022. Here’s an excerpt found on AT&T’s website as of 5/24/2019:
We currently plan to end service on our 3G wireless networks in February 2022.
I’m not sure it’s correct, but I’ve seen an indication that AT&T plans to cease activating 3G-only devices on June 30, 2019.
As far as I know, T-Mobile hasn’t made any announcements about when its 3G network will be retired. I haven’t seen any information about intentions to block 3G device activation.
A document published by Multi-Tech indicates that Sprint may sunset its 3G network on 12/31/2022. I don’t know what source Multi-Tech relies on for that date.
Sprint appears to have had an intention to stop activating 3G-only devices on 4/30/2019. I haven’t confirmed that activations actually ceased.
It’s common to see some consumers giving a cell phone carrier glowing reviews while other consumers publish long rants about the same carrier. When it comes to cell phone service, your mileage may vary.
Cell phone users differ from each other in a lot of ways. It should be unsurprising that a T-Mobile customer in New York City might feel differently about the quality of his T-Mobile’s service than another customer in rural Wisconsin feels about her T-Mobile service. Even within a single geographic region, the way people feel about the quality of a given carrier will vary substantially. People use different phones and do different things with their phones. People are also pretty different. Some people don’t mind if they occasionally don’t have service while traveling in sparsely populated areas. I hate it. Some people are frustrated if their video streaming is limited to 480p quality. That doesn’t bother me.
When choosing cell service, realizing that individual experiences with a carrier will be highly variable can prevent a lot of frustration. Taking advantage of the fact that people’s mileage may vary can allow frugal consumers to save a lot on wireless service.
The frugal strategy
I would guess that there are millions of consumers paying $50-$100 per line each month for postpaid Verizon service that could switch to Mint Mobile plans costing about $20 per line each month without experiencing a recognizable decrease in service quality. While Verizon’s postpaid service is going to generally be better than Mint’s service, many consumers have great experiences with Mint.
Even if you expect to have a good experience with a premium carrier, it may be worth trying low-cost carriers beforehand. There’s a chance the low-cost carriers are capable of providing you with good service. Let’s assume a low-cost, prepaid service would cost you $20 per month while a premium service would cost you $70 per month and require a two-year commitment. If you try the prepaid service first and end up liking it enough to choose it over the premium service, the upside is huge. You can save $50 per month on each line and don’t get locked into a long contract. Over a two-year period, the financial savings come out to $1,200 per line. If you try the prepaid service and disklike it, the downside is pretty low. You experience one month of mediocre service before upgrading to something better.