Say a country has two cellular networks. Network A offers coverage nearly everywhere. Network B offers a much smaller coverage footprint. You could crowdsource data from ordinary subscribers on the networks and track what proportion of the time users are connected to their networks.
If you compare the availability metrics for both networks, I’m unsure what insights you glean. Conventional crowdsourcing doesn’t involve random samples of people within a country. Even if Network B is crummy and has a small coverage footprint, people who use it will tend to live within its coverage footprint. People living elsewhere will opt for Network A and its larger coverage footprint. It’s a classic case of selection bias.
Even if the people you’re crowdsourcing data from represent a random sample of people within a country, a 95% availability metric wouldn’t indicate that a network covers 95% of the country’s land. People concentrate in certain areas. Although urban areas comprise a small fraction of the US, 80% of people live within them.
Opensignal’s Conundrums
Opensignal is arguably the leader in crowdsourced assessments of networks. To Opensignal’s credit, it acknowledges the limitations of its crowdsourced availability metrics:
At some point, Opensignal started publishing another coverage metric called “Reach.” As far as I can tell, Opensignal only used this metric in the US for its 5G assessments.1
Here’s how Opensignal explains Reach:
Reach addresses part of my concern with the crowdsourced availability metric. Interestingly, the metric doesn’t appear in the most recent reports on 5G performance in the US. I wonder why.2
Footnotes
- I’ve not seen the same metric in Opensignal’s broader assessments that are agnostic about the type of cellular technology a network uses.
- Perhaps Reach was a bad metric. Based on Opensignal’s description, it sounds like the geographic areas considered relevant for the metric are specific to each network. Selection bias may creep in as users on different networks visit different areas.
An alternative Reach metric could uniformly account for the same geographic area regardless of the network being assessed. But that would introduce its own issues.
