GWS’s OneScore Methodology & 2019 Results

Christian Mobile Phone Service

Global Wireless Solutions (GWS) evaluates wireless networks according to the company’s OneScore methodology. At the moment, AT&T cites GWS’s results in commercials where AT&T claims to offer the best network.

In an article about performance tests of wireless networks, GWS’s founder, Dr. Paul Carter, writes:1

With so many conflicting research reports and with every network touting itself as number one, it’s critical that wireless carriers are transparent about how and what they actually test. If what was tested doesn’t match up with the average consumer experience, then was that test truly worthwhile?

Unfortunately, GWS itself is not especially transparent about its methodology. The public-facing information about the company’s methodology is sparse, and I did not receive a response to my email requesting additional information.

As I understand it, GWS’s methodology has two components:

  • Technical performance testing in about 500 markets
  • Consumer surveying that helps determine how much weight to give different metrics

Technical testing

In 2019, GWS conducted extensive drive testing; GWS employees drove close to 1,000,000 miles as phones in their vehicles performed automated tests of networks’ performance.2

The drive testing took place in about 500 of the markets, including all of the largest metropolitan areas. GWS says the testing represents about 94% of the U.S. population.3 I expect that GWS’s focus on these markets limits the weight placed on rural and remote areas. Accordingly, GWS’s results may be biased against Verizon (Verizon tends to have better coverage than other networks in sparsely populated areas).

Consumer surveying

In 2019, GWS surveyed about 5,000 consumers to figure out how much they value different aspects of wireless performance.4 GWS finds that consumers place a lot of importance on phone call voice quality, despite the fact the people are using their phones for more and more activities unrelated to phone calls.5 GWS also finds that, as I’ve suggested, consumers care a lot more about the reliability of their wireless service than its raw speed.6

Combining components

As I understand it, GWS draws on the results of its surveying to decide how much weight to place of different aspects parts of the technical performance tests:

The consumer survey includes questions asking respondents to rank the importance of different tasks they perform on their mobile device, as well as the importance of different aspects of network performance. Our network test results are then weighted according to how consumers prioritize what’s important to them, and evaluated in eleven different network performance areas related to voice, data, network reliability and network coverage.

The methodology’s name, OneScore, and the graphic below suggest that the company combines all of its data to arrive at final, numerical scores for each network:7

GWS OneScore Visual

Oddly enough, I can’t find GWS publishing anything that looks like final scores. That may be a good thing. I’ve previously gone into great detail about why scoring systems that use weighted rubrics to give companies or products a single, overall score tend to work poorly.

2019 Results

In GWS’s 2019 report, the company lists which networks had the best performance in several different areas:

AT&T:

  • Download speed
  • Data reliability
  • Network capacity
  • Video streaming experience
  • Voice accessibility
  • Voice retainability

T-Mobile:

  • Voice quality

Verizon:

  • Upload speed

Open questions

I have a bunch of open questions about GWS’s methodology. If you represent GWS and can shed light on any of these topics, please reach out.

  • Does the focus on 501 markets (94% of the U.S.) tend to leave out rural areas where Verizon has a strong network relative to other operators?
  • Do operators pay GWS? Does AT&T pay to advertise GWS’s results?
  • What does the consumer survey entail?
  • How directly are the results of the consumer survey used to determine weights used later in GWS’s analysis?
  • What does GWS make of the discrepancies between its results and those of RootMetrics?
  • How close were different networks’ scores in each category?
  • GWS shares the best-performing network in several categories. Is information available about the second, third, and fourth-place networks in each category?
  • Does GWS coerce its raw data into a single overall score for each network?
    • Are those results publicly available?
    • How are the raw performance data coerced into scores that can be aggregated?

Footnotes

  1. Demystifying Mobile Network Testing: Part 1 by Dr. Carter is archived here.
  2. “GWS’s team of network testing specialists drove nearly one million miles across the US conducting scientifically controlled tests of network performance. Network specialists drove routes within specified geographic regions based on a weighted algorithm that reflects population demographics, targeting areas likely to have higher mobile usage.”
    From GWS’s 2019 report on U.S. networks.
  3. “Global Wireless Solutions conducts annual ‘consumer experience’ tests in over 500 markets representing 94% of the US population.”
    From GWS’s 2019 report on U.S. networks.
  4. “All 2019 survey figures are from a study that was commissioned by GWS and conducted by Toluna. Total sample size was 5,221 adults. The survey was undertaken online in June 2019. The figures have been weighted and are representative of Americans (aged 18+).”
    From from GWS’s 2019 report on U.S. networks.
  5. “Making and receiving phone calls is the most common activity and considered the most important by consumers.”
    From from GWS’s 2019 report on U.S. networks.
  6. “Consumers find network reliability far more important than network speed. In fact, they consider it to be more than twice as important.”
    From from GWS’s 2019 report on U.S. networks.
  7. The image comes from GWS’s 2019 report on U.S. networks.