Introducing OpenSignal’s new data collection methodology

Starting this week, you may notice some changes in the reports, coverage maps and other data we publish. OpenSignal has made updates to the way we collect our data from smartphones and the methods we use to parse that data. We began implementing these changes with updates to our smartphone apps in April, and we will be debuting the first results of that new methodology this week in our State of Mobile Networks report for the U.S.

Photo courtesy of Flickr user Cheryl1906

We’re still publishing the same core metrics we always have. The U.S. report will highlight LTE availability, 3G and 4G speeds and latency, and overall speeds across operators’ networks, but in coming reports we’ll add new metrics, allowing us to examine network performance more closely.

The update gives us more than just new metrics to play with, though. By improving our data collection methods, we’re now able to collect more individual measurements on each device in our crowdsourced community as well as hone the precision of those measurements. In many cases, the update merely refines our current methodology, and you won’t notice any difference in any of our results. But in other cases, we’ve made adjustments to what we actually measure in order to isolate the typical consumer experience more effectively.

For example, let’s take a look at the changes we’ve made to our speed test methodology. There are numerous ways to measure mobile network speed. You can measure speed from the phone to the base station, to the network, to an internet server a few miles away or a server on a different continent. Those are all valid ways of testing speed, but they can all yield vastly different results as each additional network hop introduces new bottlenecks into the end-to-end connection. Operators and ISPs often keep their testing entirely within the confines of their own networks, which is useful for diagnostic purposes or to measure peak speeds. But OpenSignal has always conducted our speed tests against the same internet servers mobile users access every day to surf the web, stream video and download content. We feel those measurements most accurately represent the typical consumer mobile data experience. That hasn’t changed, but in our updated methodology we’ve added many more test servers in new locations. We feel the more servers we test against, the more we can represent the incredible variety of internet destinations in our measurements. With more servers, though, the average distance between server and test device is now smaller, resulting in an overall uptick in our average speed measurements.

We’ve also made tweaks to our collection methods in other areas to improve the precision of our results. For instance, as with our speed test we’re now running our latency tests to more test servers to get a more rounded view of a network’s reaction time at any given moment. We’re now also able to drill more deeply into Wifi data, which will allow us to gauge the increasingly large role Wifi plays in mobile services.

Ultimately these changes give us a wealth of new data to work with, and help us do an even better job evaluating the performance of networks around the world. While these updates will have no impact on the network rankings of our previous reports, in order to maintain analytical consistency, we won’t be making any direct comparisons between specific results collected under the two different methodologies.

So stayed tuned. In the coming months, you will start seeing these new metrics show up in our public reports. More immediately, you should soon see a big update to our coverage maps on  the OpenSignal website and in our smartphone apps. With more data at our disposal, our maps will show many more details of where networks are available and where there are notspots.

While this new methodology will give us more data, nothing can replace the growing community of users who collect our data on daily basis. If you haven’t already, we encourage you to join OpenSignal’s crowdsourcing effort by downloading our Android or iOS app. Those apps will not only give you personal insight into how your own mobile service performs, but by using them you would contribute to our database of hundreds of billions tests.

This entry was posted in Crowdsourcing, Map Update, Networks, OpenSignal app, Reports and tagged , , , . Bookmark the permalink.

2 Responses to Introducing OpenSignal’s new data collection methodology

  1. Mariano says:

    It’s a pity you’re not showing 2G/3G separately anymore, in many areas of developing countries there is no 3G, and having the coverage in a separate map gives much more important information.

  2. Pingback: 4G, in Italia la connessione più veloce è quella di Vodafone - NewsRss24 Italia

Leave a Reply