Mobile: The Measure of All Things (Big Data Week talk)

These are the slides from a talk I gave at Big Data Week London 2014.

We carry take our mobiles with us wherever we go, we keep them fuelled with energy and, typically, there’s no other object we own that is so sensitive to our surroundings.

This makes mobiles ideal for capturing a rich dataset about human activities and the man-made & natural environment around them. In fact mobiles are far more capable than humans as measurement tools – or sensors.

Man vs Mobile sensors

  • Mobiles can sense things we can’t – like magnetism & atmospheric pressure.
  • Mobile sensors can provide quantified and accurate readings (for example “it’s 22 degrees Centigrade”) vs only qualitative ones (“It’s quite warm”).
  • Mobile sensors can exceed human sensors in terms of the range they cover – for example the microphones on devices can pick up sounds invisible to human ears – for example the New Forest Cicada.
  • Mobiles are better at regularly reading and storing information (logging), humans find this boring.

Mobiles are getting more plentiful, being equipped with more sensor hardware, and that hardware is operating at a new level of efficiency. Taken together, this means mobile sensors could become the predominant datasource of the 21st century.

Sensors such as the gas sensor that Sensirion has developed for mobile, mean that mobiles are catching up in the few places where humans have better sensory capabilities. Co-processing or sensor batching, whereby sensors do not need to run on the main system chip, means that taking and saving sensor readings is becoming 1000* more efficient.

At OpenSignal and WeatherSignal we’re exploring the ways mobile sensors can be aggregated into sensor networks, by creating engaging apps and communities of contributors.

Our main aim is to create independent coverage maps and otherwise provide an impartial and data-backed view of how mobile networks perform. In the talk I refer to three different, and surprising, ways the data can be used. We’ve written about two of these elsewhere: how phone batteries measure the weather, how WiFis are used to express political sentiment.

The sensors in phones constitute the most powerful scientific apparatus in the world. Let’s get measuring.

My thanks to: Stewart Townsend, Ande Gregson and Ben Lorica for organising and inviting me along.

EDIT: In case you’re worrying that James sounds like he basically just views humans as inefficient sensors, well… you’d be right, as the below screenshot demonstrates. -SJ  

can a sensor love

Posted in Sensors, Team, WeatherSignal | Leave a comment

UK Roads & Rails

Today we published a report into mobile coverage on UK Roads and Railways,  taking a look at how well served the transport infrastructure of the UK is by mobile connectivity. This is an area that the UK regulator, OfCom, has emphasised as important for quite some time but so far haven’t been able to properly report on because of the limitations of traditional testing methods – especially when it comes to measuring mobile coverage in trains, which themselves have an impact on mobile signal.

Mobile coverage on roads is increasingly important for the success of connected cars (and, longer term, the future of the self-driving car) as these depend on having constant internet access to function properly, so mobile coverage on roads is about more than just being able to stream spotify. For trains, mobile coverage is important because it enables remote working, helping commuters become more productive and lessening the impact of travelling between offices. As the UK spends on improving its rail networks (HS2 and Crossrail being the big examples) it is important that mobile infrastructure can keep up to ensure these give the biggest possible boost to the UK economy.

And now for the overall stats: The average user on a UK motorway has 3G/4G coverage 76% of the time, 67% of the time on A-roads and 72% of the time on UK railways. Click through to check out the routes you use most often, and to see more data on specific 4G performance. All of the data in this report is crowdsourced from approximately 40,000 UK OpenSignal users, to contribute download the app from the header above!

Posted in Mobile Trends, Reports | Tagged , , , , | Leave a comment

Become an OpenSignal iOS Beta Tester

The OpenSignal iPhone app has now been out for over a year – so we feel it’s time for a big update. If you’re an OpenSignal iPhone user, and want to be among the first to try out the new update, then we’d love for you to sign up for our Beta Test program. Just sign up through this form and we’ll send you all the instructions you need to get the brand new OpenSignal on your phone. One of the most exciting new features we’re introducing is network comparison, we’ll show you how your network has worked for you in the places you’ve been, and compare it to the others available – so you can definitely know which network will work best for you, based on the places you actually spend time.

The new version is enormously different to the last one, both in terms of looks and functionality – and we really hope our Beta testers will be able to play a big role in shaping how the app works, to ensure that we’re able to offer the best possible tool for helping people to improve their connection. In order to get you even more excited – here are the three biggest changes to the OpenSignal app that you (hopefully) know and love:

1)   Completely overhauled UI – it’s iOS7-friendly, flat and beautiful. Lots of blue and white, like it was inspired by the ocean. Sharing your speedtests/coverage stats is a lot more intuitive – making it easier to complain about your network/show off your fast Wi-Fi speeds.

2)   Background data collection– this allows you to contribute much more information to our coverage maps, making our iOS users a much more effective part of the biggest signal crowdsourcing network in the world. By using the OpenSignal app you are helping to make coverage information available for everyone, meaning that carriers can no longer hide behind inaccurate coverage maps – and everyone can access accurate coverage information before they buy a phone or switch networks. As ever, all background (and foreground) data sharing can easily be turned off in the settings by flicking a single switch.

3)   Bigger and Better stats – one of the really exciting new features we can bring you through background data collection is recommending you networks based on where you’ve been. The app looks at your connection over time and is able to compare your personal coverage with other networks (based on our extensive coverage database, built up over time from over 7 million OpenSignal users).  You never again need to say ‘will my connection be better on another network?’ the answer will be right there in your phone, written in white and blue in the OpenSignal app.

If that sounds good to you then please sign up for the OpenSignal Beta Tester Program – give us feedback, play with the app, try and break it, let us know what you like and don’t like. We’d love to hear from you – keeping our community involved is really important to us so sign up, download and get in touch!

Posted in App Update | Tagged , , , | Leave a comment

Help Needed: Mapping LTE Bandwidth

There are multiple factors that influence the data speed you experience: the device you use, the network operator, the time of day, the signal quality where you are. We’re building models of how these factors come together, we’ll be publishing our results and it’s our hope that they will guide the industry on how best to improve user experience.

We’re focussing on LTE currently because there are a lot of signal quality variables accessible. But, one important factor is missing from our model.

The LTE bandwidth is the quantity of spectrum over which the device sends information. There should be a roughly linear relationship between bandwidth and speeds, though it will also depend on the band used (which gives the area of spectrum being used – the frequency).

LTE bandwidth and band are not available through Android APIs. Some devices have have unpublished APIs which do reveal it, these devices are relatively common in South Korea and Hong Kong where we have a good picture of the bandwidths.

We need you help
We generally don’t ask for any help beyond downloading the app, but we’re stuck on this one. We want to run our model on more countries – the US in particular. But we don’t know what bandwidths are being used in each city.

The good news is, many phones are able to see what bandwidth is available. You just need to enter the service menu or field test mode, on Samsung Galaxy devices (e.g. Galaxy S4) just open the dialler and type:

*#0011#

This should open up a page of stats on your device:

GS4_1

See below for what to do if *#0011# does nothing

Here you can read the LTE stats easily. The Band is 7 (around 2600 MHz), and the LTE Bandwidth for download (LTE DL BW) is 15 MHz. Actually the Earfcn_dl gives a more precise reading of the band, 2825, so if this available use this.

If that works fill out this form and your information will be automatically mapped:

All the information is public, and responses are mapped immediately below – you will probably need to refresh this page before your data becomes visible. If it doesn’t appear after that there may be a problem geocoding please email me James (@ opensignal.com) with your observation and we’ll get it up.

Map of LTE bandwidth. Click on a dot to see the observation. Get the raw data here.

Apart from improving our model, this map will give a great indication of how operators are investing in infrastructure. For example T-Mobile stated in Q4 2013 that it would have brought 20+20 LTE (i.e. 20 MHz downlink and 20 MHz uplink) to 90% of the top 25 US markets by the end of 2014 – that’s 24 cities. Readings from T-Mobile users shed light on how that roll out is doing.

Sprint’s Spark network – making use of band 26 (800MHz) is also one to watch.

If *#0011# doesn’t work
*#0011# won’t work on all devices. You can try:

*#*#4636#*#*
*#32489#
or
*#*#7262626#*#*
However, we’ve tried these on a number of phones, and even when they do return signal and tower information we have not found them to return the bandwidth.

If you find other codes that do work, get in touch.

Posted in Help Needed!, Open Signal Maps community, Understanding signal | Leave a comment

The OpenSignal Walk

Every afternoon we take a company walk. There’s no set time, no pattern for its occurrence – someone just initiates it when they feel they’ve had enough of looking at a screen for an afternoon. It usually begins with a stretch of the legs or arms, a surreptitious glance across the table, a half nod. Then someone takes off their headphones, pushes their head above the parapet of congenial work-silence, and asks the single question:

Walk?

And that’s it – headphones are out, and concentrated stillness (though often hiding a scurrilous amount of gossiping going on over Slack) is replaced by a communal unfurling. We wander out, chatting about whatever, and stroll through Clerkenwell, usually without any particular destination in mind. We had a phase of going to look at one of the many nearby churches, including numerous repeat visits to St. Ethelreda’s Chapel – distinguished by a particularly exciting crypt. Though since the onset of spring we’ve tended to direct our walks to maximise exposure to sunlight – although that’s less of a problem when you’ve spent the morning working on the White Bear Yard roof terrace.

st ethelreda's

St. Ethelreda’s, Ely Place

So how did this tradition come about? In the best standards of all tradition, repetition has utterly obscured origin – but it serves as a constant reminder that there are more important things than being tied to a keyboard all day. An acknowledgement that if the sun is shining then a stroll and an ice-cream will only improve everyone’s mood and concentration, rather than break it.  There is always a lot of talk about ‘company culture’ in the start-up world, though this often tends towards the promotion of individualism above all else (Google’s 20% time, not tracking holidays, working from home). While we also love this relaxed laissez-faire approach – working from home is always fine and we all get to work around mid-morning – we view the walk as symbolic of the collaborative environment we want OpenSignal to be.

The ‘company culture’ (an admittedly horrible phrase that unapologetically wears the taint of middle-management doublethink) we want is one where everyone is involved and intellectually interested in every aspect of the innovative work we do (mainly in the fields of crowdsourcing, sensor networks and big data), from both a technical and theoretical perspective. The walk, a moving coterie of people having the physical and intellectual freedom to unwind, think and discuss, exactly sums up how we want OpenSignal to be.

FUN BOUNCY

Joe and I get overexcited on a recent walk

Recently, especially, walking has been held up as a panacea to the ills of modernity, a rare opportunity to stimulate both our bodies and minds in one move – like some kind of historically-inspired ambulatory yoga. The best thing about a walk, in many ways, is the pedestrian exposure to the element of change – every walk is different, in part because the environment in which it is embedded shifts in ways that range from the dramatic to the almost-imperceptible. That is the joy of the walk, the routes are familiar enough to emphasise even subtle change and every time we talk about something different, reminding us that it is the human geography that matters more than the physical.

Walks keep us healthy, curious and calm – and above all have an interesting factual and literary history. Many of Jane Austen’s most important moments of crisis/narrative development occur during walks (Louisa Falling from the Cobb in Persuasion, Harriet being rescued by Frank Churchill in Emma) – showcasing the walk as a conduit for change and excitement. On the other side of the coin we have Immanuel Kant, famous for his daily contemplative walk around Konigsberg – interrupted only once, by his inability to put down Rousseau’s Emile.  Occasionally we also get too distracted by what we’re working on to go for a walk – but that is rare, because, almost no matter what you’re doing, it’ll prove to be less important than pausing, stretching your legs in the sun and arguing about what defines a sensor network. After all, if it was good enough for Kant, it’s probably good enough to be a universal maxim.

If you like sound of our attitude to walking (and since you’ve persevered to the end of this blog post) you might well be a great fit at OpenSignal – come and join us!

Posted in Team | Tagged , , , , | Leave a comment

State of LTE: Germany

Since our Global State of LTE report, which we released earlier this year, we’ve been covering the countries included in that report with more detailed market-specific reports. This week our focus turns to LTE performance in Germany, a country that first rolled-out LTE late in 2010. In the report we cover the three main German LTE networks (E-Plus rolled out LTE 2 months ago, but that is still too recent for us to have meaningful reliable data, so it is not included).

The results are extremely interesting, with clear differences between the networks for both ‘Time on LTE’ and LTE download speed, with O2 coming top for download speed (as you can see from the graph below) and Telekom.de coming top for coverage. Click through to see the full report.

Screen Shot 2014-05-07 at 15.30.53

Posted in Uncategorized | Leave a comment

Which Network?

We are delighted to announce a new partnership with Which?  - The leading consumer advice magazine in the UK. Which? has a long history of helping consumers to make the best decisions about what to buy, from cars to washing machines, and we are delighted to be able to help them advise cell phone buyers on which network will perform best for them in the areas that they intend to use it most.

Which? are more than just a magazine, and serve as an important consumer rights advocacy group, protecting individuals from the organisations they deal with on a daily basis. Which? help protect consumers from power and information imbalances that are present in markets, and therefore we consider them a perfect fit for us. We started OpenSignal to correct a problem of imperfect market information; mobile phone users didn’t actually know how the network they were signing up with would actually work for them.  By publishing independent, crowdsourced, maps of mobile coverage we are able to put some power into consumers hands by helping them to understand the previously unknowable – the true state of mobile coverage in the areas where they live and work.

The OpenSignal coverage checker

The OpenSignal coverage checker

Consumers can now get information on handsets, network price and network performance all in one place, a fantastically useful resource for anyone planning on, or thinking about, buying a mobile phone.  We hope that by combining our maps with the consumer expertise of Which? we will be able to get closer to our aim of making network performance information an integral part of any mobile purchasing decision. This deal with Which? brings us a step closer to fixing the imperfect information problem that has led to consumers signing up with networks that won’t give them the best service.

You can see our coverage maps on the Which? website here, where they are used on the page which helps users make purchasing decisions that are related to telephony. We hope that this partnership will allow our maps to help an ever broader range of people, and that when people are considering buying a new phone or changing network they will no longer simply ask which network gives me the best price? But also which network will give me the best coverage? 

Posted in Understanding signal | Tagged , , | Leave a comment

Join the OpenSignal and WeatherSignal Beta

We’re making it easier for the OpenSignal/WeatherSignal Android community to access and test the latest features by introducing a Google Play Beta Program.

By joining the Google+ beta communities and then opting-in to receive updates, you’ll get the latest development versions of our apps delivered automatically to your devices.

For OpenSignal
Join the Google+ community here
Then opt in to get Google Play beta updates

For WeatherSignal
Join the Google+ community here
Then opt in to get Google Play beta updates
We’ve got some really fun new features going into WeatherSignal – rich contextual awareness (Atmos speaks!) and you can now collect data on how far you’re walking (not shown in UI, but you can export to CSV).

It’s a little complex (clicking two links, I know!) but you only have to do it once :)

For iOS users, you can sign up here – we have a big overhaul on iOS cooking, it’s looking gorgeous and working lovely but there’re still a few things we’re scratching our heads over.

Posted in App Update, Help Needed!, OpenSignalMaps Android App, Quantified Self, WeatherSignal | Leave a comment

Does the Galaxy S5 have temperature and humidity sensors?

tldr; No.

The Galaxy S4 surprised the world by adding Sensirion’s SHTC1 chip – a tiny, low power chip that can take accurate measurements of relative humidity and ambient air temperature. These readings are used in S-Health, we also collect data from the chip in WeatherSignal – a real-time weather crowdsourcing app (join the biggest mobile meteorological network: download the app).

Much has been made of two of the Galaxy S5’s new additions: fingerprint & heart-rate sensors. An Infra-red gesture sensor – which could possibly be hacked to provide surface temperature readings – and RGB ambient light sensor have also been added, these will likely have more of an impact as they passively monitor the environment while fingerprint/heart-rate requires effort from the user. But what of temperature and relative humidity?

Sources have been a bit confused, the official S5 site does not mention these sensors but GSM Arena, generally a reliable source for phone specs does list humidity and temperature sensors. So we decided to take a look at our own data.

Although the S5 is not officially released for another 2 days, we’ve seen numerous S5’s sending us data – almost 70 in fact, from 15 countries (Korea we expected to see as the S5 is already on sale there, the US, Israel, Brazil are also included). Among the data we collect is a one-off scan of device specs, this forms the basis of our Android Fragmentation reports, we also provide this data to device testing firms and OEMs.

Across all 69 Galaxy S5’s, covering 9 distinct precise models (e.g. SM-G900L, SMG900V) not a single one provides humidity or temperature APIs. Unless Samsung has included these sensors but made them invisible to developers – which would be perverse – these sensors are not present.

It’s highly unusual to see an OEM removing a sensor. We’d love to be proven wrong, if anyone knows different, get in touch.

UPDATE: A few people have pointed this out, I think Geetee on Hacker News was the first: “It’s most likely due to the water-proofing. The S4 rugged version has the same limitation.” Makes sense – air temperature and humidity sensors have to be exposed to the air, perhaps hard to square that with water-proofing.

UPDATE 2: After talking with Dominic Boeni of Sensirion, it seems that waterproofing is not the problem, indeed many Japanese phones with SHTC1 humidity/temperature sensors are waterproofed to IP68 standard – you can leave them an hour at a depth of 2m.

Posted in Sensors, WeatherSignal | 5 Comments

US LTE performance: T-Mobile fastest, Verizon best coverage

Yesterday we released a report into the state of LTE in the US, with a particular focus on how network performance changes over time. Mobile networks in the United States have been under particular scrutiny lately, with Verizon and T-Mobile publicly sparring over who has the best network and the way those tests were measured.

See the full report here.

One of the specific criticisms levelled at Verizon by T-Mobile was that the data used in their adverts was out of date – and so our report is averaged over the last 3 months.  What we see is that, for the US as a whole, T-Mobile have comfortably the fastest speeds with an average download speed of 11.5 Mbps.  Contrasting with this, we see that Verizon have the best LTE coverage – with the average Verizon LTE user spending 83.2% of the time with access to the network. Sprint come bottom for both metrics, showing how far their network lags behind their competitors but also the extent to which they committed to the slower HSPA+ technology (which we class as a form of 3G, even though it is often marketed as 4G).

Finally the report looks at how networks are changing over time, with the overall trend being that networks are improving for LTE coverage but that their speeds are generally getting slower. This is understandable, as more users slow down the network but operators continually roll out to new areas – providing better coverage for their subscribers.

Screen Shot 2014-04-01 at 13.12.22

Posted in Mobile Trends, Reports | Tagged , , , , | 4 Comments