Mobile Sensor Networks: Creating a Social Laboratory
by Samuel Johnston & James Robinson

'We find strewed over its irregular and far-spread surface, patches of cultivation upon spots chosen without discrimination and treated on no common principle … Meteorological instruments have been for the most part treated like toys'
From the first published report of the British Association for the Advancement of Science, 1833.

The rise of the mobile sensor network is one of the most exciting technological innovations of recent times. Billions of mobile phones are now capable of participating in networks to collect valuable data from the physical world and its intersection with human behaviour. Much like the earliest meteorological instruments, so far smart devices have been used as toys and tools equally, whether one is using them to access Google Maps or to play Angry Birds, but their potential to form part of a global network of connected sensors is only now being explored.

25% of Android devices globally now contain a barometer - 300 Million potential nodes for a barometric sensor network
Source: The OpenSignal Sensor Library

The proliferation of sensors in mobile devices, from barometers to luxometers, means that cell phones are now capable of being more than simply toys or tools for their immediate user. The rise of the smartphone means that the face of the earth is being covered in a dense network of environmental and behavioural sensors (such as the GPS chip which tracks the user’s movement) capable of reporting back almost instantaneously to a centralised location. This network of billions of connected devices may be considered the world’s most powerful piece of scientific equipment, offering unmatched potential to measure the built and natural environments and capable of transforming the ways we gather information about the world.

Sensors measure physical properties. From lodestones, to weathervanes and sundials, they have a long and varied history of use. The simplest definition of a sensor is anything that translates an external property to data, frequently a property that is precisely unquantifiable to an individual without the use of the sensor – although the property itself is often acknowledgeable[1]. With the development of the electronics industry, sensors have become increasingly more accurate and cheaper to produce. Ten years ago, you would already have had dozens of sensors in your house: smoke and carbon monoxide alarms, and even the doorbell and light switches. Now, however, they are in everything – from fridges and cars, to mobiles and wearables.

So while sensors in general are not new, four recent trends are remodelling the landscape:

These four processes have the transformed the capability, range and potential purpose of sensor networks. Sensor networks are not new, but sensor networks at the distributed scale made possible by mobile phones are a new phenomenon. This report traces the history and development of the sensor network, from the lone sensor up to the modern mobile sensor network. Historically one of the main drivers of sensor, and sensor network, development has been meteorology - appropriate given our own attempts to build the world’s most comprehensive meteorological mobile sensor network: WeatherSignal. For this reason, many of the main historical and current examples drawn on in this paper to illustrate the process which has led to the current feasibility of the mobile sensor network are taken from the history of meteorology, and humankind’s attempts to measure atmospheric conditions with increased accuracy.

Sensors At Scale

There are more sensors in the world today than at any point in history, and several recent trends are serving to increase the speed at which that number is increasing.

Smartphone Adoption

The clearest example of the global proliferation of sensors is the rise of the sensor-laden smartphone. One in five people on the planet currently own a smartphone, a ratio that is increasing rapidly. These devices are only becoming ‘smarter’, the typical modern smartphone contains many more sensors than its older versions, so sensor proliferation through mobile phones is being compounded by the increase of specific sensors included in each device. Sensors offer OEMs another way of differentiating their offerings; which is perhaps clearest in Samsung’s progressive addition of sensors to the Galaxy S series of phones.

Fingerprint scanner
Heartrate
RGB Ambient Light
Relative humidity
Env. temperature
Barometer
NFC
Gyroscope
Accelerometer
Bluetooth radio
WiFi radio
FM radio
Cell radio
Front camera
Rear camera
GPS
Magnetic field
Light flux
Battery temp.
Microphone
Touch
S S II S III S4 S5

The Internet of things is an Internet of sensors

Sensor proliferation is far from limited to smartphones, with sensors increasingly being included in an ever-widening variety of places. The adjectives ‘smart/connected’ are often applied to traditional machinery that now contains sensors and is networked – a state which is referred to as the ‘Internet of Things’, a communicative network that requires no human intervention once set-up to function. Alongside mobile devices we see wearables, connected cars and smart buildings in the vanguard of the ‘Internet of Things’ – consolidating the presence of networked sensors.

The potential importance of the sensor data that can be gathered from these networks cannot be understated, even if it initially appears as a by-product of the device’s functionality. Google’s recent statements suggest they see sensor data as not merely epiphenomenal to wearables - a useful side effect - but as a defining feature. Sundar Pichai, head of Android, announced at SXSW:

Wearables, mobile phones and smart devices are forming a 'world of sensors'

"When we think of wearables, we think of it as a platform. We see a world of sensors. Sensors can be small and powerful and gather a lot of information that can be useful for users. We want to build the right APIs for this world of sensors.”

Indeed, unlike smartphones, the principal utility of many wearables - Fitbits, Fuelband, Narrative Clip - is the automated collection of data. By cutting out all the other hardware these wearables are able to gather data at a resolution that smartphones can only achieve with severe sacrifice of battery life. When we look at logging, we will see how changes in smartphone design are set to make sensory systems much more efficient, and it is possible that such improvements will reduce the marketshare of activity tracker wearables. However, new types of wearables such as Motorola ATAP’s “Digital Tattoo” (a circuitboard temporarily tattooed to the skin) will capture data fundamentally off-limits to smartphones and tablets as they exist in their current form.

Sensor Repurposing: how sensors measure more than you think

“Sensor repurposing” is a term we coined to describe how sensors may be put into a device for one purpose, but their data used for entirely different means. One of the clearest examples of sensor repurposing is in our making use of mobile barometers (included in high-end smartphones to improve GPS positioning) to collect that pressure data for meteorological purposes in the WeatherSignal project. Sensor repurposing is the great advantage that using mobile sensor networks gives to researchers, as it reduces the capital expenditure (a barrier to entry to doing sensor-based environmental research) to almost zero, as it allows for the repurposing of an existing network rather than the deployment of a new one. One of the main benefits of using sensors in mobile devices is that they are non-proprietary to the project using them – one mobile barometer might be accessed by 15 different apps simultaneously, so it could be used as part of 15 distinct sensor networks. While a phone may contain only one physical barometer, it can operate as numerous virtual ones, contributing to a variety of projects. This has a huge impact on scale, with each sensor counting for each of the systems making use of it.

Repurposing existing sensors reduces the capital expenditure involved in setting up a sensor network to almost zero - reducing the main barrier to entry in doing sensor network-based research

Measurements from battery temperature sensors (present on all Android devices) can be used to measure outdoor ambient temperature when taken from many devices. The light sensor is present on many devices in order to regulate the screen brightness, but its readings also give clues to the weather (overcast, sunny, misty). Similarly the cellular radio can be thought of a sensor, its intended purpose is to enables call, SMS and data transfer, but with OpenSignal we’re using it to measure the quality and availability of mobile networks.

The strength of the mobile phone as both a device for personal scientific investigation, as well as being a part of a large-scale low-cost sensor network, is its versatility. Two great examples of apps that add new hardware to increase the feature-set of the smartphone are Square and Vaavud. With Square a small card-reader is attached to the headphone jack, the electrical pattern of a magnetic strip being swiped is sent to the device via the jack. Vaavud is perhaps even more remarkable; it is a cup-anemometer (or wind-meter), also attached to the headphone jack, that contains no electronic parts, rather the cups are magnetised and their frequency of motion results in a periodic distortion of the magnetic field around the device. This magnetic distortion is measured by the Vavuud app and interpreted as wind speed.

Both Square and Vavuud rely on some external hardware to interact with the phone’s sensors, which is no means necessary for making use of a smartphone sensor network. They do, however, show the extent to which devices can become even more sophisticated, and therefore an even more valuable resource for the gathering of data. As more sensors are put into phones, multiplier effects will be discovered as data from sensors can be used in combination. For example, the light sensor might be used to give clues as to whether a device is indoors or outdoors, this information could then be used to clean the dataset of temperature readings – by determining the environment in which the individual reading was taken. By combining these readings it would make it possible to easily exclude all indoor readings, meaning that the network would be more effective at reporting on outdoor ambient air temperature.

The number of sensors in the world has grown even faster than smartphone and tablet shipments, through multiplier effects caused by projects using the same sensors and the proliferation of sensors per-device. Scale has transformed the way we are able to study the world through sensors, the sheer number of them means that much more data is available than ever before.

Networking

When taken in isolation, readings of a single smartphone’s battery temperature tell you next to nothing about the temperature outside. However, it has been proven that, by taking readings from many devices, it is possible to extract reliable and accurate measurements of ambient outdoor temperature. The wider importance of this result goes beyond meteorology: it is a proof of concept of how messy - “unscientific” (i.e. not lab conditions) - data can be used to derive valuable insights, a concept often discussed under the banner of ‘Big Data’, a notable buzzword du jour.

Signal scales differently to noise. In the battery temperature example, the signal is produced by outdoor ambient temperature while the noise is produced by indoor temperature, the proportion of time spent in different locations and the particular device model. For one device, noise greatly outweighs signal, but crucially noise does not add up in the same way as signal: across many devices the signal can be heard.

The development of big data technologies has allowed for new data sources to become useful, allowing the signal to be heard more easily amongst the noise

Outdoor air temperature is not unique in this property, it is true for many classes of dataset that signal scales proportional to the size of the dataset, while noise scales proportional to the square route of the size of the dataset. This has far-reaching implications; too frequently it’s impossible for researchers to test their hypotheses due to lack of data. An intriguing study suggested people changed their behaviour on brighter days (becoming more risky), but it is only now possible to measure the movements of millions of people and contrast it with the light levels they experience.

With the development of cheap data processing technologies (‘Big Data’) it has never been easier access the infrastructure needed to make sense of the vast datasets sensors are starting to produce and this trend, and technology, is part of the reason that the mobile sensor network has taken off. The mobile sensor network is more versatile and generally at such a greater scale than a purposefully-deployed network that it allows for whole new kinds of analysis to become possible. Data is valuable, even when it is ‘messy’ and this has fuelled the rise of networks that are more concerned with volume than with cleanliness – leading to the testing of hypotheses that would have been impossible using traditional methods. For the first time in human history it is possible to measure things at scale, rather than model them through sample populations – billions of people around the world can be accessed simply by the building of a single app. New technologies, both on the backend and in users’ pockets – has transformed the world into a mesh of networked devices – capable of sensing and reporting instantaneously.

A force to unite data

Smartphones alone account for over a billion sensor nodes. Considering Galaxy S4, Galaxy S5 and iPhone 5S sales at the end of 2014 over 100M devices will have co-processors, enabling highly efficient data collection from several sensors (including accelerometers, gyroscope, magnetometer, location, light, colour, pressure) allowing of order 1000 readings per hour to be taken on each sensor. Networking is the gravitational force that unites the data from these.

Data networks are far older than the Internet, and have been used to link sensors for hundreds of years. The Medici sponsored a weather collection network with its hub in Renaissance Florence. British navy ships - with their rigorously kept observational weather logs - can also be considered a sensor network; while the telegraph brought about a step change since it “for the first time allowed weather data to travel faster than the winds themselves” - an elegant framing from Katharine Anderson.

Robert Halley, creator of the first meteorological map ‘the map of trade winds’ in 1686, relied upon British naval logs. The weather was obviously of primary importance to ship’s captains - as it was quite often a literal matter of life and death. Maps of historical weather conditions however, while interesting, are not the best way of demonstrating that ships were a proto-sensor network. Halley’s map of prevailing winds is an excellent example of where collected data from nodes could be centrally compiled into a useful resource, with the sensor in this case as simple as a flag showing wind direction. A more sophisticated example that ships could be used as sensor networks, and one which nicely shows that ships – like mobiles – can be understood as containing a bundle of different sensors, is in early attempts to map the earth’s magnetic field.

Repurposed sensor networks are not new, with naval vessels an early example. Navy log books were used to map both typical wind direction and the earth's magentic declination

Declination (the amount magnetic north differs from true north) is obviously of immense importance to seafarers, often guided purely by compass-point after daylight hours. As Jonkers (2008) put it ‘It was primarily seafaring that exposed ever more of Earth’s peculiar magnetic features, prompting formal representations to follow suit’. The first notation of declination was made by Portuguese sailors in the late 16th century, although the phenomenon was not fully understood at the time. The first actual maps of the earth’s magnetic field were gathered from ships who regularly logged the difference between true and magnetic north – making it possible for this data to be analysed on their return to port.

While these maps were not particularly accurate - as modern scholarship has shown - they still grasped the fundamental idea of a varying magnetic field across the surface of the globe. One such early cartographer, Nicholas Cruquius, had access to the navigational logbooks of the Dutch East-India Company, from which he made his conclusions about the state of the earth’s magnetic field early in the 18th century – putting the newly-available network of roving compasses to effective scientific use. By using the compass-data collected from over a thousand Dutch East India Company ships, Cruquius turned the world’s largest trading fleet into a sensor network of remarkable size and sophistication.

The oldest sensor network, the Cheugugi rain-gauge network of the Joseon Dynasty in Korea created in 1442, was one of the simplest and most effective. The Cheugugi was a fantastically simple network, essentially a series of buckets. Despite its simplicity, it included two great innovations: it allowed for standardised measurement of a quantity (rainfall) and it allowed for the centralised collection of this data. For hundreds of years the Cheugugi was used across Korea, its data forming the basis of tax calculation on land[2].

The networking of sensors, whether automatic or manual, allows for large amounts of data to be harnessed and put to central use – elevating the sensor from the restricted sphere of personal scientific investigation to part of a wider enterprise.

Future sensor networks will be built by consensus, not by mandate

The creation of the Cheugugi rain-gauge network contributed to a more equitable system of taxation, but the system - while conceptually simple - required a mandate from the king. Every smart device is connected - even if its connection is intermittent - and is therefore capable of forming a node in a sensor network. The way these networks are built in the future will be an inversion of the Cheugugi network, the mandate will not come from the top, but instead individual device users will decide to participate based on the individual and collective utility of the network – in other words, the network will be crowdsourced.

Crowdsourced sensor networks, which are built through an open call for participation, must provide utility to the user in order to incentivize use. In many ways crowdsourcing is the only way that mobile sensor networks can be accessed by third parties, as they have no prior claim on the sensors they hope to make use of. Waze, Moovit, Sensorly, Rootmetrics, PressureNet and our apps OpenSignal and WeatherSignal are among the clearest examples of apps that try to offer users something in return for their participation in a sensor network. In the case of Waze and Moovit the return is an immediate optimization of a user’s journey. For OpenSignal the returns are twofold for the user: an immediate way of seeing how their data network is working (and some clues as to how to improve it, such as nearby WiFi points and the direction of their connection) along with the satisfaction of contributing to a more long term project – building impartial maps of network quality.

Linking app usage to its sensor network aspect is not the only way to build a sensor network. Google Maps is the largest sensor network worldwide, with over one billion active nodes collecting data on cell tower and WiFi locations. Using these sensors is one of the ways in which Google updates its location engine, which relates WiFi and cell tower readings to position coordinates. While Waze (now part of Google) makes much of its community aspect, rewarding users with badges for their data contribution, the data collection in Google Maps is only mentioned while clicking through the Terms of Service.

Google Maps is the world's largest mobile sensor network

Considering that Google Maps is the largest sensor network, is their approach better than Waze? Should apps de-emphasise any sensor network aspect and solely focus on immediate utility? Perhaps the answer depends on other factors. Google Maps started in a position of power in the app world, pre-loaded onto most Android phones and with a map database seeded by street view cars and data partnerships. Waze started with no such database, in many places their maps were empty, Waze users were encouraged to be like frontiersmen, they would get little immediate utility for their adoption of the service, but they would build a platform others could use. In Waze’s case the power of the crowd filled in the gaps, with the Waze maps being drawn by dedicated users committed to both the personal and collective benefits of using the app. This is true crowdsourcing in action, the collected data being beneficially reflected back on the user-base that has helped to gather it.

Logging

One of the most important aspects to the success of any sensor network is successful logging of the data, especially for networks where there is non-instantaneous transmission of the data to the central node (a term often used to describe sensor networks with on-node logging capability is 'distributed'). It is no coincidence that some of the earliest sensor networks, naval vessels measuring magnetic declination and recording weather conditions, literally involved the manual noting down of sensor readings in a log. This is also very important for modern sensor networks, where delayed upload can be beneficial for the contributing device (uses less cellular data or battery life) and for the network itself (in case the central server goes down). Data logging is vital to the success of the distributed mobile sensor network, especially considering that the nodes may not always have a reliable connection to the network – but are still capable of taking geo-located sensor readings. Automated logging is especially important because it removes the need for constant manual intervention from the human user, meaning much more sensor data is capable of being collected by each participating device.

The Automated Collection of Sensor data

Logging is key to making use of sensor data, but what may be less apparent is how challenging logging can be. There are two main aspects to the problem: automating the taking of sensor readings, and making them energy efficient.

“Hooke’s clock contributed nothing to the systematization of meteorological observation” according to Robert Multhauf, it is however an excellent example of the historical problem of automating data collection.

Perhaps the earliest example of automated sensor logging was the “weather clock” - conceived in 1663 jointly by school friends Sir Christopher Wren (the architect responsible for rebuilding much of London after the great fire of 1666) and Sir Robert Hooke (a polymath best known his law concerning the extension of springs). The weather clock was a mechanical instrument able to record various atmospheric variables on paper, these included wind direction, wind speed, rainfall, temperature, humidity and pressure - a stunning achievement for the time. The weather clock united cutting edge sensors in the operating system of the day, cogs, escapements and (of course) springs. It was a complex affair, and as Maurice Crewe remarks “Not surprisingly, it spent more time being repaired or developed than actually working”.

Despite its technological sophistication, the fact that two of the most accomplished minds of the age could not create a device that could reliably log atmospheric conditions is testament to the scale of the challenge. Today, however, thanks to the development of modern electronics, this problem has been solved. Using Android APIs and a Galaxy S4 you can now build a Weather Clock that will never miss a beat. For an experienced developer, it would be possible to build an app analogous to the weather clock in no more than an hour – and a novice could soon master the required techniques were he/she so inclined.

On most devices that we have tested WeatherSignal and OpenSignal on, sensors return values within a fraction of a second and when we’ve found errors it has usually been in our code. But the reliability of sensors returning values is not the only factor that can hold back sensor data: energy efficiency remains a serious barrier to continuous logging.

Energy savings through co-processing

The energy efficiency of sensory systems limits the amount of data that may be collected without causing a significant impact on battery life. While sensors themselves use relatively little energy taking a reading, the operating system framework for processing readings is costly. Sizeable energy savings are obtained by switching the work from the CPU to less powerful processors optimized for interpreting (for example turning accelerometer readings into step counts) and logging sensor data.

Apple and Google have both woken up to this. Apple’s M7 co-processor (first introduced in the iPhone 5S in September 2013) essentially silos the sensor work to a low-powered chip. Not long after, in October 2013, Android announced ‘sensor batching’ (introduced in Android KitKat), which is a generic means of achieving the same effect as the M7: sensor data is collected by the sensors (or a co-processor) and communicated to the CPU in batches, rather than continuously.

An iPhone 5S circuit board

On Android, which contains a much richer sensor set than iOS, it may take longer for the impact of sensor batching to be felt, as - unlike APIs - manufacturers are not required to support it. One highly promising data point is that the Galaxy S4 supports sensor batching, even though it was released half a year before the feature was announced, this degree of coordination suggests Samsung has been working closely with the Android team on this. Whatever the case, both the biggest smartphone manufacturer and the biggest smartphone OS are committed to sensors.

As developers of sensor networks, we have first hand experience of the difficulty of balancing data collection and battery life. With WeatherSignal, the world’s largest weather crowdsourcing platform, readings are taken hourly by default. Co-processors will allow us to take readings every minute with minimal battery impact, building up a much more detailed picture of atmospheric conditions and how we react to them.

Increased ability to log data is not the only result of improving sensor efficiency, it opens up the possibility for devices being more pro-active: interpreting their environment continuously and without user prompting. Some possible examples include increasing the ringtone in very noisy environments, foregrounding a fitness app if it detects you are running or a travel app if it detects you are in hurry.

More on energy efficiency: post-processing of location

While co-processing improves the efficiency of collection across a wide range of sensors, location readings remain costly to take. Fortunately there are techniques already available that can solve this. For GPS location readings, much of the energy cost is in receiving the almanac data (data about satellite routes through space) and performing calculations on this. Device GPS radios could collect much less information and leave the processing of this (and the union with almanac data) to be done in the cloud. The gains here are striking, 1000x increases in efficiency are possible.

While reducing the energy used in GPS requires some changes to the firmware, there are methods that Android apps can already make use of to derive location in a more efficient manner than the system default. This can be done through using the network that the device is connected to, rather than the satellite-powered GPS. Deriving location from the network typically relies on calling Apple or Android APIs directly and these APIs take information from the device itself on the Wi-Fi and cellular access points (hotspots and towers) that are within range of the device. This data is then sent to a server (or a local cache is consulted), a response is sent (or a local calculation performed on the cached data) with an estimate of the device location. With Android it is possible to get data directly from the access points, which can be saved and batch processed against a third party location-as-a-service API that translates them back into locations. This condenses much back-and-forth across the network to a single query allowing for significant energy savings.

On the overall energy cost of smart phones

Despite the potential for improvement, it is striking how energy efficient mobile devices already are. Indeed it is their very mobility – that they are removable from mains power - that has forced efficiency as a design priority, making them about 100x more energy efficient than desktop computers. A typical phone has a 5 amp hour battery that operates at 5V (generally, anything that charges from mini-USB has a 5V battery) this means it contains 5*5 = 25Wh of energy, typically you’d expect to get a day of use out of a phone battery, or 24 hours, so average power usage: ~1W. Being conservative it is probable to suppose that, with the energy that’s lost charging phones, it is more like 1.5W.

To give a little more perspective, a 50km journey by car uses around 40kWh of energy, so a typical daily commute would fuel your phone for 3 years. Or another way of thinking about it: a single Boeing 747 consumes 140MW. If we shift the 1.5W estimate of smartphone energy consumption to 1.4W for mathematical convenience, then a Boeing 747 consumes as much power as a 100 million smartphones.

100 million smartphones consumes about as much power as a single Boeing 747, making mobile sensor networks highly efficient to run

The efficiency of smartphones is a big part of their suitability to form nodes in a mobile sensor network, as the demands of participating in (often) multiple networks at once will have only minimal impact on battery life. This means that it will soon be possible to continuously monitor the environment that surrounds one quarter of the Earth’s population for the same energy cost as flying a dozen jumbo jets. While sensors without logging or networking can be of great value (think of a smoke alarm, or an offline pedometer), a new level of usefulness is reached when information is recorded and united. As mobiles become more efficient at processing sensor information, so the downside for the user is decreased – which will lead to much greater participation in crowdsourced sensor networks. Mobile phones are now capable of providing a dense, high-efficiency and always-on network through which to study the world – and advances in technology are only making the network more efficient at monitoring and then logging data.

Smarter Sensing

The ways in which the data collected from physical sensors is interpreted and processed on the device itself is changing – leading to more efficient and more accurate data collected from devices previously incapable of contributing high-quality data. Sensor fusion and virtual sensors are two, somewhat overlapping, ways that sensors can be turned into “sense”[3]. With ‘sensor fusion’, readings from multiple sensors are combined to produce a better result. Invensense pioneered the introduction of sensor fusion to smartphones, in particular using the readings of the accelerometer, compass and gyroscope in a device to complement one another in order to create more accurate readings for each of them.

Virtual sensors, also known as composite sensors, use readings from one or multiple sensors and interpret them: offering them up to the operator system as events in the exact same way as raw sensors. The step-counter introduced in Android’s KitKat is a virtual sensor, taking information from the accelerometer (and possibly also gyroscope, or a sensor-fusion of the two) and continuously analysing these patterns to look for a rhythmic motion that looks like walking. On the Galaxy S4 we’ve seen reasonably reliable performance whether the phone is in a pocket, a bag or is held.

The guidelines on the Android Open Source Project clarify good practices for building virtual sensors:

Do not conclude that virtual sensors are not ground breaking merely because software alone could detect the same events - for example a pedometer app could itself interpret the raw sensor readings. Virtual Sensors will help incubate a new generation of sensor apps for two reasons:

So far there are few virtual sensors. On Android there is the significant motion sensor, the step detector and the step counter. It is, however, easy to think of more that could be added including running, indoor/outdoor, going upstairs/downstairs or falling over.

When it makes sense to sense

The significant motion sensor is perhaps the vaguest and lowest resolution of Android sensors, as it returns only one of two values: 1 and 0. Nonetheless, it is hugely useful as a means of optimizing apps. It serves two purposes: firstly, it introduces a more efficient way of informing apps when location might be about to change (much less energy intensive than running a location poll) and secondly, it informs apps when activity may have changed (the user may have started running or walking). This lets apps know when they should fire-up more intensive sensor scans and processes.

Is Android’s Activity Recognition a virtual sensor?

Android’s Activity recognition, which can determine the likely activities of users (such as walking, running, cycling) is not implemented as a virtual sensor in terms of API functions but can nonetheless be onsidered a sort of virtual sensor for the way in which it creates sense from raw sensor data on the developer’s behalf. There are several possible reasons behind the design choice, one is that activity recognition is implemented through Google Play services (basically an app that auto-updates on device which has no frontend but serves as the backend to many other apps), it is not implemented in the Android OS directly - so it can be updated without a system update. A more fundamental reason may be that Google do not want developers to treat Activity Recognition as a sensor as it is currently not as low power as virtual sensors.

Sensors are becoming more plentiful, more efficient and more able to automatically collect data at greater speed. It is now possible to easily unite collected data and the sensors themselves are becoming faster. If mobile sensor networks provide a laboratory-overlay on the world, then that laboratory is becoming increasingly well stocked.

Putting Sensor Networks To Use

Currently the potential of smartphone hardware exceeds the ambitions of app developers, but this is set to change. The developments outlined above have meant that the world is covered in cheap and easily accessible sensors capable of reporting from a unique position at the intersection of human and physical geographies. People carry their phones everywhere, to every location, meaning that they are especially useful for examining how ambient conditions impact on human behaviour. The bundling of distinct and sophisticated sensors in a device means that there are hundreds of potential use-cases for mobile sensor networks, and developers are currently working on exploring the options – with many of the current projects listed below.

Indoor location

GPS receivers require ‘line of sight’ to communicate with satellites. For this reason GPS navigation is ineffective indoors, but a combination of other sensors can provide a strong substitute. Mobile sensor networks have been spectacularly successful for outdoor mapping, see Waze and Google Maps, but the next challenge is adapting them for indoor mapping.

Network location - or location provided by observations of WiFi routers and cellular antennas - can be made to work indoors, but presents some problems. Firstly, cellular location gives only a very rough indication of location - whether used indoors or outdoors typical accuracy is over 100m, not even good enough to tell you which building you are in. WiFi location currently gets down to around 10m of accuracy, enough to help you navigate a shopping centre, airport, or even underground station, but not without one key difficulty: a ‘ground truth’ dataset.

For the position of the user to be worked out from observations of WiFi routers, we need to know where those routers are, or - and this is slightly easier to find out - we need to know where people are when they see those hotspots as visible. Unfortunately, precisely because GPS does not work indoors, it is difficult to build up a Ground Truth dataset.

42% of Android devices have a gyroscope and 99% have an accelerometer - typically used to provide phone orientation
Source: The OpenSignal Sensor Library

There are many potential approaches for solving this problem, for example the user could be tracked by monitoring where he/she enters a building, then using the compass (which is typically fused with the gyroscope and accelerometers to give a better bearing) and the pedometer (a virtual sensor) to estimate distance moved. As the user moves further into the building the location accuracy will drop off, to a certain extent this can be compensated for by knowing the floorplans (another Ground Truth dataset) and using this to adjust for sensor errors and inaccuracy, this is a technique explored by MIT’s Jun-geun Park. For example a small error in compass reading, or estimated step size, might lead to the algorithm predicting the user has walked through a wall. By using floorplan data a much better estimate of user location can be achieved.

How can floorplan data be collected? Various apps already exist that help users measure spaces using phone sensors such as the camera (MagicPlan) or gyroscopes and accelerometers (RoomScan). The ideal method would not require any particular input from users beyond walking through each room; and with Project Tango this becomes possible. Google’s Project Tango is a phone-sized Android device incorporating a stereoscopic camera, high-grade sensors and software to map its environment from these inputs. Its unlikely that the Project Tango hardware will make it to billions of devices any time soon, but it’s quite possible that Google will send people to walk through buildings to map them, in the same way street view was used to populate street map and Google’s WiFi/Cellular database.

Fingerprinting

If we want to go beyond the resolution allowed by WiFi, it will be necessary to use fingerprinting techniques to improve location accuracy. After all, accuracy to 10m might be enough to stop us getting lost in the supermarket but it won’t be enough to show us exactly where to find the dried oregano. The answer here could be to unite data from many sensors in a technique called fingerprinting. While WiFi observations might be identical within a small area of a building, observations from other sensors may vary in a predictable way across the area, for instance the hum of the cooler may be stronger on one side of the room than the other - this solution is called Acoustic Background spectrum and has been explored by Steven Tarzia. Very likely the magnetic field is distorted in different ways by the metal shelves - a technique used by IndoorAtlas in their indoor mapping technology. FM radio waves could also be effective - as explored by Yin Chen et al. - though FM radios are becoming less common in mobile devices.

The idea behind fingerprinting is similar to the idea behind sensor fusion: when used in combination, sensors give much richer information.

Context Awareness

Context awareness lets devices know what sort of an environment they are in, both in terms of place and behaviour - for example, distinguishing between home and work. Currently most approaches to this rely on looking up the geographic location against some database of contexts (such as 4Square) which are often compiled manually. However more detail on the context could be discovered by directly consulting the sensors of a single device or aggregated over many from the same location over time.

Activity Recognition

Activity recognition can also be thought of as a form of “fingerprinting”, but instead of extracting a placed from a pattern of sensor readings, the aim is to analyse what the user is doing. As mentioned above, Android now incorporates a framework for doing this for some basic activities - walking, cycling, driving - Fitbit and Jawbone cut a little finer, including monitoring sleep and running. With co-processing in mobile devices, activity recognition could potentially run in the background, allowing apps to be informed of changes in user activity as they happen, and giving them a complete overview of user activities throughout the day.

Now that simple activity recognition is becoming more readily available to developers - by the Android Activity Recognition framework and virtual sensors such as the pedometer - we can expect to see the envelope pushed on the recognition of more complex activities: more forms of transport taking, dancing or using lifts and escalators.

Activity recognition is key to fitness trackers and is of particular use in mHealth, but the field goes far beyond this. Social networks allow us to give our own reports on how we say we interact with our environment, but activity recognition provides an impartial view unaffected by reporting bias. Data collected could be used to map the behaviour associated with places, for example it could reveal to a bar whether people are mostly seated, stood up or dancing. Behaviour could also be correlated against other external factors such as the weather and world events.

Intention and Attitude Recognition

If one thinks of activity recognition as identifying the verbs associated with current user behaviour (walking, running, cycling) then attitude recognition would provide the adverbs (leisurely, hurriedly, purposefully) to complete the full sentence of our behaviour. As more analysis is done on phone sensor data, it will be possible to add levels of detail to our understanding of user behaviour and, by making analysis of frequently occurring patterns possible, it might even become possible to predict it.

mHealth

Mobile devices are positioned to revolutionise health care in two distinct ways: Telemedicine, the more developed of the two ways, is the provision of health care at a distance. The camera on a device and the manual inputs (physical keyboard or touchscreen and microphone) alone are sufficient to make many diagnoses. Telemedicine could be particularly important in providing healthcare to rural communities in developing countries, though the speed and reliability of an internet connection problems poses problems - “The biggest challenge is connectivity.” Adrian Pacheco, director of the Centro Nacional de Excelencia Tecnológia en Salud.

The second form of mHealth is monitoring the user’s behaviour and environment to proactively diagnose, predict and recommend. The Galaxy S5 includes a heart-rate sensor, the Basis activity monitor tracks sweat-rate, the Dash smart headphones promise to provide both these and also blood oxygen levels. The Galaxy S4 contains a humidity sensor and the smartphones of 2015 may well contain gas sensors that can measure pollutant levels and even act as breathalyzers - with companies such as Sensirion aiming to preview these to OEMs this year.

Within the current hardware horizon, mHealth has a bright future, but even with the most basic smartphone sensors a lot can be learnt. For his PhD thesis, Anmol Madan collected data on 70 residents of an undergraduate dorm at MIT, in order to see how behaviour could be used to predict depression and other illnesses. Madan’s app ran on Windows Phone and accessed WiFi (for approximate location), Bluetooth (to monitor when participants were spending time with one-another) and SMS/call records (to monitor other interactions); manual input data was also collected. Neither GPS nor accelerometers were accessed - let alone blood pressure - yet Madan’s experiment showed how depression, light colds and fevers were reflected in highly characteristic behaviour patterns. Madan has turned the ideas from his thesis into a venture capital backed company, Ginger.io.

Meteorology

From ships to rain gauges, the history of sensor networks is closely linked with improvements in meteorology. Current forecasts are based on data collected from a network of weather stations, highly calibrated and carefully regulated sensors that report on multiple atmospheric variables which are then inputted into forecasters’ models. One of the great challenges of successful forecasting lies in accurately measuring the starting parameters, as small errors in measurement can lead to radically different outcomes. Weather systems are chaotic in nature and modern forecasting is carried out using the ensemble method, inputting a number of distinct snapshots (the ECMWF use 21 for their forecast) of current atmospheric conditions and running them through the same model to see how the outcomes differ. The extent to which the various outputs agree represents the degree of confidence that can be placed in the consolidated output. Mobile sensor networks (such as WeatherSignal and Pressurenet) can increase the granularity and accuracy of atmospheric readings, which has the potential to lead to increased confidence in the data that is run through the ensemble method.

Meteorology has historically been a key driver of both sensor and sensor network development, and this continues to be true for the mobile sensor network

Clifford Mass, at the University of Washington, published a paper based on data collected from the WeatherSignal sensor network entitled ‘Surface Pressure Observations from Smartphones: A Potential Revolution for High-Resolution Weather Prediction?’. Most modern smartphones contain a barometer, used for determining altitude for improving GPS fixes, which can be repurposed to provide pressure readings for forecasters. Pressure suffers from few of the observational problems that can impact other smartphone sensor readings, as pressure readings can be taken without undue impact from external conditions, such as the device being indoors or outdoors. Previous studies including Whitaker et. al (2004) and Dirren et al. (2007) have shown that surface pressure readings are exceptionally valuable in modelling long-term weather trends. Early work by Wheatley and Stensrud (2010) found that surface pressure observations facilitated accurate depictions of the mesoscale pressure patterns associated with convective systems. Above all, ensemble data assimilation has proven to be particularly effective at incorporating surface pressure readings into the models, with Madaus et al. (2014) showing that that dense pressure observations produced improved high-resolution forecasts. By 2015, it is estimated that there will be over 500 million pressure-capable Android devices - all of which would be capable of being leveraged as networked nodes in a meteorological mobile sensor network.

The research carried out by Cliff Mass suggests several areas where a pressure-focussed sensor network could be most useful, beyond positively impacting general weather prediction. Providing an ‘early-warning’ indication of severe convection and improving short-term wind forecasts by improved front-observation are two clear areas of weakness for conventional atmospheric data gathering that could be improved through an increased density of observations. Making use of pre-existing barometric sensors in smartphones would appear to be the lowest-cost approach to achieving the required density of readings. In addition to pressure, smartphones have the potential to contribute other data of meteorological utility with temperature initially the most obvious. While few smart devices currently carry an ambient thermometer, research carried out by Overeem et al. (2013) on battery-temperature readings from smartphones (based on data collected by OpenSignal) showed that, averaged over a sufficient sample size, it was possible to create useful measures of daily average air temperature over major cities. As the sensors in smartphones become increasingly diverse, the smartphone will play an ever-increasing role in the future of meteorology, with crowdsourcing an increasingly important vehicle for the creation of these networks. As with the development of the sensor network, it is meteorology that may well prove to be the driving force in the development of the mobile sensor network.

Conclusion

Mobile sensor networks present the opportunity to gather data about the world at a never-before-seen scale and low cost. Smartphone users participating in crowdsourced mobile sensor networks are able to form part of a social laboratory, acting as gatekeepers of the devices that constantly record data about the environments in which they find themselves. What is particularly significant about this web of sensors is that it allows for a unification between the ‘Internet of People’ and the ‘Internet of Things’, as individuals are able to participate both actively and passively in the collection of data. Crowdsourcing is the tool that will allow these scaled networks to be built, tapping into users’ ‘cognitive surplus’, the period of time that they are not using their brains for anything productive, in order to turn people into networked nodes. The global reach of smarpthones and the app store allows people to be easily reached and networked as part of a dispersed social laboratory, collecting data about both human behaviour and the physical world.

The key issue that has not been so far addressed in this paper is issues of privacy that arise from participation in mobile sensor networks. We have made clear that we believe the mobile sensor networks of the future will be built by consensus, through the active recruitment of users driven to participate either through transactional or utilitarian benefits - and transparency about what data is being collected. However, not all mobile sensor networks will function in this way - for example, Google Maps only mentions it uses location data for improving the maps in its terms of service. In The Human Condition, Hannah Arendt described the necessity of privacy to a fully functioning society, writing ‘A life spent entirely in public, in the presence of others, becomes, as we would say, shallow. While it retains its visibility, it loses the quality of rising into sight from some darker ground which must remain hidden if it is not to lose its depth in a very real, non-subjective sense.’ Temporarily setting aside concerns about the ability of the state to access information about its citizens (an important subject for democracy and one that has been debated, though as yet unsatisfactorily resolved in the face of changing technology) it is a further quality of privacy that might potentially be impacted by constant participation in data collection.

One of the challenges of a networked society is finding a balance between participation in global mobile sensor networks and freedom to feel that one has withdrawn from public space, a tension that is difficult to resolve. Mobile sensor networks offer the opportunity to turn the world into a social lab, reducing the barriers of entry to scientific exploration and allowing a whole new body of data to transform the way we analyse everything from weather forecasting to social trends. As more sensors are added to smartphones, and more developers take advantage of them, mobile sensor networks may prove to be the most significant scientific apparatus of the 21st century.


[1] Consider this example, ambient temperature is clearly acknowledgeable to an individual (who is capable of saying that a day is hot or cold) but also precisely unquantifiable without a thermometer (the individual cannot say that it is 24 as opposed to 25 degrees Celsius).
[2] An interesting historical note: the Cheugugi was invented by a peasant, Jang Yeong-sil, who was brought out of poverty by the meritocratic court system of the time. In later life, Jang built King Sejong a sedan that broke with him inside, for which he was subsequently jailed despite opposition from the king himself. The meritocracy went both ways.
[3] Our thanks to Sensirion for setting us thinking about the differences between “sensors” “sensing” and “sense”.

Contact Sales

Are you having trouble with the OpenSignal Apps and need some help?

Are you a mobile operator or analyst and want to contact a sales expert?

Share Graph