Part 3: Audio context database for next generation context aware applications
Context aware applications
Today’s smart phone apps are just that: smart. They know what we are doing (jogging or sleeping) and can suggest a good place to eat in the neighborhood. One of the reasons our mobile devices and apps are smarter is the increasing number of sensors built into them.
GPS positioning lets our phones know where we are so restaurant apps can advise us on the best places to eat, and tell us how to get there. Fitness and health apps use accelerometers to work out how many calories we have burned off or how well we slept. But there is another set of sensors in our mobile devices that has, until now, been mostly overlooked – the audio sensors or microphones.
Audio context apps
There are apps that use our devices’ microphones to assist us. Personal assistant apps like Apple’s Siri, can perform web searches and control other phone functions based on spoken commands. And music recognition apps like Shazam or Soundhound can identify a song just by listening to it for a few seconds. These work by comparing the song to a database of audio signatures.
Being able to recognize a song is one clear use case for audio recognition. After all, how many questions have been answered, or arguments resolved, by being able to identify the name of that song or the band that played it? What if our phones could identify other sounds? What other sort of apps would be possible?
For example, you are driving in your car and you hear a siren in the distance but cannot tell exactly where it is coming from. If your phone can recognize the sound of the siren it could let you know that that an ambulance is approaching from behind you so you need to pull over. Or that a fire engine is coming from the left so you need to leave the junction ahead clear to let it pass. There are many other possible use cases but for them to work, your phone needs to be able to identify that sound.
Audio context database
And that is just what NXP Software, together with Eurecom is doing. We are building an audio context database which contains many different types of sounds recorded in many different environments and situations. We are then using that database to identify sound signatures that can be used by mobile devices to recognize a specific sound or class of sounds.
This will enable a whole new range of apps that will take these sounds into account. These smart apps will also learn from and adapt to our behavior, based on our user preferences and other sensor inputs. This will enable them to provide us with more accurate and relevant information, so we can make better and more informed decisions about how we live our lives.