Design of sound recognition and augmentation software for implementation upon a mobile device. Project conceived and constructed over a month long residency at the Pervasive Media Studios, Bristol in April of 2009 with support from HP Labs, Bristol.
Bristol is famous for its central harbour and waterways, and has numerous fountains in prominent locations throughout the city. Each of these water features has it’s own distinct sound pattern that is clearly audible against the din of the surrounding environment.
For the Fountain Experiment a mobile interface was designed, which is capable of detecting the presence of a specific fountain in Bristol through listening to the soundscape surrounding the mobile carrier. Furthermore, by use of stereo microphones and a custom made frequency recognition model, the system is able to predict the distance the carrier is from the tracked fountain and panoramically where this fountain is positioned. With this understanding, the mobile interface can augment flow of the fountain with new sound events. In this first experiment, the sound of bubbles bursting and striking the metallic surface of the fountain’s construction was affixed. This sound layer reacts in the same manner as the real events it follows, shifting both in frequency and position as the device carrier navigates and orientates around the fountain structure.
Although the presented example detects the presence of only one particular fountain, the expectation is that the designed system could be enhanced so as to be able to detect many more everyday sounds (e.g. passing cars, air conditioning and ventilation systems, and the hum of the crowd) and attach new events on to these. The Fountain Experiment served to demonstrate to HP Labs a new functionality that their hardware and software systems could incorporate, which would firstly allow for greater control over more of sound’s parameters (e.g. control of a sound’s frequency range) and secondly, would achieve a contextual awareness through listening to the mobile carrier’s surrounding sound field. In addition, the proposed framework would facilitate the appending of new sound onto an existing, real sound field rather than replacing it, as is the case with HP’s present system.