Augmenting music to fit the surrounding spatial environment

I’ve always wanted to find ways for digital technology to reconnect people with their environment. Screens, with the exception of augmented reality, tend to disconnect people from their immediate surroundings. The glowing events on the screen are out of time and place, so when your attention is on the screen you’re not attending to your own place and time. Even haptic interactions tend to take your attention away from your environment and focus it on the device.

What if your earbuds reconnected you to your current surroundings?

Many people leave their home and if they are walking, plug headphones in to listen to their favorite tunes as they  commute or run errands. It’s easy to see the appeal, instead of honking cars, loud strangers or distracting conversations you can kind of cocoon into your own audio track. The only problem with this us that it strongly isolates you. Your ears are entirely preoccupied with sounds recorded at a different time and in a static studio space; they have no connection to the present.

What if your device could reconnect you part of your current experience? Your device would sense the spatial characteristics of the environment you were walking through and perform signal processing on the sound as it headed toward your ears. You would still largely get the musical overlay, blocking out the unpleasant noise of the urban environment, but you’d also get a subtle reintroduction to the space itself. If you walked into a tiny room, the music would deaden, and sound like it’s being played in a small space. When you walked into a large space with lots of reflective surfaces, the music would reverberate and appear to echo into the cavern.

Demonstration of the concept

It’s hard to talk about the concept because it’s such a sensual idea. I’m neither an awesome videographer nor an audio engineer, but I hacked together a video walkthrough of different spaces and a Logic Pro audio track that plays with the reverb of the track to approximate the effect of moving into different environments. Even with this crude prototype you get the sense of how the idea might play out.

You’ll want to put on headphones to evaluate the following experience.

Musical track: Driving along the Sunset by Mark Tyner