The immersive power of binaural sound: an interview with Amaury La Burthe (Audiogaming)
What is binaural sound and how can you use it in your work? Amaury La Burthe, who specializes in audio tools for digital media, explains how this technology contributed to the success of the Notes on Blindness virtual reality experience.
New narrative formats create a breeding ground for sound designers, and there are more and more innovations being developed. Among these innovations, binaural sound is a technology of particular interest namely in that it is highly compatible with mobile and virtual reality experiences. Indeed, binaural sound creates the impression of being plunged in a surround sound (360°) environment.
To grasp the potential of this specific type of sound, I met with Amaury La Burthe, founder of Audiogaming and Novelab and designer of interactive experiences in which the audio dimension often plays a fundamental role. One of his latest co-productions titled Notes on Blindness was almost universally acclaimed as a major virtual reality masterpiece. It’s an experience in which binaural sound is combined with a fascinating visual environment to allow us to experience the reality of a man who has gone blind.
What is so special about binaural sound?
We’ll get back to Notes on Blindness, but let’s start with the nature of binaural sound. Curiously, it’s a technology the origins of which date back to over a century ago! Indeed, as early as 1881, the theatre phone developed by French engineer Clément Ader made it possible to listen to performances of the Garnier Opera within a radius of two kilometres. Users had to place two phone handsets on their ears.
Of course, today’s binaural sound is much more complex, but the basic concept remains the same: faithfully reproducing our natural listening mode to be able to locate sound in space.
Put on your finest headset and listen to this video to live the experience:
From our earliest childhood, our brain learns to locate sounds in space by interpreting the slightest shifts in perception between the right and left ears. Binaural sound seeks to recreate this perception: a sound seems to arrive from the left if it reaches your right ear after a slight delay.
Source: James Gasson
It’s this shift that ‘tricks’ our brain into believing that the sound travels through a ‘spatialized’ acoustic environment when we hear the sound through our audio headset. Inversely, a stereo sound will create a mono perception in a headset because the sound seems to emanate from within our head.
This 360° binaural perception is therefore the perfect match for a visual environment of the same nature: virtual reality. This pairing is full of promise when it comes to immersive experiences. However, a few obstacles still need to be overcome…
Limitations of binaural sound
It goes without saying that the first obstacle is using a headset. Today, many people have one with them almost permanently, but that has not always been the case. That’s why this technology was somewhat ignored during the time when television ruled. There was no reason for wearing a headset then.
Today, the fact that mobile and virtual reality devices have gone mainstream explains why binaural technology has become interesting once again. However, producing binaural sound remains a challenge.
The issue at hand is neither the technology nor the production costs. “It’s not the same material so there are a few additional costs, but they are far from prohibitive. It doesn’t quadruple the audio costs,” assures Amaury La Burthe.
No, the main problem with binaural sound is of a physical nature. We all have different heads and our brains learned to interpret acoustic signals differently.
Our morphology and the shape of our ears influence the frequency and amplitude of the sounds we hear. My head and ears are shaped differently from yours and we will therefore perceive a binaural sound differently.
Source: Shawn Feeney
“People whose head is very different from the so-called average head will not hear a sound that is very externalized, [it might sound] almost stereo. We have not yet succeeded in overcoming that limitation.” And when La Burthe invokes the ‘average head’ here, he’s not simply using a metaphor. Most often, to record binaural sound, two microphones are placed on a silicone head to record the sound as it is supposed to reach our ears.
Source: Neumann binaural head
Binaural sound can also be produced in a studio (in which case it is qualified as synthesized rather than native binaural). A synthesized binaural sound is in reality a mono sound that is converted using specialized audio editing equipment based on algorithms. That’s the method that was used by the creators of Notes on Blindness, a unique work in which the story is told foremost through sound.
Notes on Blindness, binaural sound at the service of history
Notes on Blindness is above all a superb short film by Peter Middleton and James Spinney that showcases the audio recordings of American theologian John Hull, who documented his gradual loss of sight at the beginning of the 1980s. The project later evolved to a feature film format with ARTE, which was also seeking to produce a one-of-a-kind digital experience.
Source: ARTE
The result is the virtual reality work Notes on Blindness: Into Darkness, co-produced by Agat Films / Ex Nihilo, Archer’s Mark, Audiogaming and ARTE.
“In Notes on Blindness, John Hull tells the story of his first three years during which his brain tried to reconstruct something up until when he admits in one of his last tapes that he has decided to let go of what he once knew and embrace his new perception as a blind person.
“His tapes account for thirty or so hours of audio, i.e., between 300 and 400 transcribed pages. It’s an enormous base to work from! I therefore began by locating all of the passages in which he talked about sound and that’s what made it possible for me to visualize things, not only descriptive passages but also passages in which reading created a vision of the scene,” recalls La Burthe.
As is the case with many innovative works, the format of Notes on Blindness has evolved greatly. Initially, it was intended as a playful rather than poetic mobile app. Since then, things changed quite a bit and the original concept took on a visual dimension with virtual reality that made it possible to tell John Hull’s story more eloquently. However, one thing did not change: the importance given to sound.
“We were looking for a very specific and immersive sound. We knew that we were going to use real-time binaural sound.” Indeed, all of the sounds in Notes on Blindness are said to be spatialized because they correspond to a very specific position in space. For example, if this is what I see in my field of vision:
… I will therefore perceive that the sound of the merry-go-round is coming from my left. However, if I turn my head and the merry-go-round is now on my right, the corresponding binaural sound will when be adjusted in real time to give me the right acoustic perception…
“The calculations are made as the person turns his head. It’s instantaneous, […] which caused us some problems because we had to run between 30 and 40 sounds in real time on a mobile device—far from an easy feat. We spent a crazy amount of time optimizing the app. Initially, the phones would crash after two or three minutes in the park (the first scene of the experience [author’s note]).”
Proposing such a rich and adaptable audio environment in real time based on what a person is looking at is a major technological accomplishment. It explains most of the emotional power of Notes on Blindness, all the more so seeing as these ‘spatialized’ sounds create a wonderful contrast with John Hull’s voice recorded in mono. Consequently, we have the impression that his words are reaching our head directly whereas the environment arouses our curiosity through these 360° acoustic and visual elements.
Tests had nevertheless been conducted to position John Hull’s voice right next to our ear and create the impression that Hull was secretly whispering his story to us. But the final result was “very disturbing” according to La Burthe. This demonstrates well that binaural sound does not guarantee immersion in itself. It needs to be used properly!
Is binaural sound always worth it?
Notes on Blindness should convince you beyond a shadow of a doubt of the immersive power of binaural sound. Despite certain constraints that are inherent to this technology, binaural sound should further develop as virtual reality becomes increasingly mainstream.
However, there is no scenario in which binaural sound would become omnipresent. La Burthe considers that its use must follow an artistic process that is not compatible with all genres and supports.
“In virtual reality, binaural sound is amazing. We benefit from visual depth and acoustic depth. […] As part of other immersive experiences such as video gaming, I also believe that it could have benefits and help to give depth to the experience.
“If the acoustics are adequate, binaural sound can make sense. For example, take classical music. However, is it worth using binaural to produce pop? I don’t know. In any case, to this day, I have not yet heard anything that convinced me of that.
“Forcibly, it can be appropriate for horror. 360° horror is a niche market for which binaural sound is really well adapted. […] Conversely, in a movie or a series like The Walking Dead, do I really feel like listening to binaural sound on a flat visual background? I’m far from convinced of that.”
Many uses of binaural sound have yet to be discovered. Therefore, don’t hesitate to assess its relevance for your next project. In the meantime, I invite you to download Notes on Blindnessand also recommend you have a look at RTBF’s Burkland series. It’s one of the first of its kind to have incorporated binaural sound. Your ears will thank you for it...
