Élèves en liberté: A Classroom in Augmented Reality Sound
A headset that places you in a large room and generates a world of sounds based on your movements and interactions. That is the augmented reality experience proposed by Élèves en liberté. It’s an innovative device for a immersive documentary that explores a school that practises active learning.
More and more schools are resorting to alternative pedagogical methods, which are said to be active and by virtue of which ‘traditional’ teaching has been extensively rethought. A form of utopia for some, a desirable future for others, in practice, these new methods remain little known.
To correct this information deficit, Patrick Séverin and his team at HorsZone, in coproduction with Radio-télévision belge de la Communauté française (RTBF), decided to develop a widespread transmedia device named Élèves en liberté.
The project, which has you enter a Belgian school that practises active learning, has known multiple variations: an interactive, radio, televised (spring 2019) and immersive documentary. Here, we are going to explore the documentary’s immersive facet.
In collaboration with Brussels-based studio Demute, the team developed an in situ room-scale augmented audio experience: “Of course, it is possible to visit this school in Liège. However, whenever you visit it, it is empty!” points out Patrick Séverin. “Visiting an empty school makes no sense. The idea was therefore to propose a virtual visit that sets itself apart from devices that do not seem to work, in my opinion, such as, for example, certain 360° videos.”
Experiencing room-scale augmented reality
Accompanied by two other people, you enter a classroom. A blackboard, posters on the wall, tables, chairs… and a teacher who follows your every move. Your only school supplies are earphones plugged into your smartphone. You are invited to take a seat and the teacher starts slow by asking you how you enjoyed your vacation.
You hear [virtual] students answer his question. Surround sound is used and you begin to understand that a rich layer of sound is superimposed on top of this very real décor. You even realize that the teacher’s voice is also a recording and that lip sync is at play since the beginning.
After a few minutes, the teacher invites you to stand up and explore the room. By touching various objects, you trigger sounds, namely interviews conducted with the teachers and students of this very particular school. These sounds combine to constitute a common sound space shared with all of the other spectators. If another participant pairs up with you, he or she will hear exactly what you hear!
The experience is truly one of a kind. It cannot be categorized as a virtual reality experience, which goes much further in terms of sensory deprivation. It cannot be categorized as ‘visual’ augmented reality, which to this day is hindered by the impractical aspect of having to hold a smartphone in front of you or wearing somewhat cumbersome glasses. We are also quite far from the podcast or a simple audio guide...
The fact of being able to move around freely and of going out to locate and discover sounds in the spatial environment instantly provides an interesting depth to the classroom’s space. Triggering sounds by manipulating items making up the décor, such as small figurines for example, gives the experience a playful aspect that renders the immersion even more convincing.
“Sounds draw on a stronger emotional connection than images,” according to François Fripiat, founder of the Demute studio. “The human eye is a more absolute instrument than the human ear. However, sounds provide quicker access to emotions and it is easier to trick the mind through sounds than it is through images.”
How does it work?
The technology that is concealed behind the Élèves en liberté experience is different in that it is very transparent. “No one is under the impression of entering a high-tech device,” claims Patrick Séverin. “Putting on a headset is something we do on a daily basis! From children to grandparents, the work remains accessible and no one has asked how it works.”
Therefore, a simple headset connected to a smartphone that you insert into a small pouch worn around your neck is the whole of the technology that is ‘visible’ to the user.
Of course, when you go into further detail, the story becomes more complex. The Demute studio specializes in creating innovative sound and developed a headset that incorporates a gyroscope (to detect your movements) and a positioning beacon (to locate your spatial position) among others.
The information thus captured is sent to the phone, which serves as an on-board computer. It will ‘create’ the world of sound in which you will evolve and, of course, play it back to your ears.
This world of sound is also influenced by the participant’s actions. Certain décor components are interactive and will trigger specific sounds that are audible to everyone. “A common sound universe was a demand made by the creators,” recalls François Fripiat. “But in the case of museum apps, for example, it will maybe be more relevant—and certainly easier!—to create individual sound spaces.”
Of course, the experience proposed by Élèves en liberté is nothing more than an embodiment of this technology’s potential. Museum experiences, escape games, immersive sound documentaries and works of fiction, the possibilities are almost endless.
“It’s a technology that has the potential of eliminating the audio guide model,” estimates Patrick Séverin. “As for its cultural and narrative applications, we’re going to have to continue experimenting!”
The strong tie between the technology and the work
This project perfectly illustrates the virtuous circle that can set itself up between the time a new technology is developed and the moment a particular work is created. “Élèves en liberté truly was our very first project to use this device that we had conceived two years prior. We therefore piloted the R&D to suit the needs of the project.”, says Patrick Séverin.
At the very beginning, the team came up with the idea of an experience in which sounds are placed throughout the room and played in a loop. In this version, the audience would have simply moved around the room without any other impact on the content making up the experience.
Two major evolutions took form as the work was being developed: the addition of interactive elements enabling the audience to trigger the sounds itself and—it goes without saying—the passage from an individual experience to a collective one.
“This technology gives us the same level of conviction as we have with respect to virtual reality, namely that the experience must be social in nature,” states François Fripiat.
These improvements and these new functionalities were made possible by a close level of collaboration between technology and content proponents. The latter contribute an interesting outside view: “I sometimes did not share the same perception as the people at the Demute studio,” recalls Patrick Séverin. “My ear is not as developed as theirs and I was sometimes unable to establish the direction from which a sound was reaching my ear.”
A major effort was therefore deployed to adapt the sound universe and better emphasize the contrasts between the sounds to allow the average user to better perceive them.
It goes without saying that all of this research & development work contributed to extending the time required to produce Élèves en liberté to such an extent that “as late as one week before release, we finally decided to contact the press because we still until then had no certitude that everything would work!” recognizes Patrick Séverin.
It took none months for the Demute studio’s team to master this ambitious technology. Nevertheless, the time spent was productive: “Today, the time required to produce a similar project would be anywhere between a month and a half and two months. And that is without counting on the creative tools that we will be developing!”
The studio is now working to develop a headset that will embed all of the technology needed to eliminate the need for wearing a smartphone around the neck as well as the software tools required to accelerate the creation and potentially license other creators to use the technology.
These are evolutions that should continue to materialize thanks to the development of new projects. Some are already contemplated and the studio remains attentive to the creators...
The audience’s reaction
For its premiere, Élèves en liberté was set up in the same school where the documentary’s sequences had been recorded. The audience is impressed by the surround sound experience and the ease with which it can be accessed. Some get sufficiently caught up in this sound universe that they answer the teacher’s questions as early as the first part of the experience...
“I nevertheless expected people to ‘play’ with the experience somewhat more. But most people listen without attempting to push the experience to its limits!” points out Patrick Séverin. “By moving the figurines, by turning around sound sources… Finally, people get so caught up by the sound narrative that they sometimes lose sight of some of the experience’s dimensions.”
Since then, the experience has been set up two other times in Belgium and other venues will surely follow even though “what is complicated for a small studio like ours is that we invest all of our energy into production. We are not distributors,” admits Patrick Séverin.
The work will therefore probably not be a huge success with audiences, but nevertheless serves to clear the experimentation achievement field . Also, “I do not know if Élèves en liberté would actually make for a good tour product. Maybe its content is not spectacular enough? It remains a documentary without dramatization. It’s impressive from a technological standpoint but, for the general public that is not interested in the subject, there will surely be something missing.”, adds Séverin.
A work that uses this technology and is destined for the general public is certainly worthy of emerging. Élèves en liberté has demonstrated its immersive potential, its accessibility and its relevance in a cultural context where solutions enabling the marriage of what is real with what is virtual are in increasing demand.
