Be the first to find out about GRAMMY nominees, winners, important news, and events. Privacy Policy
Photo: Rob Monk/Future Music Magazine via Getty Images
feature
Immersive audio tech allows sound technicians to think and work in three dimensions, and for listeners to get "inside" the sound. GRAMMY.com takes a deep dive into the past, present and future of immersive audio for home, the studio and live audience.
The quest to provide listeners with the highest fidelity and most realistic audio experience has been ongoing for centuries — Venice's Basilica of San Marco underwent structural modifications in the 16th century to ensure that the seat designated for the city’s top elected official received the best possible sound. Hundreds of years later, we are still seeking new auditory experiences, and the pace of recent innovations suggests that more advances lie ahead.
Immersion has been a throughline throughout our history of audio improvements. Engineers at the Basilica of San Marco employed a split choir to create a "stereo" effect that immersed listeners in a three-dimensional swirl of sound; more recently, works by 20th century composers Karlheinz Stockhausen and Edgard Varèse sought to place listeners in the center of the sound. In the late 1960s, Pink Floyd developed the Azimuth Coordinator, a joystick device that afforded real-time control of sound output from speakers placed all over the concert hall. Today, many performing artists find ways to incorporate immersive audio into the concert experience.
Audio recording technology developed along similar lines. As early as 1939, Disney engineers developed the stereo precursor, Fantasound (which was used to great effect on the Fantasia soundtrack). Stereophonic records came into wide use in the late 1950s and, by 1970, four-channel quadraphonic audio technology sought to create an enhanced and even more realistic sound experience for home listeners.
The mid 1970s saw the advent of 5.1 surround audio, developed by Dolby Labs. Created for home theater and cinema, the format became the audio standard for digital broadcasting. Today, 7.1 surround — a system with eight speakers — is common in home theater applications and recording studios.
All of these breakthroughs have a common core goal: Giving the listener – at home, in a concert hall, and ideally anywhere – an audio experience that fully envelops them in sound for a three-dimensional experience. Recent advances have been nothing short of breathtaking, perhaps fundamentally changing the way professionals and average listeners experience sound. Yet more than 150 years after the dawn of recorded sound, most experts agree that immersive audio is still in its infancy.
Most sound engineers mix live concert audio in "distributed monaural," says Marc Lopez, Vice President of Marketing Americas for d&b audiotechnik — a German company that has been designing and manufacturing amplifiers, loudspeakers and sound systems since 1981. He explains that live sound technicians are mostly just "trying to get sound distributed," but the result can be removed from reality. "It’s almost like you're watching and hearing from somewhere else," he tells GRAMMY.com.
Immersive audio technologies aim to put listeners in a specific place, localizing "sound — not just in front of you or inside your head but all around you," notes Guillaume Le Nost, Executive Director of Creative Technologies for L-Acoustics. Based in Marcoussis, France, L-Acoustics manufactures loudspeakers, amplifiers and signal processing equipment primarily for live sound. Immersive audio is a growing segment of the company’s business. 
Le Nost points out that stereophonic (two-channel) sound has serious shortcomings, especially in a live music context. "When your speakers are 30 meters away from each other, it starts to be very difficult to have a nice stereo experience except [for] people exactly at the center," he says, adding that immersive audio places "the listener inside of the sound." 
While quadrophonics provided four discrete audio channels, today’s audio technology moves far beyond, into three dimensions. With immersive audio, says Le Nost, "we can place objects in space." And immersive technology "doesn't define a loudspeaker format delivery," says David Gould, Senior Director of Content Creation Solutions at San Francisco-based Dolby Laboratories. "It just loosely defines something that’s above and beyond."
Instead of merely providing four discrete channels of audio, immersive sound technology is object-based: "You can place a sound in a specific position independently from how many loudspeakers you have; they’re completely independent of each other; that’s really the magic of the object-based approach," Le Nost says.
"It's a lot of work to finagle all the instruments and frequencies to work in two channels and give a transparent mix," observes Steve Ellison, Director of Spatial Sound for Berkeley, Calif. based Meyer Sound Laboratories. Meyer designs and builds audio gear for the professional sound reinforcement and recording industries. 
While quadrophonics attempted to improve upon stereo, a worthy goal, quad sound could be gimmicky in its 1970s applications. "The stereotype of quad is a kind of ping-ponging effect: ‘Where’s the sound?’" Ellison notes. 
The immersive approach circumvents these issues by allowing sound technicians to think and work in three dimensions instead of two. To use an example from geometry, stereo and quad work with the X and Y axes, but the spatial/immersive approach adds the Z axis. "With traditional mixing, you’re mixing a bunch of single channels directly to the outputs," Lopez explains. The object-based approach employs what he half-jokingly calls "mystery processing. You just move the interface around, and it will figure out the best loudspeaker output for you." 
The result is infinitely more versatile and lifelike than quad could ever be.
Lopez places the development of immersive audio into an evolutionary timeline. "Monophonic delivery provided sound, but not necessarily experience," he says. "Stereo was a happy medium, a practical [way] for living rooms to deliver more of that experience. Quadraphonic sound attempted to put the listener either into the audience or into the band, and 5.1 surround was a more defined execution of that."
For a long time, if you wanted an audio experience beyond stereo, "there was only one way of doing it: you had to get four speakers," says Gould. He notes that with current immersive audio technologies like Dolby Atmos, the user has far more latitude and flexibility. "Think about things like sound bars, upward-firing speakers and even binaural renderers on cell phones," he says. "Now we have far more ways to get that experience."
Immersive audio takes things to another level entirely, he adds. And by definition, it’s not limited to a specific setup of one, two, four or some other number of speakers.
The object-based approach brings another important innovation: scalability. Freed from the requirement of a specific audio setup, technologies like Dolby Atmos are engineered to adapt to the application and listening environment. Debuting in 2012, Atmos enabled overhead sound to cinema audio; two years later it became available for home theaters, and today it’s gaining widespread use in recording studios. 
"Rather than being ‘burned’ into channels," Gould explains, "you now get these discrete audio objects that can [effectively ask] 'Okay, what is my playback environment, and how do I best render this audio within that playback environment?'" Metadata encoded into the digital audio files also provides critical three-dimensional audio positioning information. "So whatever speakers you have, each sound is coming from the right place," Gould continues. 
Theoretically, there are no limitations on how complex an object-based audio mix can be. In practice, however, it’s more limited but still quite impressive. "In the cinema, Dolby Atmos supports up to 64 loudspeakers," Gould says. "In our home renderer, we support up to 30 speakers." 
Even though immersive audio playback technology is widespread and well within the grasp of the consumer, music still has to be made into an immersive format. Doing so is the realm of the recording studio.
Audio producer and engineer Webster Tileston first began to explore immersive audio in 2020 when he experienced binaural audio on headphones. Binaural audio is a kind of three-dimensional effect that can be realized with just two channels, making it ideally suited for listeners using headphones or earbuds. Immersive audio has since become a significant part of his work. 
"My excitement came from the fact that I could mix in Dolby Atmos and deliver a single file format," says Tileston, an Atmos Mix Engineer at Nashville studio Axis Audio. "And I can use that one mixing process for pretty much any playback the client needs." That scalability makes spatial/immersive audio an attractive option. 
Growing use is bringing down the cost of immersive audio technology. "There’s a misconception that you have to spend a ton of money," Tileston tells GRAMMY.com. "There are a lot of affordable ways to get into it." The technology is also remarkably user-friendly, allowing for more creative control. "I'm able to place things in a much more musical way than I would have been able to in stereo," Tileston says. "Instead of having to try to fake depth perception and 'dimensionality' with reverb, I'm now able to accomplish that right away with the space that's around me."
Gould admits that as recently as two years ago, there was still skepticism about Atmos. But at least three popular music streaming services — Amazon Music Unlimited, Apple Music and Tidal — have begun providing Atmos content; Apple's acceptance of immersive audio in May 2021 was a game-changer. 
"Everyone went from, ‘How will anyone hear this?’ to 'Oh, this is real, and I need to be on board with it,'" says Dolby’s Gould. His company has seen rapid growth, with more than 550 studios now outfitted with Atmos. "A lot of that has come in the past 12 months. Everyone — producers, engineers and artists — has been very excited about the possibilities," he says.
Recognizing immersive audio’s importance as both a technological innovation and a tool for creative artistic expression, the Recording Academy created a new award in 2005. "A group of highly respected members from the Academy’s Producers and Engineers Wing initiated the idea," explains Michael Almanza, manager of the Academy’s Immersive Audio category. 
Originally called Best Surround Sound Album and renamed in 2019 Best Immersive Audio Album, the category debuted at the 47th GRAMMY Awards. Winners that year were the production team behind Ray Charles’ Genius Loves Company. Subsequent winners have included producers of recordings by the Beatles, Beyoncé, Dire Straits, Roger Waters and Alicia Keys.
Eligible recordings must be commercially released for sale or streaming and provide "an original immersive mix — not electronically re-purposed — of four or more channels," Almanza says. Yet a sometimes confusing array of competing file formats and physical media have made entering the immersive audio space challenging. In the past two years, the recording industry has begun to coalesce around Dolby Atmos and immersive streaming, eliminating much of the confusion.
Headed by GRAMMY-winning mastering engineer Michael Romanowski and George Massenburg, a committee at the Producers and Engineers Wing is currently developing a set of technical guidelines for immersive audio. 
When Leonard Bernstein composed his MASS in 1971, the work featured a pit orchestra, choirs, "street musicians" and rock band. MASS would be performed widely, but capturing and conveying the scope of the piece remained challenging. 
"The idea that Bernstein had was to make the audience feel immersed in the live concert experience," says Keith Lockhart, Conductor and Artistic Director of the Brevard Music Center’s Summer Festival. In 2018, BMC staged an ambitious reading of MASS using d&b audiotechnik's Soundscape system — which Bjork and Kraftwerk have also used on tour. 
Soundscape allowed for an effortless combination of recorded and live elements, North Carolina-based Lockhart notes. D&b is one of a handful of audio companies working on the leading edge of immersive audio for live performance. L-Acoustics and Meyer Sound have developed their own proprietary technologies as well.
L-Acoustics’ immersive technology, L-ISA, has been used by a diverse range of artists including alt-J, Soundgarden, Bon Iver, Katy Perry and Blue Man Group. Meyer Sound’s immersive technology has been employed in performance spaces like Brooklyn’s National Sawdust, theatrical presentations and the popular Immersive Van Gogh exhibition. The increasing prevalence of immersive audio technology demonstrates that audiences expect to be dazzled.
"Anything you can do to differentiate your show from the other shows out there," says Best Immersive Album GRAMMY nominee Steven Wilson, who has used immersive sound mixes in his concerts since 2011. "Whether it's having surround audio or incredible visual production – increasingly these things make a difference to justify the price of tickets and get people out of their front rooms and into the concert hall."
"Painting in sounds," says Wilson. "That’s really what making music is." As a solo artist, leader of Porcupine Tree and in-demand collaborator, Wilson has been a go-to remix producer for over a decade, re-imagining classic stereo albums from Genesis, Chicago, Black Sabbath, Tears for Fears, Kiss and others in immersive formats.
Wilson concedes that it’s possible to overdo things when all of these new tools are at hand. "The 'glue' of the track [can be] pulled apart if, for example, things become too isolated," he says. "In an Atmos mix you’ll have a guitar solo or a backing vocal isolated in one of the rear speakers; I’m wary of that." 
Wilson believes that some genres are especially well-suited for immersive audio. "It’s absolutely fantastic for electronic music," he says, noting that the nature of the music encourages expansive use of technology. Wilson’s approach to immersive mixes for rock music tends to be more conservative. "There's something about rock and roll," he says. "The drums and the bass and the guitars are all fighting each other coming out of a stereo or mono positioning; when you start to pull those elements apart, it starts to sound wrong." 
Emphasizing that he’s completely self-trained on Dolby Atmos and immersive technologies, Wilson notes that there’s only one way to determine what works and doesn’t. "You only discover those things by experimenting, by trial and error. I always feel I'm still learning." He’s clearly a quick study: his 2021 album The Future Bites received a Best Immersive Audio Album nomination.
"Audio has always been the experience of hearing sound," says Axis Audio’s Webster Tileston. "The immersive side of things gives us a more natural way to create and consume it." He notes that he has created immersive mixes from a few live recordings, just for fun. "I remember playing them back and thinking, 'I feel like I’m sitting in this club right now.' And that’s a cool feeling." 
L-Acoustics’ Le Nost believes that immersive audio "is going to become very mainstream, much more in our daily lives. I think this technology has a bright future." Ellison of Meyer Sound acknowledges immersive audio’s role in recorded music, but sees live performance as a major growth market. "You go somewhere that's a shared experience, and you can hear content in a way that you couldn't hear at home," he says.
Lopez of d&b audiotechnic notes that Broadway theaters have been employing immersive technology for nearly 20 years. Likewise, many artists desire to bring audio up to a level that matches their visual spectaculars. Immersive audio in the live context is "a higher-resolution way to [present] what’s going on, to recreate that intimacy, that emotional connection," Lopez says.
"At its core, immersive audio is an experience, regardless of how it is listened to," says the Recording Academy’s Almanza. "Immersive audio is its own art form. And as long as there is a pursuit of excellence in this now-widening field, the Academy will continue to recognize and award those creators."
We're Probably On An Irreversible Course Into The Metaverse. What Role Will Music Play In It?
RECORDING ACADEMY
MEMBERSHIP
GRAMMYS
ADVOCACY
MUSICARES
LATIN GRAMMYS
GRAMMY MUSEUM
@ 2022 - Recording Academy. All rights reserved.
Some of the content on this site expresses viewpoints and opinions that are not those of the Recording Academy. Responsibility for the accuracy of information provided in stories not written by or specifically prepared for the Academy lies with the story's original source or writer. Content on this site does not reflect an endorsement or recommendation of any artist or music by the Recording Academy.

source