Eclipsa Audio is the brand name for a new, open-source 3D spatial audio technology. It's built upon the Immersive Audio Model and Formats (IAMF) specification, developed as a collaborative effort from the Alliance for Open Media (AOMedia). IAMF technology is available under a royalty free license from AOMedia.
An open source Eclipsa Audio plugin is now available for Digital Audio Workstations (DAWs) and Non-Linear Editing (NLE) software :
- Open source code is available at: https://github.com/google/eclipsa-audio-plugin
- Pre-compiled plugin binaries are made available to download from www.eclipsaapp.com
IAMF: A New Era for Immersive Audio
IAMF is a new open-source audio container specification poised to revolutionize how we experience sound. Developed by AOMedia, with significant contributions from industry, including Google and Samsung, IAMF aims to deliver truly immersive, three-dimensional audio across a wide array of applications, platforms, and devices.
At its core, IAMF is designed to deliver a realistic and engaging 3D soundscape. IAMF allows audio to be anywhere in space, including above, below, and behind the listener, creating a vivid three dimensional sphere of sound. This creates a more lifelike "3D audio" experience.
IAMF is designed as a versatile and open-source audio container format with several key technical characteristics to enable immersive and interactive audio experiences:
- Codec-Agnostic Container: IAMF itself is not a codec but a container format. This means it can carry audio data compressed by various existing and future codecs, such as Opus, PCM, AAC, and FLAC.
- Support for Multiple Audio Types: IAMF can handle different types of audio presentations, also called Audio Elements in the IAMF specification:
- Channel-based audio: Such as 5.1.2 and 7.1.4, according to the Rec. ITU-R BS.2051-3
- Scene-based audio: Full ambisonics spherical soundfield
- 3D Spatial Audio Rendering: Open source based rendering to loudspeakers and binaurally for headphones.
- Metadata for Rendering and Customization: IAMF includes Mix Presentation metadata that specifies how to render, process and mix one or more Audio Elements:
- Creators can make user selectable Mix Presentations, for example enabling users to adjust dialog channel volume.
- Open Source Reference Software: AOMedia provides various open-source tools for developers:
- libiamf open source reference decoder
- iamf-tools for processing IAMF files
- Integration with Standard Media Containers: IAMF is designed to be integrated into common media container formats like MP4 (ISO-BMFF) for delivery with video content.
The IAMF specification includes a definition for profiles which determine how many audio elements and audio channels a corresponding IAMF file can include. The table below summarizes the profile requirements for the current IAMF specifications.
| Feature | IAMF v1.0 | IAMF v1.1 | |
|---|---|---|---|
| Profile | Simple | Base | Base Enhanced |
| Audio codec | Opus, AAC, FLAC, PCM | Opus, AAC, FLAC, PCM | Opus, AAC, FLAC, PCM |
| Max # of Audio Elements | 1 | 2 | 28 |
| Max # of audio channels | 16 | 18 | 28 |
Eclipsa Audio support in YouTube
Since January 2025, YouTube now accepts files with Eclipsa Audio (IAMF v1.0) and consumers can now play the content on a growing range of compatible devices, including Samsung's 2025 TV and soundbar lineup.
Eclipsa Audio playback in a YouTube TV app can be verified with two different ways (see the screenshot below):
- "Eclipsa Audio" should be visible in the Settings menu
- "Stats for nerds" view should show the "iamf.001.001.Opus" string in the Codecs section
Here's an example of Eclipsa Audio content on YouTube. The actual audio track in this video consists of 3rd order ambisonics and stereo, thus it includes two audio elements and in total 18 channels of audio. Ambient sounds are all in the 3rd order ambisonics track (16 channels) and narrative parts in the stereo track (2 channels). YouTube uses the Opus open source codec for compressing the audio channel data.
Eclipsa Audio Plugins for Sound Design
The Eclipsa Audio plugin consists of two parts:
- Eclipsa Audio renderer plugin: central hub for monitoring, configuration and export
- Eclipsa Audio element plugin: connects your audio elements (channels) to the renderer plugin, with optional basic panning functionality
First release of the Eclipsa Audio plugin is available for Avid Pro Tools with macOS support. While downloading the plugin binaries from www.eclipsaapp.com, you can sign up to receive updates on the upcoming new releases.
The Eclipsa Audio Renderer Plugin manages the overall 3D audio mix, enabling you to configure speaker setups, monitor your mix, and export the final mix in the IAMF format. Additionally, it's used to create audio elements and configure mix presentations, both of which are required for playback.
The Eclipsa Audio Renderer Plugin provides comprehensive export options to ensure your 3D audio mix is correctly formatted and optimized for immersive playback systems. Once the final mix is ready for export, you can also select a video track to be muxed with the IAMF audio track. The final MP4 file after export is ready to be uploaded to YouTube.
The Eclipsa Audio Element plugin should be added on every track you want to spatialize. This setup ensures each sound source is routed to the correct audio element and fully integrated into the 3D mix. To reduce the number of panners needed, Pro Tools' buses can also be used to route multiple tracks through an Audio Element plugin instance before routing the audio to the Eclipsa Audio Renderer Plugin. Pro Tools includes a great selection of built-in panning tools so it is recommended to use these tools for the actual sound mixing and use the pass-through option in the Audio Element plugin.
Next Steps
The Eclipsa Audio plugins continue to evolve. As an open source project, we invite developers to join and contribute.
By
