Introducing Open Source DAW Plugin for Eclipsa Audio

Eclipsa Audio is the brand name for a new, open-source 3D spatial audio technology. It's built upon the Immersive Audio Model and Formats (IAMF) specification, developed as a collaborative effort from the Alliance for Open Media (AOMedia). IAMF technology is available under a royalty free license from AOMedia.

An open source Eclipsa Audio plugin is now available for Digital Audio Workstations (DAWs) and Non-Linear Editing (NLE) software :

• Open source code is available at: https://github.com/google/eclipsa-audio-plugin
• Pre-compiled plugin binaries are made available to download from www.eclipsaapp.com

IAMF: A New Era for Immersive Audio

IAMF is a new open-source audio container specification poised to revolutionize how we experience sound. Developed by AOMedia, with significant contributions from industry, including Google and Samsung, IAMF aims to deliver truly immersive, three-dimensional audio across a wide array of applications, platforms, and devices.

At its core, IAMF is designed to deliver a realistic and engaging 3D soundscape. IAMF allows audio to be anywhere in space, including above, below, and behind the listener, creating a vivid three dimensional sphere of sound. This creates a more lifelike "3D audio" experience.

IAMF is designed as a versatile and open-source audio container format with several key technical characteristics to enable immersive and interactive audio experiences:

• Codec-Agnostic Container: IAMF itself is not a codec but a container format. This means it can carry audio data compressed by various existing and future codecs, such as Opus, PCM, AAC, and FLAC.
• Support for Multiple Audio Types: IAMF can handle different types of audio presentations, also called Audio Elements in the IAMF specification:
  • Channel-based audio: Such as 5.1.2 and 7.1.4, according to the Rec. ITU-R BS.2051-3
  • Scene-based audio: Full ambisonics spherical soundfield
• 3D Spatial Audio Rendering: Open source based rendering to loudspeakers and binaurally for headphones.
• Metadata for Rendering and Customization: IAMF includes Mix Presentation metadata that specifies how to render, process and mix one or more Audio Elements:
  • Creators can make user selectable Mix Presentations, for example enabling users to adjust dialog channel volume.
• Open Source Reference Software: AOMedia provides various open-source tools for developers:
  • libiamf open source reference decoder
  • iamf-tools for processing IAMF files
• Integration with Standard Media Containers: IAMF is designed to be integrated into common media container formats like MP4 (ISO-BMFF) for delivery with video content.

The IAMF specification includes a definition for profiles which determine how many audio elements and audio channels a corresponding IAMF file can include. The table below summarizes the profile requirements for the current IAMF specifications.

Feature	IAMF v1.0		IAMF v1.1
Profile	Simple	Base	Base Enhanced
Audio codec	Opus, AAC, FLAC, PCM	Opus, AAC, FLAC, PCM	Opus, AAC, FLAC, PCM
Max # of Audio Elements	1	2	28
Max # of audio channels	16	18	28

Eclipsa Audio support in YouTube

Since January 2025, YouTube now accepts files with Eclipsa Audio (IAMF v1.0) and consumers can now play the content on a growing range of compatible devices, including Samsung's 2025 TV and soundbar lineup.

Eclipsa Audio playback in a YouTube TV app can be verified with two different ways (see the screenshot below):

• "Eclipsa Audio" should be visible in the Settings menu
• "Stats for nerds" view should show the "iamf.001.001.Opus" string in the Codecs section

Here's an example of Eclipsa Audio content on YouTube. The actual audio track in this video consists of 3rd order ambisonics and stereo, thus it includes two audio elements and in total 18 channels of audio. Ambient sounds are all in the 3rd order ambisonics track (16 channels) and narrative parts in the stereo track (2 channels). YouTube uses the Opus open source codec for compressing the audio channel data.

Eclipsa Audio Plugins for Sound Design

The Eclipsa Audio plugin consists of two parts:

• Eclipsa Audio renderer plugin: central hub for monitoring, configuration and export
• Eclipsa Audio element plugin: connects your audio elements (channels) to the renderer plugin, with optional basic panning functionality

First release of the Eclipsa Audio plugin is available for Avid Pro Tools with macOS support. While downloading the plugin binaries from www.eclipsaapp.com, you can sign up to receive updates on the upcoming new releases.

The Eclipsa Audio Renderer Plugin manages the overall 3D audio mix, enabling you to configure speaker setups, monitor your mix, and export the final mix in the IAMF format. Additionally, it's used to create audio elements and configure mix presentations, both of which are required for playback.

The Eclipsa Audio Renderer Plugin provides comprehensive export options to ensure your 3D audio mix is correctly formatted and optimized for immersive playback systems. Once the final mix is ready for export, you can also select a video track to be muxed with the IAMF audio track. The final MP4 file after export is ready to be uploaded to YouTube.

The Eclipsa Audio Element plugin should be added on every track you want to spatialize. This setup ensures each sound source is routed to the correct audio element and fully integrated into the 3D mix. To reduce the number of panners needed, Pro Tools' buses can also be used to route multiple tracks through an Audio Element plugin instance before routing the audio to the Eclipsa Audio Renderer Plugin. Pro Tools includes a great selection of built-in panning tools so it is recommended to use these tools for the actual sound mixing and use the pass-through option in the Audio Element plugin.

Next Steps

The Eclipsa Audio plugins continue to evolve. As an open source project, we invite developers to join and contribute.

By Jani Huoponen, Felicia Lim, Jan Skoglund - Open Media Audio Team

YouTube releases scripts to help partners and creators to optimize their work

At YouTube Technology Services, we believe that open source software is essential for driving innovation and collaboration in the YouTube ecosystem. We want to make automation on YouTube more accessible by providing publicly available scripts to automate common use cases, aiming to decrease the cost for partners and creators to handle the most common scenarios when managing their content on YouTube.

In order to do so, we are announcing a new GitHub Organization, YouTubeLabs, where you will find open source code examples in the code-samples repository. We are providing open source scripts for a variety of use cases, including but not limited to:

• Bulk video editing
• Analyzing your YouTube Analytics data
• Downloading and analyzing your YouTube Studio Content Manager Reports
• Content democratization and internationalization

Most code samples rely on public YouTube APIs or Google APIs and are well-documented and well-commented, in order to be easily modified by partners and creators.

We are delivering code that aims to be as accessible as possible to our partners and creators, with minimal configurations and minimal installation required. That's why we rely on Colaboratory Notebooks (Colab) and AppsScript as the main pillars of our open source offering. Colab is a free, cloud-based Jupyter notebook environment that makes it easy to run Python code in the browser, and it is integrated with Google Drive. AppsScript is a serverless platform that allows you to write scripts that run on Google's servers.

We believe that open source software is key to the future of the YouTube ecosystem. By making our code available to the public, we are helping to empower partners and creators to do more with YouTube.

Want to get started? Check out some of the code examples already available in YouTubeLabs’ code-sharing repository:

• Tutorial on how to use Colab – this is a great place to start if you’re new to using Colab!
• Create Shorts from a longform video – create a few shorts for your audience from your popular longform videos (local video file required).
• Download your available Studio Content Manager reports using the Reporting API – instead of manually downloading your reports from the Studio Content Manager, consider downloading them from the Reporting API to your Google Drive using this code sample.
• Translate captions and synthesize - create automated caption translations as well as automated synthetisation (Text to-speech generation) for a given video.

We look forward to continuing to build out our open source examples in the coming months, so don’t forget to “like and subscribe” to our repository to stay tuned for more!

By Federico Villa and Haley Schafer – Partner Technology Managers on behalf of YouTube Technology Services

Podcast to YouTube: an open source story

Almost a year ago Mark Mandel and I started the Google Cloud Platform Podcast, a weekly podcast that covers topics related to Google Cloud Platform, among other things. It's been a pretty successful podcast, but that’s not what I want to write about today.

After a while we started receiving emails from listeners that wanted to access our podcast on YouTube. Even though this might seem strange for those that love podcasts and have their favorite app on their phones, we decided that the customer is always right: we should post every episode to YouTube.

Specifications

Ok, so … how? Well, to create a video I need to merge the mp3 audio from an episode with a static image. Let's include the title of the episode and the Google Cloud Platform Podcast logo.

But once we post the video to YouTube we're going to need more than that! We need a description, some tags, and probably a link to the episode (SEO FTW!).

Where can we get that information from? Let's think about this for a minute. Where are others getting this information from? The RSS feed! Would it be possible to create a tool to which I could say "post the video for episode 46" and a couple minutes later the video appeared on YouTube? That'd be awesome! Let's do that!

Architecture

The application I wrote parses an RSS feed and given the episodes to publish it downloads the metadata and audio for an episode, generates the corresponding videos, and pushes them to YouTube.

Diagram of the flow of data in podcast-to-youtube

The hardest parts here are the creation of the image and the video. The rest is sending HTTP requests right and left.

Image Maker: rendering images in pure Go

After trying a couple of different tools I decided that the easiest was to create the image from scratch in Go using the image package from the standard library and a freetype library available on GitHub.

Probably the most fun part was to be able to choose a font that would make the title fit the image correctly regardless of the length in characters. I ended up creating a loop that:

• chooses a font and measures the width of the resulting text
• if it's too wide, decreases the font size by one and repeats.

Surprisingly, for me, this is actually a pretty common practice!

It is also worth mentioning the way I test the package: Using a standard image that I compare to the one generated by the package, then showing a "diff" image where all the pixels that differ are highlighted in red.

Diff image generated when using a wrong DPI.

The code for this package is available here.

Video maker: ffmpeg is awesome

From the beginning I knew I would end up using ffmpeg to create my video. Why? Well, because it is as simple as running this command:

$ ffmpeg -i image.png -i audio.mp3 video.mp4

Easy right? Well, this is once ffmpeg has been installed and correctly configured, which is actually not that simple and would make this tool hard to install on any machine.

That's why the whole tool runs on Docker. Docker is a pretty widespread technology, and thanks to Makefile I'm able to provide a tool that can be run like this:

$ make run

Conclusion

It took me a couple of days to write the tool and get it to a point where I could open source it, but it was totally worth it. I know that others will be able to easily reuse it, or even extend it. Who knows, maybe this should be exposed as a web application so anyone can use it, no Docker or Makefile needed!

I am currently using this tool weekly to upload the Google Cloud Platform Podcast episodes to this playlist, and you can find the whole code on this GitHub repository.

Any questions? I'm @francesc on Twitter.

By Francesc Campoy, Developer Advocate