This Week in Open Source #7

This Week in Open Source for 08/08/2025

A look around the world of open source
by Daryl Ducharme, Google Open Source

Upcoming Events

• August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
• August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.
• September 5-7: NixCon 2025 is happening in Switzerland. It is the annual conference for the Nix and NixOS community where Nix enthusiasts learn, share, and connect with others.

Open Source Reads and Links

• The Asymmetry of Open Source - Open source software projects need funding, but users are not obligated to pay for them. Companies should invest in open source to maintain quality and avoid issues, while hobbyists can contribute without financial pressure. Proper boundaries and mutual responsibility between companies and developers are essential for a healthy open source ecosystem. How do we find and set those boundaries?
• Linux Foundation Announces Intent to Form Developer Relations Foundation - The Linux Foundation has created the Developer Relations Foundation which aims to unify best practices and enhance the role of developer relations in technology. The DRF will focus on collaboration and shared knowledge. Having an open source organization behind this, helps to make sure DevRel is always of service to developers along with whoever is employing them.
• 5 tips to get started on accessibility - Not exactly open source and yet super important. So important to the open source community that All Things Open posted it on their site. Accessibility (A11y) is always useful. The more it gets used properly, the more useful it is for everyone.
• Bringing open source development to Trust and Safety - Ever open source champion, former Googler and now COO at Roost, Anne Bertucio discusses how some teams still have a difficult time understanding open source. The standards that they are used to don't always occur within the transparent world of open source. This means, bringing open source to those teams requires understanding where they are coming from and discussing its limitations as well as its benefits.
• How we made JSON.stringify more than twice as fast - One of the beautiful things about open source is the transparency in projects. Google's Chromium V8 engine is no exception. This walk through of the technical structuring that led to a faster JSON.stringify is a great way to learn some approaches to solving software bottlenecks that you may not have thought of. With it being open source, you can also visit the repository and follow along with the history of these code changes.

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

What's new in Apache Iceberg v3?

A Deeper Dive into Apache Iceberg V3: How New Designs Are Solving Core Data Lake Challenges

The Next Chapter for Apache Iceberg: Welcoming the Iceberg V3 Spec
by Talat Uyarer, BigQuery Managed Iceberg & Shane Glass, Google Open Source Programs Office

The data community has long grappled with the challenge of how to bring database-like agility to petabyte-scale datasets stored in open cloud storage. The trade-off has often been between the scalability of data lakes and the performance and ease-of-use of traditional data warehouses. Executing fine-grained updates or evolving table schemas on massive tables often required slow, expensive, and disruptive operations.

The Apache Iceberg project is taking on this challenge. Early versions introduced a revolutionary metadata layer that brought reliability and ACID transactions to data lakes. However, certain operations still presented performance bottlenecks at scale.

With the ratification of the V3 specification, the Apache Iceberg community has introduced new designs that directly address these core issues. These advancements represent a significant leap forward in the mission to build an open and high-performance data lakehouse architecture. Let's explore the technical details of these solutions.

More Efficient Row-Level Transactions with Deletion Vectors

A primary challenge for data lakes has been handling row-level deletes efficiently. Previous approaches, like positional delete files, were a clever solution but could lead to performance degradation at query time when a reader had to reconcile many small delete files against large data files.

The Iceberg V3 spec introduces binary deletion vectors, a more performant and scalable architecture. The core idea is to attach a bitmap to each data file, where each bit corresponds to a row, marking it as deleted or not.

When a query engine reads a data file, it also reads its corresponding deletion vector. As it scans rows, it can check the bitmap with minimal overhead and skip rows marked for deletion. This design is made exceptionally efficient through the use of Roaring bitmaps. This data structure is ideal for this task because it can compress sparse sets of integers—like the positions of deleted rows—into a tiny footprint.

The practical difference is profound:

• Previous Model (Positional Deletes): A query might involve reading a central log of deletes, like deletes.avro, containing tuples of (file_path, row_position).
• V3 Model (Deletion Vectors): Each data file (e.g., file_A.parquet) is paired with a small, efficient sidecar file (e.g., file_A.puffin) containing a Roaring bitmap of its deleted rows.

This change localizes delete information, streamlines the read path, and dramatically improves the performance of workloads that rely on frequent Change Data Capture (CDC) or row-level updates.

Simplified Schema Evolution with Default Column Values

Another common operational hurdle in managing large tables has been schema evolution. Adding a column to a table with billions of rows traditionally required a "backfill"—a costly and time-consuming job to rewrite all existing data files to add the new column.

Iceberg V3 eliminates this friction with default column values. This feature allows a default value to be specified directly in the table's metadata when a column is added.

ALTER TABLE events ADD COLUMN version INT DEFAULT 1;

This operation is instantaneous because it only modifies metadata. No data files are touched. When a query engine encounters an older data file without the version column, it consults the table schema, finds the default value, and seamlessly populates it in the query results on the fly. This simple but powerful mechanism makes schema evolution a fast, non-disruptive operation, allowing data models to evolve quickly.

Improved Query Engine Compatibility with Enhanced Data Types and Lineage

Beyond these headline features, V3 broadens the capabilities of Iceberg to support more advanced use cases:

• Row-Level Lineage: For robust auditing and reliable CDC pipelines, V3 formalizes the tracking of row history. By embedding metadata about when a row was added or last modified, Iceberg tables can now provide a clear lineage, simplifying data governance and enabling more efficient downstream data replication.
• Rich Data Types: V3 closes the gap with traditional databases by introducing a more expressive type system. This includes a VARIANT type for handling semi-structured data like JSON, native GEOMETRY and GEOGRAPHY types for advanced geospatial analysis, support for nanosecond-precision timestamps with the new timestamp_ns and timestamptz_ns data types, a significant increase from the previous microsecond limit.

Building the Future of the Open Data Lakehouse

These V3 features—deletion vectors, default values, row lineage, and richer types—are more than just individual improvements. Together, they represent a cohesive step toward a new paradigm where the lines between the data lake and the data warehouse are erased. They enable faster, more efficient, and more flexible data operations than previously thought possible.

This progress is a testament to the collaborative spirit of the Apache Iceberg community. At Google, we are proud to contribute to and support open-source projects like Iceberg that are defining the future of data architecture. We are excited to see the innovative applications the community will build on this powerful new foundation.

Want to get started with Iceberg? Check out this blog post to learn more about how Google Cloud's managed Iceberg offering, BigLake tables for Apache Iceberg in BigQuery, makes building Iceberg-native lakehouses easier by maximizing performance without sacrificing governance.

---

This Week in Open Source #6

This Week in Open Source for 08/01/2025

A look around the world of open source

by Daryl Ducharme & amanda casari, Google Open Source Programs Office

Diving into the open source world this week, we'll cover upcoming events that foster collaboration and innovation, alongside new reads and links that highlight significant advancements and discussions within the open source community. From new Google projects enhancing package ecosystem confidence to thought-provoking articles on open source funding, we hope this keeps you aware of new areas of the ecosystem.

Upcoming Events

• August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
• August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.
• September 5-7: NixCon 2025 is happening in Switzerland. It is the annual conference for the Nix and NixOS community where Nix enthusiasts learn, share, and connect with others.

Open Source Reads and Links

• [Blog] Google introduced OSS Rebuild, a new project designed to enhance confidence in open source package ecosystems through the reproduction of upstream artifacts.
  • [Article] Hacker news posted Google Launches OSS Rebuild to Expose Malicious Code in Widely Used Open-Source Packages which also discusses the usefulness and importance of this tool
• [Story] SF-Based Internet Archive Is Now a Federal Depository Library. What Does That Mean? - The Internet Archive is a foundational reference and repository for open-access information and digital archives.The San Francisco-based digital library now has federal depository status, joining a network of over 1,100 libraries that archive government documents and make them accessible to the public — even as ongoing legal challenges pose an existential threat to the organization.
• [Video] Keynote: Building community through collaborative datasets - Mago Torres' keynote from csv,conf 8, on her work building collaborative datasets for award-winning data journalism, captures the spirit and focus on where open technology enables communities to accomplish more together.
• [Paper] Anubis Pilot Project Report - June 2025 - In May and June 2025, Duke University Libraries (DUL) successfully implemented Anubis, a configurable open source web application firewall (WAF), to combat persistent AI-related bot scraping. During this pilot (May 1 - June 10, 2025), aggressive bot scraping caused outages for three critical library platforms (Duke Digital Repository, Archives & Manuscripts, and the Books & Media Catalog); Anubis mitigated the problem in each instance.
• [Article] Microsoft-owned GitHub says open source needs to be funded - The Register published this editorial which asks whether open source software has reached the point that it should be managed as infrastructure and funded by governments that rely on it? Some studies show impressive numbers in how much it contributes to many economies.
• [Blog] Open Source Explained Like You're Five (But Smarter) - Explaining open source to people outside the tech world is tough. This article uses some good metaphors along with some details you may not have known to better explain it and spread the word. Or, you could just send them this article and hope they read it. 😜

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

This Week in Open Source #5

This Week in Open Source for July 25, 2025

A look around the world of open source

by Daryl Ducharme & amanda casari, Google Open Source Programs Office

We hope everyone is having a good summer. The world of open source is, with more events and news that caught our attention.

Upcoming Events

• July 31-August 3: FOSSY (Free and Open Source Software Yearly) will be held in Portland, Oregon and is focused on the creation and impact of free and open source software, uplifting contributors of all experience.
• August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
• August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.

Open Source Reads and Links

• [Press Release] Tech Veterans Anne Bertucio and Vinay Rao Join ROOST - A bit of a bittersweet post as our recent, now former Head of Open Source Programs Office, Anne Bertucio, joins ROOST as COO and the previous Head of Safeguards at Anthropic, Vinay Rao, joins as CTO.
• [Article] An open-source SDK for finding dead code - Maintaining dead code is a waste of resources. So, having good tools for finding dead code in your applications is important. The open sourcing of Reaper for iOS and Android applications might be a worthwhile part of your toolbelt.
• [Blog] Why I used to prefer permissive licenses and now favor copyleft - Choosing the right license for your open source projects is a very personal choice. A choice that is worth revisiting once in a while to see if your values have shifted and if there are new ideas for what might constitute free software that better align with those new values.
• [Blog] Announcing FOKS: The Federated Open Key Service - Security and authentication are key to the tech world and open source is a good way to get many eyes on the problems to find solutions. A new federated open key service, FOKS, built from the ground up and based on concepts while working with Keybase is available now.
• [Article] Kubernetes Surges in Enterprise, But What Can Take It Mainstream? - Different teams in the development work streams have their own ideas about the tech stack. Many teams using Kubernetes have made it quite popular for use in enterprise work, but some are still using systems that have been tried and tested in their own domains. What work needs to be done to get all teams on-board with using Kubernetes?
• [Blog] Death by a thousand slops - The lead maintainer for the open source project, curl, continues to blog on where low-quality recommendations to curl's Bug Bounty program are increasing the work for the security team.
• [Article] From A2A to MCP, a look at the protocols that might one day help AI automate you out of a job - Click-bait headline aside, a good overview of where these protocols are at, what they do, and a certain view on whether that's useful or not. We have our opinions, but we are probably biased ;)
• [Article] How the Free Software Foundation battles the LLM bots - There are many bots out there crawling the web. In the early days of search, the solution was the robots.txt files and bots crawling the web slow enough for the systems to continue to run smoothly. However, many LLM bots are ignoring robots.txt, being greedy with site resources, and that's on top of other bot traffic to deal with. Looking at how a large organization approaches this current trend has some great shared knowledge.

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Stop Leaked Credentials in Their Tracks with Veles, Our New Open-Source Secret Scanner

by Kevin Dungs, Charl de Nysschen & Sarah Lucas, Google

In today's complex software supply chain, a single leaked credential—an API key, a service account token, a password—can be all an attacker needs to breach your systems. These secrets can be accidentally committed to a source code repository, embedded in a container image, or attached to a support ticket, creating a critical and often invisible risk.

To help developers and security teams proactively find and fix these exposures, we are excited to announce Veles, a new open-source secret and credential scanner from Google.

Veles is designed to detect unintended exposure of sensitive credentials across your organization's internal systems. It helps you find secrets where they don't belong, so you can prevent them from being abused.

Why Veles? Key Features

Veles is a new, standalone module within our OSV-SCALIBR (Software Composition Analysis LIBRary) ecosystem, but it is built to be used independently. This means you can easily integrate it into your existing security tooling or use it as a standalone scanner.

In its initial release, Veles helps you find high-risk secrets in source code and user-provided artifacts. Our detection library currently identifies:

• Google Cloud Platform (GCP) API Keys
• GCP Service Account Keys
• RubyGems API Keys

This is just the beginning. Veles is built to be extensible, allowing for the rapid addition of new secret types.

Battle-Tested at Google: Powerful Real-World Integration

At Google, we're not just releasing Veles; we're actively using it to protect our own systems and the open-source ecosystem.

• Internal Protection: Veles is already scanning Google's internal source code repositories and artifacts, helping us find and remediate leaked secrets before they become a problem.
• Securing the Open Source Ecosystem: The Google Open Source Security Team is incorporating Veles into its pipeline that powers deps.dev, scanning hundreds of millions of open-source artifacts (packages, Docker images, and repositories) to detect and remediate leaked credentials across the community.
• Integration with Google Cloud Products: Veles is being integrated directly into Google Cloud security services to bring secret scanning to our customers:
  • Artifact Analysis & Artifact Registry: Veles will power secret scanning in Artifact Registry, with findings surfaced through the Container Analysis API and, eventually, in the Artifact Registry UI.
  • Security Command Center (SCC): SCC's integration will provide comprehensive secret detection across the entire cloud lifecycle. This means scanning "left" into the development pipeline (like Infrastructure as Code) and "right" into active runtime environments (like Compute Engine and GKE). SCC will then unify these findings, helping you prioritize the most critical exposures and visualize potential attack paths.

The Road Ahead: What's Next for Veles?

This first release is a foundational step. Our roadmap for Veles includes:

• Broader Detection: We will continuously expand the library of supported secret and credential types.
• Automated Validation: We plan to add functionality to intelligently validate if a discovered secret is active.
• Remediation Workflows: In the future, we aim to help automate the revocation of confirmed, leaked secrets.

Get Started with Veles Today

Veles is open-source and ready for you to use. You can integrate it into your CI/CD pipeline, run it against your existing repositories, or contribute to its development. Protecting your organization from leaked credentials is a critical part of a strong security posture, and Veles is here to help.

Ready to start scanning? Head over to the Veles GitHub repository to get started!

This Week in Open Source #4

This Week in Open Source for July 18, 2025

A look around the world of open source
by Daryl Ducharme & amanda casari, Google Open Source Programs Office

Getting into the middle of July, we've been reading lots of various articles. Here's the upcoming events and some of our favorites.

Upcoming Events

• July 24-29: GUADEC 2025, the Gnome community's largest conference is in Brescia, Italy.
• July 31-August 3: FOSSY (Free and Open Source Software Yearly) will be held in Portland, Oregon and is focused on the creation and impact of free and open source software, uplifting contributors of all experience.
• August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.

Open Source Reads and Links

• [Article] Open Source at the Crossroads of AI Opportunity and Cybersecurity Necessity - A follow up to many of the events from Open Source Summit North America with much discussion around AI's impact on labor and cybersecurity.
• [Paper] A Toolkit For Measuring The Impacts Of Public Funding On Open Source Software Development - Funding in the open source world is complex. This toolkit seeks to inform discussions about the value of public funding for OSS.
• [Release Notes] Mastodon 4.4 - The open source microblogging platform has had another release milestone. Check out enhancements like: Featured posts & tags, improved list management, media control improvements (run a podcast from a mastodon account!), quote posts are coming, and more.
• [Articles] Digital sovereignty leads to multiple governments, from local to national, choosing to move from proprietary software to open source options. For examples Denmark Set to Replace Microsoft Office with Open Source Alternative and the French City of Lyon Kicks Out Microsoft and Data Sovereignty in Uncertain Times discussing how you should look at all your cloud provider options.

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Unlocking High-Performance AI/ML in Kubernetes with DRANet and RDMA

DraNet Enters Beta! High-Performance Networking in Kubernetes

by Antonio Ojea & Federico Bongiovanni, Kubernetes/GKE

We are excited to announce that DraNet has officially entered a beta state! This marks a major leap forward in our mission to streamline and enhance high-performance networking for AI and HPC workloads within Kubernetes. As we progress towards a stable General Availability (GA) release, we are eager to gather your feedback on the current state of the project.

Why DraNet?

DraNet was born from the lessons we learned at Google, observing the challenges end-users faced when running AI and HPC workloads on Kubernetes. The existing networking solutions, often repurposed from traditional networking or bespoke and complex, fell short of providing a good user experience and efficient operational models.
For instance, managing RDMA (Remote Direct Memory Access) interfaces often involved a complex combination of CNI chaining and device plugins. This not only created an unnecessary operational overhead for administrators but also led to coordination issues between different components that needed to work in harmony impacting resilience and scalability.
Another significant pain point we identified was the need for fine-grained interface tuning. AI workloads, for example, are extremely sensitive to latency. The presence of some eBPF programs on network interfaces, or the need to configure specific NIC parameters, could severely impact performance latency and/or throughput. Users were often forced to create custom init containers just to apply these settings, adding another layer of complexity.

Introducing DraNet: A Native and Declarative Solution

DraNet is a native integration with Kubernetes that uses the core Dynamic Resource Allocation (DRA) API to address these challenges by treating high-performance network interfaces as first-class citizens in Kubernetes. Here's how:

• Simplified RDMA Management: DraNet manages RDMA interfaces natively, handling the different requirements to offer a unified and seamless user experience. No more need for coordinating different components.
• Declarative Interface Tuning: With DraNet, you can declaratively set interface properties. Need to disable eBPF programs to reduce packet processing overhead or set specific NIC parameters? You can now do this directly in your Kubernetes manifests, eliminating the need for custom scripts or init containers.
• Standalone and Secure: DraNet is designed as a standalone binary, allowing it to run in a distroless container. This significantly reduces the attack surface and the frequency of security-related updates for the container image. By interacting directly with the kernel via stable APIs like netlink, it avoids dependencies on third-party projects, improving both resilience and performance.
• Lightweight and Fast: The DraNet container image, with a compressed size of less than 50MB, has a direct impact on node startup times, allowing for faster deployment and scaling of your workloads.

Beta Release and the Road to GA

DraNet is now in a beta state, signifying that it is ready for broader community testing and feedback. This move to beta is aligned with the maturation of the Kubernetes Dynamic Resource Allocation (DRA) KEP (KEP-4381), a foundational technology for DraNet. We are continuing our active development as we work towards a future General Availability release.

We Welcome Your Feedback and Contributions!

DraNet is an open-source project, and we believe that community involvement is key to its success. As we work towards our GA release, we welcome your feedback, whether it's on the design, user experience, or performance.
You can contribute in many ways:

• Code contributions: We have a fast-paced development cycle and welcome new contributors. Check out our contributing guidelines to get started.
• Documentation: Help us improve our documentation to make it easier for new users to get started with DraNet.
• Share your opinion: Your feedback is invaluable. Let us know how you are using DraNet and what we can do to make it better.

To learn more about DraNet and get started, please visit https://dranet.dev/. We look forward to building the future of high-performance networking in Kubernetes with you!

This Week in Open Source #3

This Week in Open Source for July 11, 2025

A look around the world of open source
by Daryl Ducharme, Erin McKean & amanda casari, Google Open Source Programs Office

We took a break as there was a holiday in the US that shortened our work week, but we are back to share what our open source world has to offer.

Upcoming Events

• July 14-19: The 26th annual Debian Conference (DebConf) for Debian contributors and users interested in improving Debian is in Brest, France.
• July 24-29: GUADEC 2025, the Gnome community's largest conference is in Brescia, Italy.
• July 31-August 3: FOSSY (Free and Open Source Software Yearly) will be held in Portland, Oregon and is focused on the creation and impact of free and open source software, uplifting contributors of all experience.

Open Source Reads and Links

• [Article] libxml2 Maintainer Ends Embargoed Vulnerability Reports, Citing Unsustainable Burden - Solo maintainer of small project won't embargo security vulnerabilities any more; will just treat them like regular bugs. This might bring up security concerns for those that use it but it highlights the need for more contributions required for important work.
• [Podcast] Managing community contributors with Alya Abbott - In this episode of The Business of Open Source podcast the COO of Zulip discusses the management of community contributions, including their downsides.
• [Blog] Towards Digital Sovereignty: Decisions About Control We Make Every Day - Digital sovereignty. What is it? Where did it come from? And how might we best think about it based on our personal or company needs?
• [Report] The Python Software Foundation 2024 Annual Impact Report - PSF has released their annual organization report for community accountability, transparency, and discussion.
• [Mailing List message] What's Next? - This mailing list message shows new life in the development of the copyleft-next license.
• [Blog] Command line magic: Extracting links with awk, grep, and tr - Sometimes it's nice to be reminded of the tools we have at our disposal and how we can use them to make everyday tasks easier. This article from an ATO contributor shows some tips and tricks for a little command line magic that we may have forgotten about (or never thought of).

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

This Week in Open Source #2

This Week in Open Source for June 27, 2025

A look around the world of open source
By Daryl Ducharme & amanda casari - Google Open Source Programs Office

With Open Source Summit North America (OSSNA) this week, it has been an exciting week.

OSSNA Keynote Announcements and more you may have missed

• Google transfers Agent 2 Agent (A2A) protocol to the Linux Foundation.
• Model Signing v1.0 was released by OpenSSF in April.
• Nvidia shipped over 1 Billion RISC-V cores in 2024.
• Overture Maps Foundation now has over 1 billion points of interest.
• Tazama - Open source software to provide realtime prevention of financial fraud. Especially useful in developing areas.
• Cloudflare now supports C2PA - With our current technology, content provenance assurances are more important than ever.
• CNCF launched GitJobs.
• Fair Package Manager Project for Wordpress provides open source content management system stability.

Upcoming Events

• July 7-13: The 24th annual SciPy conference will be held in Tacoma, Washington. It brings together attendees from industry, academia, and government to showcase their latest Python projects, learn from skilled users and developers, and collaborate on code development.
• July 8-9: The Beam Summit is happening in New York City. It is the leading conference for Apache Beam, the unified programming model for batch and stream data processing.
• July 14-19: The 26th annual Debian Conference (DebConf) for Debian contributors and users interested in improving Debian is in Brest, France.
• July 24-29: GUADEC 2025, the Gnome community's largest conference is in Brescia, Italy.

Open Source Reads and Links

• Everything Open Source - Open Source Events - Everything Open Source does many things in and for open source - maintaining a community list of events for the past three years is just one of them.
• Keeping the Web Up Under the Weight of AI Crawlers - Interesting explanation of robots.txt open protocol and how the open web is being challenged by needed updates/changes from AI crawlers.
• A Family of Forks - Daniel Stenberg, lead maintainer of curl, describes the work required to keep curl getting built with 11 modern Transport Layer Security (TLS) libraries
• Introducing the new API for OSI approved licenses - OSI has a new API for checking against approved open source licenses.
• What's new in Apache Iceberg 3.0 - The open data lakehouse keeps getting better with Iceberg v3. This article goes into the details and also takes a peak at what should be coming in v4.
• What would Kubernetes 2.0 look like? - Kubernetes is all over the tech world. So, what change to the defaults in K8s would keep it moving in that forward direction?
• Ubuntu 25.10 replaces sudo with a Rust based equivalent - While not quite at release yet, sudo-rs, a Rust based sudo replacement will soon be the norm on Ubuntu. Will other distros follow suit?

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

This Week in Open Source - Inaugural Post

This Week in Open Source for June 20, 2025

A look around the world of open source
By Daryl Ducharme - Google Open Source Programs Office

We're starting a new series here at the Google Open Source Programs Office. In an effort to spread the word of open source, we'll be writing a weekly series discussing announcements, events, and interesting articles about many different FOSS related topics from around the ecosystem.

Upcoming Events:

• June 23-25: The Open Source Summit North America (OSSNA) is next week in Denver, Colorado. A SWE from the Google Agent 2 Agent team will be delivering an exciting keynote on the future of the protocol.
• July 7-13: The 24th annual SciPy conference will be held in Tacoma, Washington. It brings together attendees from industry, academia, and government to showcase their latest Python projects, learn from skilled users and developers, and collaborate on code development.
• July 8-9: The Beam Summit is happening in New York City. It is the leading conference for Apache Beam, the unified programming model for batch and stream data processing.
• July 14-19: The 26th annual Debian Conference (DebConf) for Debian contributors and users interested in improving Debian is in Brest, France.

Open Source Reads

• [Article] New compiler faster than LLVM - A new compiler that is faster than the standard? Color me interested. Three researchers from the Technical University of Munich have developed TPDE, a new compiler backend framework. It combines multiple background tasks into a single pass.
• [List Article] 14 Open Source Tools To Become The Ultimate Developer - Yes, these types of articles come out all the time. But a curated list of new tools to look at is always a great way to get motivated and learn new things.
• [Announcement] GUAC 1.0 is now available - With current regulations (and pragmatically, just good software development practices) keeping up with your software bill of materials is important. However, the dependencies can be complex! GUAC helps you tame this complexity by applying graph logic to it.
• [Blog] Cloud Native and Open Source Help Scale Agentic AI Workflows - Why use a Large Language Model when a Smaller Language Model will work? You can with a few open source tools that happen to be Google grown - Kubernetes, KNative, and Istio.

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Introducing Open Source DAW Plugin for Eclipsa Audio

Eclipsa Audio is the brand name for a new, open-source 3D spatial audio technology. It's built upon the Immersive Audio Model and Formats (IAMF) specification, developed as a collaborative effort from the Alliance for Open Media (AOMedia). IAMF technology is available under a royalty free license from AOMedia.

An open source Eclipsa Audio plugin is now available for Digital Audio Workstations (DAWs) and Non-Linear Editing (NLE) software :

• Open source code is available at: https://github.com/google/eclipsa-audio-plugin
• Pre-compiled plugin binaries are made available to download from www.eclipsaapp.com

IAMF: A New Era for Immersive Audio

IAMF is a new open-source audio container specification poised to revolutionize how we experience sound. Developed by AOMedia, with significant contributions from industry, including Google and Samsung, IAMF aims to deliver truly immersive, three-dimensional audio across a wide array of applications, platforms, and devices.

At its core, IAMF is designed to deliver a realistic and engaging 3D soundscape. IAMF allows audio to be anywhere in space, including above, below, and behind the listener, creating a vivid three dimensional sphere of sound. This creates a more lifelike "3D audio" experience.

IAMF is designed as a versatile and open-source audio container format with several key technical characteristics to enable immersive and interactive audio experiences:

• Codec-Agnostic Container: IAMF itself is not a codec but a container format. This means it can carry audio data compressed by various existing and future codecs, such as Opus, PCM, AAC, and FLAC.
• Support for Multiple Audio Types: IAMF can handle different types of audio presentations, also called Audio Elements in the IAMF specification:
  • Channel-based audio: Such as 5.1.2 and 7.1.4, according to the Rec. ITU-R BS.2051-3
  • Scene-based audio: Full ambisonics spherical soundfield
• 3D Spatial Audio Rendering: Open source based rendering to loudspeakers and binaurally for headphones.
• Metadata for Rendering and Customization: IAMF includes Mix Presentation metadata that specifies how to render, process and mix one or more Audio Elements:
  • Creators can make user selectable Mix Presentations, for example enabling users to adjust dialog channel volume.
• Open Source Reference Software: AOMedia provides various open-source tools for developers:
  • libiamf open source reference decoder
  • iamf-tools for processing IAMF files
• Integration with Standard Media Containers: IAMF is designed to be integrated into common media container formats like MP4 (ISO-BMFF) for delivery with video content.

The IAMF specification includes a definition for profiles which determine how many audio elements and audio channels a corresponding IAMF file can include. The table below summarizes the profile requirements for the current IAMF specifications.

Feature	IAMF v1.0		IAMF v1.1
Profile	Simple	Base	Base Enhanced
Audio codec	Opus, AAC, FLAC, PCM	Opus, AAC, FLAC, PCM	Opus, AAC, FLAC, PCM
Max # of Audio Elements	1	2	28
Max # of audio channels	16	18	28

Eclipsa Audio support in YouTube

Since January 2025, YouTube now accepts files with Eclipsa Audio (IAMF v1.0) and consumers can now play the content on a growing range of compatible devices, including Samsung's 2025 TV and soundbar lineup.

Eclipsa Audio playback in a YouTube TV app can be verified with two different ways (see the screenshot below):

• "Eclipsa Audio" should be visible in the Settings menu
• "Stats for nerds" view should show the "iamf.001.001.Opus" string in the Codecs section

Here's an example of Eclipsa Audio content on YouTube. The actual audio track in this video consists of 3rd order ambisonics and stereo, thus it includes two audio elements and in total 18 channels of audio. Ambient sounds are all in the 3rd order ambisonics track (16 channels) and narrative parts in the stereo track (2 channels). YouTube uses the Opus open source codec for compressing the audio channel data.

Eclipsa Audio Plugins for Sound Design

The Eclipsa Audio plugin consists of two parts:

• Eclipsa Audio renderer plugin: central hub for monitoring, configuration and export
• Eclipsa Audio element plugin: connects your audio elements (channels) to the renderer plugin, with optional basic panning functionality

First release of the Eclipsa Audio plugin is available for Avid Pro Tools with macOS support. While downloading the plugin binaries from www.eclipsaapp.com, you can sign up to receive updates on the upcoming new releases.

The Eclipsa Audio Renderer Plugin manages the overall 3D audio mix, enabling you to configure speaker setups, monitor your mix, and export the final mix in the IAMF format. Additionally, it's used to create audio elements and configure mix presentations, both of which are required for playback.

The Eclipsa Audio Renderer Plugin provides comprehensive export options to ensure your 3D audio mix is correctly formatted and optimized for immersive playback systems. Once the final mix is ready for export, you can also select a video track to be muxed with the IAMF audio track. The final MP4 file after export is ready to be uploaded to YouTube.

The Eclipsa Audio Element plugin should be added on every track you want to spatialize. This setup ensures each sound source is routed to the correct audio element and fully integrated into the 3D mix. To reduce the number of panners needed, Pro Tools' buses can also be used to route multiple tracks through an Audio Element plugin instance before routing the audio to the Eclipsa Audio Renderer Plugin. Pro Tools includes a great selection of built-in panning tools so it is recommended to use these tools for the actual sound mixing and use the pass-through option in the Audio Element plugin.

Next Steps

The Eclipsa Audio plugins continue to evolve. As an open source project, we invite developers to join and contribute.

By Jani Huoponen, Felicia Lim, Jan Skoglund - Open Media Audio Team

Introducing New Open Source Documentation Resources

Today we're introducing two new open source documentation resources for open source software maintainers, a Docs Advisor guide and a set of Documentation Project Archetypes. These tools are intended to help maintainers make effective use of limited resources when it comes to planning and executing open source documentation work.

The Docs Advisor is a guide intended to demystify documentation work, including help picking a documentation approach, understanding your audience and available resources, and how to write, revise, evaluate, and maintain your documentation.

Documentation Project Archetypes are a set of thirteen project field guides. Each archetype represents a different type of documentation project, the problems it can solve, and how to bring the right collaborators together on the project to create great docs.

Origin story

More than 130 open source projects wrote 200+ case studies and project reports as a part of their participation in the Google Season of Docs program from 2019 to 2024. These case studies and project reports represent a variety of documentation projects from a wide range of open source groups. In these wrap-ups, project maintainers and technical writers describe how they approached their documentation projects, capturing many successes and more than a few challenges.

These reports are a treasure trove of lessons learned–but it's unrealistic to expect time-crunched open source maintainers to read through them all. So we got in touch with Daniel Beck and Erin Kissane to chat about ways to help organize and summarize some of these lessons learned.

These conversations turned into the Docs Advisor guide (‘like having an experienced technical writer hanging over your shoulder') and the thirteen Documentation Project Archetypes.

Our goal with these resources was to turn all of the hard-won experience of the Google Season of Docs participants into explicit documentation advice and guidance for open source maintainers.

More about the Docs Advisor

The Docs Advisor guide is intended to demystify the work of good documentation. It collects practices and processes from within technical writing and docs communities and from user experience, information architecture, and content strategy.

• In Part 1, you'll pick an overall approach that suits the needs of your project.
• In Part 2, you'll learn enough about your community and their needs to ensure that your hard work will be helping real people.
• In Part 3, you'll assess your existing resources and pull together everything you need to move quickly and confidently through the work of creating and revising your docs.
• In Part 4, you'll get to work writing and revising your docs and set yourself to successfully evaluate your work and maintain it.

The Docs Advisor guide also includes a docs plan template to help you accomplish your docs plan work, including:

• What approach will you take to your documentation work, as a whole?
• What risks do you need to mitigate?
• Are there any documents to make or steps to perform to increase your chances of success?

The Docs Advisor incorporates guidance from interviews with open source maintainers and technical writers as well as from the Google Season of Docs case studies, and integrates the Documentation Project Archetypes into the recommendations for maintainers planning docs work.

More about the Archetypes

Documentation Project Archetypes are meant to help you recognize common types of documentation work (whether you're writing a new user guide or replatforming your docs site), the situations in which they apply, and organize yourself to bring the work to completion.

The archetypes cover the following areas:

• Planning and evaluating your docs: Experiment and analysis archetypes support future docs work, by learning more about your existing docs, your audience, and your capacity to deliver meaningful change.
• Producing new docs: Creation archetypes make new docs that directly help your audience complete tasks and achieve their goals.
• Revising and transforming existing docs: Revision archetypes modify existing docs, to improve quality, reduce maintenance costs, and reach wider audiences.
• Equipping yourself with docs tools and process: Tool and process archetypes adopt new tools or practices that help you make more, better, or higher quality docs.

All of the archetypes are available on GitHub.

Doc tools in the wild

We are excited to share these tools and are looking forward to seeing how they are used and evolve.

Daniel demoed the concept and first completed archetype, The Migration, at FOSDEM 2025 in his talk Patterns for maintainer and tech writer collaboration. He also talked about the work on the API Resilience Podcast episode "Patterns in Documentation."

We hope to get valuable feedback during a proposed Doc Archetypes session at Open Source Summit Europe 2025 (acceptance pending).

We are also excited to be developing some Doc Archetype illustration cards with Heather Cummings — a few previews are already live on The Edit, The Audit, and The Factory.

If you have questions or suggestions, please raise an issue in the Open Docs repo.

By Elena Spitzer & Erin McKean, Google Open Source Programs Office

Transforming Kubernetes and GKE into the leading platform for AI/ML

The world is rapidly embracing the power of AI/ML, from training cutting-edge foundation models to deploying intelligent applications at scale. As these workloads become more sophisticated and demanding, the infrastructure required to support them must evolve. Kubernetes has emerged as the standard for container orchestration, but AI/ML introduces unique challenges that push traditional infrastructure to its limits.

AI training jobs often require massive scale, needing to coordinate thousands of specialized hardware like GPUs and TPUs. Reliability is critical, as failures can be costly for long running, large-scale training jobs. Efficient resource sharing across teams and workloads is essential given the expense of accelerators. Furthermore, deploying and scaling AI models for inference demands low latency and faster startup times for large container images and models.

At Google, we are deeply invested in the AI/ML revolution. This is why we are doubling down on our commitment to advancing Kubernetes as the foundational open standard for these workloads. Our strategy centers on evolving the core Kubernetes platform to meet the needs of the "next trillion core hours," specifically focusing on batch and AI/ML. We then bring these advancements, alongside enterprise-grade management and optimizations, to users through Google Kubernetes Engine (GKE).

Here's how we are transforming Kubernetes and GKE:

Redefining Kubernetes' relationship with specialized hardware

Kubernetes was initially designed for more uniform CPU compute. The surge of AI/ML brought new requirements for seamless integration and efficient management of expensive, sparse, and diverse accelerators. To support these new demands, Google has been a key investor in upstream Kubernetes to offer robust support for a diverse portfolio of the latest accelerators, including multiple generations of TPUs and a wide range of NVIDIA GPUs.

A core Kubernetes enhancement driven by Google and the community to better support AI/ML workloads is Dynamic Resource Allocation (DRA). This framework, developed in the heart of Kubernetes, provides a more flexible and extensible way for workloads to request and consume specialized hardware resources beyond traditional CPU and memory, which is crucial for efficiently managing accelerators. Building on such foundational open-source capabilities, GKE can then offer features like Custom Compute Classes, which improve the obtainability of these resources through intelligent fallback priorities across different capacity types like reservations, on-demand, and Spot instances. Google's active contributions to advanced resource management and scheduling capabilities within the Kubernetes community ensure that the platform evolves to meet the sophisticated demands of AI/ML, making efficient use of these specialized hardware resources more broadly accessible.

Unlocking scale and reliability

AI/ML workloads demand unprecedented scale and have new failure modes compared to traditional applications. GKE is built to handle this, supporting up to 65,000 nodes in a single cluster. We've demonstrated the ability to run the largest publicly announced training jobs, coordinating 50,000 TPU chips with near-ideal scaling efficiency.

Critically, we are enhancing core Kubernetes capabilities to support the scale and reliability needed for AI/ML. For instance, to better manage distributed AI workloads like serving large models split across multiple hosts, Google has been instrumental in developing features like JobSet (emerging from earlier concepts like LeaderWorkerSet) within the Kubernetes community (SIG Apps). This provides robust orchestration for co-scheduled, interdependent groups of Pods. We are also actively working upstream to improve Kubernetes reliability and stability through initiatives like Production Readiness Reviews, promoting safer upgrade paths, and enhancing etcd stability for the benefit of all Kubernetes users.

Optimizing Kubernetes performance for efficient inference

Low-latency and cost-efficient inference is critical for AI applications. For serving, the GKE Inference Gateway routes requests based on model server metrics like KVCache utilization and pending queue length, reducing serving costs by up to 30% and tail latency by 60% compared to traditional load balancing. We've even achieved vLLM fungibility across TPUs and GPUs, allowing users to serve the same model on either accelerator without incremental effort.

To address slow startup times for large AI/ML container images (often 20GB+), GKE offers rapid scale-out features. Secondary boot disks allow preloading container images and data, resulting in up to 29x faster container mounting time. GCS FUSE enables streaming data directly from Cloud Storage, leading to faster model load times. Furthermore, GKE Inference Quickstart provides data-driven, optimized Kubernetes deployment configurations, saving extensive benchmarking effort and enabling up to 30% lower cost, 60% lower tail latency, and 40% higher throughput.

Simplifying the Kubernetes experience and enhancing observability for AI/ML

We understand that data scientists and ML researchers may not be Kubernetes experts. Google aims to simplify the setup and management of AI-optimized Kubernetes clusters. This includes contributions to Kubernetes usability efforts and SIG-Usability. Managed offerings like GKE provide multiple paths to set up AI-optimized environments, from default configurations to customizable blueprints. Offerings like GKE Autopilot further abstract away infrastructure management, aiming for the ease of use that benefits all users.
Ensuring visibility into AI/ML workloads is paramount. Google actively supports and contributes to the integration of standard open-source observability tools within the Kubernetes ecosystem, such as Prometheus, Grafana, and OpenTelemetry. Building on this open foundation, GKE then provides enhanced, out-of-the-box observability integrated with popular AI frameworks & tools, including specific insights into workload startup latency and end-to-end tracing.

Looking ahead: continued investment in Open Source Kubernetes for AI/ML

The transformation continues. Our roadmap includes exciting developments in upstream Kubernetes for easily deploying and managing large-scale clusters, support for new GPU & TPU generations integrated through open-source mechanisms, and continued community-driven innovations in fast startup, reliability, and ease of use for AI/ML workloads.

Google is committed to making Kubernetes the premier open-source platform for AI/ML, pushing the boundaries of scale, performance, and efficiency while maintaining stability and ease of use. By driving innovation in core Kubernetes and building powerful, deeply integrated capabilities in our managed offering, GKE, we are empowering organizations to accelerate their AI/ML initiatives and unlock the next generation of intelligent applications built on an open foundation.

Come explore the possibilities with Kubernetes and GKE for your AI/ML workloads!

By Francisco Cabrera & Federico Bongiovanni, GCP Google Kubernetes Engine

Announcing LMEval: An Open Source Framework for Cross-Model Evaluation

Authors: Elie Bursztein - Distinguished Research Scientist & David Tao - Software Engineer, Applied Security and Safety Research

Simplifying Cross-Provider Model Benchmarking

At InCyber Forum Europe in April, we open sourced LMEval, a large model evaluation framework, to help others accurately and efficiently compare how models from various providers perform across benchmark datasets. This announcement coincided with a joint talk with Giskard about our collaboration to increase trust in model safety and security. Giskard uses LMeval to run the Phare benchmark that independently evaluates popular models' security and safety.

Example of LMEval running on a multimodal benchmark across two models.

Rapid Changes in the Landscape of Large Models

New Large Language Models (LLMs) are released constantly, often promising improvements and new features. To keep up with this fast-paced lifecycle, developers, researchers, and organizations must quickly and reliably evaluate if those newer models are better suited for their specific applications. So far, rapid model evaluation has proven difficult, as it requires tools that allow scalable, accurate, easy-to-use, cross-provider benchmarking.

Introducing LMEval: Simplifying Cross-Provider Model Benchmarking

To address this challenge, we are excited to introduce LMEval (Large Model Evaluator), an open source framework that Google developed to streamline the evaluation of LLMs across diverse benchmark datasets and model providers. LMEval is designed from the ground up to be accurate, multimodal, and easy-to-use. Its key features include:

• Multi-Provider Compatibility: Evaluating models shouldn't require wrestling with different APIs for each provider. LMEval leverages the LiteLLM framework to offer out-of-the-box compatibility with major model providers including Google, OpenAI, Anthropic, Ollama, and Hugging Face. You can define your benchmark once and run it consistently across various models with minimal code changes.

• Incremental & Efficient Evaluation: Re-running an entire benchmark suite every time a new model or version is released is slow, inefficient and costly. LMEval's intelligent evaluation engine plans and executes evaluations incrementally. It runs only the necessary evaluations for new models, prompts, or questions, saving significant time and compute resources. Its multi-threaded engine further accelerates this process.

• Multimodal & Multi-Metric Support: Modern foundation models go beyond text. LMEval is designed for multimodal evaluation, supporting benchmarks that include text, images and code. Adding new modalities is straightforward. Furthermore, it supports various scoring metrics to support a wide range of benchmark formats from boolean questions, to multi-choices, to free form generation. Additionally, LMEval provides support for safety/punting detection.

• Scalable & Secure Storage: To store benchmark results in a secure and efficient manner, LMEval utilizes a self-encrypting SQLite database. This approach protects benchmark data and results from inadvertent crawling/indexing while they stay easily accessible through LMEval.

Getting Started with LMEval

Creating and running evaluations with LMEval is designed to be intuitive. Here's a simplified example demonstrating how to evaluate two Gemini model versions on a benchmark:

Results from the Phare benchmark that leverages LMEval for evaluation

The LMEval GitHub repository includes example notebooks to help you get started.

Visualizing Results with LMEvalboard

Understanding benchmark results requires more than just summary statistics. To help with this, LMEval includes LMEvalboard, a companion dashboard tool that offers an interactive visualization of how models stack up against each other. LMEvalboard provides valuable insights into model strengths and weaknesses, complementing traditional raw evaluation data.

LMEvalboard UI allows to quickly analyze how models compares on a given benchmark

LMEvalboard allows you to:

• View Overall Performance: Quickly compare all models' accuracy across the entire benchmark.
• Analyze a Single Model: Dive deep into a specific model's performance characteristics across different categories using radar charts and drill down on specific examples of failures
• Perform Head-to-Head Comparisons: Directly compare two models, visualizing their performance differences across categories and examine specific questions where they disagree.

Try LMEval Today!

We invite you to explore LMEval, use it for your own evaluations, and contribute to its development by heading to the LMEval GitHub repository: https://github.com/google/lmeval

Acknowledgements

LMEval would not have been possible without the help of many people, including: Luca Invernizzi, Lenin Simicich, Marianna Tishchenko, Amanda Walker, and many other Googlers.