opensource.google.com

Menu

Posts from 2025

This Week in Open Source #10

Friday, September 19, 2025

This Week in Open Source for 09/19/2025

A look around the world of open source

by Daryl Ducharme & amanda casari, Google Open Source

As we enter the Autumn of 2025, AI is still on the top of everyone's mind in tech and the world of open source is no different. This week we delve into various facets of AI's impact on the open source world, from its presence in the Linux kernel and the need for official policy, to the discussion around copyright in AI and how it affects open source licenses. We also explore ways companies can actively support open source, the challenges federated networks like Mastodon face with age verification laws, and the emerging concept of spec-driven development with AI as a design tool.

Upcoming Events

  • September 23 - 27: Nerderarla 2025 is happening in Buenos Aires. It is a 100% free, world-class event in Latin America with high-quality content in science and technology.
  • September 29 - 30: Git Merge 2025 celebrates 20 years of Git in Sunnyvale, California.
  • October 2 - 3: Monktoberfest is happening in Portland, Maine. The only conference focused on how craft, technology and social come together. It's one of the most unique events in the industry.
  • October 12 - 14: All Things Open 2025 is happening in Raleigh, North Carolina. The largest open source / tech / web conference on the U.S. east coast will feature many talks, including some from 4 different Googlers on varying topics — from creating mentorship programs, to security, to kubernetes and how open source already has solutions to your data problems that you may be trying to solve with AI.

Open Source Reads and Links

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Introducing Kotlin FHIR: A new library to bring FHIR to Multiplatform

Tuesday, September 16, 2025

Build once, deploy everywhere: multiplatform FHIR development on Android, iOS, and Web

by Jing Tang, Google Research

The mission of Google's Open Health Stack team is to accelerate digital health innovation by providing developers everywhere with critical building blocks for next-generation healthcare applications. Expanding its existing components, the team has released Kotlin FHIR (currently in alpha), a new open-source library now available on GitHub. It implements the HL7® FHIR® data model on Kotlin Multiplatform (KMP), enabling developers to build FHIR apps and tools for Android, iOS, and Web simultaneously.

Demo app launched as Android, iOS, and Web applications
Demo app launched as Android, iOS, and Web applications

Tools to support health data exchange using a modern standard

HL7® FHIR® (Fast Healthcare Interoperability Resources) is a global interoperability standard for healthcare data exchange. It enables healthcare systems to exchange data freely and securely, improving efficiency and transparency while reducing integration costs. Over the years, it has seen rapidly growing adoption, and its use has been mandated by health regulations in an increasing number of countries.

Since March 2023, the Open Health Stack team at Google has introduced a number of tools to support FHIR development. For example, the Android FHIR SDK helps developers to build offline capable FHIR-native apps that can help community health workers carry out data collection tasks in remote communities. With FHIR Data Pipes, developers can build analytics solutions more easily to generate critical insights for large healthcare programmes more easily. Today, apps powered by these tools are used by health workers covering over 75 million people across Sub-Saharan Africa, South and Southeast Asia.

A leap forward to multiplatform development

In low-resource settings, it is imperative to develop apps that can reach as many patients as possible at a low development cost. However, a lack of infrastructure and tooling often hinders this goal. For example, Kotlin Multiplatform (KMP) is a new and exciting technology rapidly gaining traction, but existing FHIR libraries are not suitable for KMP development due to their platform-specific dependencies. Consequently, developing FHIR apps on KMP has not been possible, causing developers to miss out on a significant opportunity to scale their solutions.

Introducing Kotlin FHIR. It is a modern and lightweight implementation of FHIR data models designed for use on KMP with no platform-specific dependencies. With Kotlin FHIR, developers can build FHIR apps once, and deploy them to Android, iOS, Web, and other platforms.

"Any library that helps implementers use FHIR is my favourite, but I'm particularly thrilled to see a new library from the awesome Open Health Stack team.

– Grahame Grieve, Creator of FHIR, Product Director at HL7

Modern, lightweight, and sustainable

Kotlin FHIR uses KotlinPoet to generate the FHIR data model directly from the specification. This ensures that the library is complete and maintainable. The data model classes it generates are minimalist to provide the best usability for developers: it has everything that you need, nothing less and nothing more. It uses modern language features in Kotlin such as sealed interfaces to ensure type-safety and to give developers the best coding experience. It supports all the FHIR versions: R4, R4B and R5, and will be updated when new FHIR versions are released.

The library is currently in alpha, but has received positive feedback from the developer community. To try FHIR multiplatform development using the library, head to the repository.

Beyond data models: more on multiplatform

Our mission is to empower the digital health community, and the Kotlin FHIR library is our latest step in that effort. But handling the FHIR data model on KMP is just the beginning. Rich features provided by the Android FHIR SDK libraries will also be needed on KMP. This is a collaborative effort, and we invite the FHIR community to join us in defining and building the cross-platform tools you need most. To learn more about how you can get involved, head to the Open Health Stack developer site.

Kubernetes 1.34 is available on GKE!

Wednesday, September 10, 2025

by Benjamin Elder & Pradeep Varadharajan, Google Kubernetes Engine

Kubernetes 1.34 is now available in the Google Kubernetes Engine (GKE) Rapid Channel in just 5 days after OSS release! For more information about the content of Kubernetes 1.34, read the official Kubernetes 1.34 Release Notes and the specific GKE 1.34 Release Notes.

Kubernetes Enhancements in 1.34:

The Kubernetes 1.34 release, themed 'Of Wind & Will', symbolizing the winds that shaped the platform, delivers a fresh gust of enhancements. These updates, shaped by both ambitious goals and the steady effort of contributors, continue to propel Kubernetes forward.

Kubernetes 1.34 Release Logo, “Of Wind & Will (O' WaW)” which features bears on a wooden sail powered boat, with the Kubernetes logo stylized into the image frame and the ship’s sails

Below are some of the Kubernetes 1.34 features that you can use today in production GKE clusters.

DRA Goes GA

The Kubernetes Dynamic Resource Allocation (DRA) APIs are now GA. This is a huge step in the evolution of Kubernetes to stay the undisputed platform for AI/ML workloads. DRA improves Kubernetes' ability to select, configure, allocate, and share GPUs, TPUs, NICs and other specialized hardware. For more information about using DRA in GKE, see About dynamic resource allocation in GKE. You can use DRA now with self-installed drivers and can expect more improvements in upcoming releases.

The Prioritized list and Admin access features have been promoted to beta and will be enabled by default, and the kubelet API has been updated to report status on resources allocated through DRA.

KYAML

We've all been there: a stray space or an unquoted string in a YAML file leads to frustrating debugging sessions. The infamous "Norway Bug" is a classic example of how YAML's flexibility can sometimes be a double-edged sword. 1.34 introduces support for KYAML, a safer and less ambiguous subset of YAML, specifically designed for Kubernetes and helps avoid these common pitfalls.

KYAML is fully compatible with existing YAML parsers but enforces stricter rules making your configurations more predictable and less prone to whitespace errors. This is a game-changer for anyone using templating tools like Helm, where managing indentation can be a headache.

To start using KYAML, simply update your client to 1.34+ and set the environment variable KUBECTL_KYAML=true to enable use of -o kyaml. For more details, check out KEP-5925.

Pod-level resource requests and limits

With the promotion of Pod-level resource requests and limits to beta (and on-by-default), you can now define resource requests and limits at the pod level instead of the container level. This simplifies resource allocation, especially for multi-container Pods, by allowing you to set a total resource budget that all containers within the Pod share. When both pod-level and container-level resources are defined, the pod-level settings take precedence, giving you a clear and straightforward way to manage your Pod's resource consumption.

Improved Traffic Distribution for Services

The existing PreferClose setting for traffic distribution in Services has been a source of ambiguity. To provide clearer and more precise control over how traffic is routed, KEP-3015 deprecates PreferClose and introduces two new, more explicit values:

  • PreferSameZone is equivalent to the existing PreferClose.
  • PreferSameNode prioritizes sending traffic to endpoints on the same node as the client. This is particularly useful for scenarios like node-local DNS caches, where you want to minimize latency by keeping traffic on the same node whenever possible.

This feature is now beta in 1.34, with its feature gate enabled by default.

Ordered Namespace Deletion for Enhanced Security

When a namespace is deleted, the order in which its resources are terminated has, until now, been unpredictable. This can lead to security flaws, , such as a NetworkPolicy being removed before the Pods it was protecting, leaving them temporarily exposed. With this enhancement, Kubernetes introduces a structured deletion process for namespaces, ensuring secure and predictable resource removal by enforcing a deletion order that respects dependencies, removing Pods before other resources.
This feature was introduced in Kubernetes v1.33 and became stable in v1.34.

Graceful Shutdowns Made Easy

Ensuring a graceful shutdown for your applications is crucial for zero-downtime deployments. Kubernetes v1.29 introduced a "Sleep" for containers' PreStop and PostStart lifecycle hooks, offering a simple approach to managing graceful shutdowns. This feature allows a container to wait for the specified duration before it's terminated, giving it time to finish in-flight requests and ensuring a clean handoff during rolling updates.
Note: Specifying a negative or zero sleep duration will result in an immediate return, effectively acting as a no-op (added in v1.32).
This feature graduated to stable in v1.34.

Streaming List Responses

Large Kubernetes clusters can push the API server to its limits when dealing with large LIST responses that can consume gigabytes of memory. Streaming list responses address this by changing how the API server handles these requests.

Instead of buffering the entire list in memory, it streams the response object by object, improving performance and substantially reducing memory pressure on the API server. This feature is now GA and is automatically enabled for JSON and Protobuf responses with no client-side changes.

Resilient Watch Cache Initialization

The watch caching layer in the Kubernetes apiserver maintains an eventually consistent cache of cluster state. However, if it needs to be re-initialized, it can potentially lead to a thundering herd of requests that can overload the entire control plane. The Resilient Watch Cache Initialization feature, now stable, ensures clients and controllers can reliably establish watches.

Previously, when the watch cache was initializing, incoming watch and list requests would hang, consuming resources and potentially starving the API server. With this enhancement, such requests are now intelligently handled: watches and most list requests are rejected with a 429, signaling clients to back off, while simpler get requests are delegated directly to etcd.

In-Place Pod Resize Gets Even Better

In-place pod resize, which allows you to change a Pod's resource allocation without a disruptive restart, remains in Beta, but continues to improve in v1.34. You can now decrease memory limits with a best-effort protection against triggering the OOM killer. Additionally, resizes are now prioritized, and retrying deferred resizes is more responsive to resources being released. A ResizeCompleted event provides a clear signal when a resize completes, and includes a summary of the new resource requirements.

MutatingAdmissionPolicy Gets to Beta

MutatingAdmissionPolicy, working as a declarative, in-process alternative to mutating admission webhooks, goes to Beta in Kubernetes 1.34.

Mutating admission policies use the Common Expression Language (CEL) to declare mutations to resources. Mutations can be defined either with an apply configuration that is merged using the server side apply merge strategy, or a JSON patch. This feature is highly configurable, enabling policy authors to define policies that can be parameterized and scoped to resources as needed by cluster administrators.

Acknowledgements

As always, we want to thank all the Googlers that provide their time, passion, talent and leadership to keep making Kubernetes the best container orchestration platform. We would like to mention especially Googlers who helped drive some of the open source features mentioned in this blog: Tim Allclair, Natasha Sarkar, Jordan Liggitt, Marek Siarkowicz, Wojciech Tyczyński, Tim Hockin, Benjamin Elder, Antonio Ojea, Gaurav Ghildiyal, Rob Scott, John Belamaric, Morten Torkildsen, Yu Liao, Cici Huang, Joe Betz, and Dixita (Dixi) Narang.


And thank the many Googlers who helped bring 1.34 to GKE!

This Week in Open Source #9

Friday, September 5, 2025

This Week in Open Source for 09/05/2025

A look around the world of open source
by Daryl Ducharme, amanda casari & Shane Glass, Google Open Source

Upcoming Events

  • September 5-7: NixCon 2025 is happening in Switzerland. It is the annual conference for the Nix and NixOS community where Nix enthusiasts learn, share, and connect with others.
  • September 9: Kubernetes Community Day 2025 SF Bay Area event, the ultimate gathering for cloud native enthusiasts! This full-day event, sponsored by the Cloud Native Computing Foundation (CNCF), is packed with insightful cloud native talks and unparalleled opportunities for community networking.
  • September 11 - 14: ASF Community over Code is happening in Minneapolis, Minnesota. It is for ASF members, committers, and open source developers from around the world, focusing on Search, Big Data, Internet of Things, Community, Geospatial, Financial Tech, and many other topics. Google Open Source's own Stephanie Taylor will be giving a talk on cultivating contributors through mentorship.
  • September 12 - 16: PyCon AU 2025 is happening in Narrm/Melbourne. It is the national conference for the Python programming community, bringing together professional, student and enthusiast developers, sysadmins and operations folk, students, educators, scientists, statisticians, and many others besides, all with a love for working with Python.
  • September 23 - 27: Nerderarla 2025 is happening in Buenos Aires. It is a 100% free, world-class event in Latin America with high-quality content in science and technology.
  • September 29 - 30: Git Merge 2025 celebrates 20 years of Git in Sunnyvale, California.
  • October 2 - 3: Monktoberfest is happening in Portland, Maine. The only conference focused on how craft, technology and social come together. It's one of the most unique events in the industry.

Open Source Reads and Links

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

This Week in Open Source #8

Friday, August 15, 2025

This Week in Open Source for 08/15/2025

A look around the world of open source
by Daryl Ducharme & amanda casari, Google Open Source

Upcoming Events

  • August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
  • August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.
  • September 5-7: NixCon 2025 is happening in Switzerland. It is the annual conference for the Nix and NixOS community where Nix enthusiasts learn, share, and connect with others.
  • September 9: Kubernetes Community Day 2025 SF Bay Area event, the ultimate gathering for cloud native enthusiasts! This full-day event, sponsored by the Cloud Native Computing Foundation (CNCF), is packed with insightful cloud native talks and unparalleled opportunities for community networking.
  • September 12 - 16: PyCon AU 2025 is happening in Narrm/Melbourne. It is the national conference for the Python programming community, bringing together professional, student and enthusiast developers, sysadmins and operations folk, students, educators, scientists, statisticians, and many others besides, all with a love for working with Python.

Open Source Reads and Links

  • [Article] Google Brings the A2A Protocol to More of Its Cloud - Last month, Google transferred the A2A protocol to the Linux Foundation and we are still continuing to improve it. Be it updating the spec, integrating it into Cloud Run and GKE we are still happy to see excitement about the future of this protocol.
  • [Book] OSPO Book - Open Source Programs Offices are an important part of connecting open source communities to your company (if we do say so ourselves). If you are an open source enthusiast who thinks they can start one in their company, here is a good guide from CNCF. There's also a github repo for it.
  • [Analysis] The RedMonk Programming Language Rankings: January 2025 - Redmonk's regular analysis of programming languages. Trends are remaining mostly steady across languages, which is an interesting trend of itself!
  • [Blog] One Event at a Time: Funding Your Community the Realistic Way - Great writeup, from a PSF Board member, advising event organizers in the Python community on developing responsible and sustainable funding plans for their community events.
  • Python Software Foundation News: The PSF has paused our Grants Program - The PSF is temporarily pausing their Grants Program after reaching their 2025 grant budget cap earlier than expected. While they know how important this program is to many in the community, this is a necessary step to protect both the future of the program and the short- and long-term sustainability of the PSF. (If this moves you immediately to donate to the PSF, we welcome your contributions via our donations page).

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Google Summer of Code 2025: Contributor Statistics

Thursday, August 14, 2025

The Numbers Are In: A Deep Dive into GSoC 2025 Stats

Google Summer of Code (GSoC) is an online global program that introduces students and beginner developers to open source software development. For our 21st year of the program we welcomed 1280 Contributors from 68 countries who are coding for 185 Mentoring Organizations.

With the coding period starting June 2nd, GSoC contributors are focused on their 2025 projects alongside their Mentors and the thriving open source communities they are working with. We are excited to share some statistics about the accepted contributors in this year's program.

Accepted GSoC Contributors

  • 92.32% are participating in their first GSoC
  • 43.04% had not contributed to open source before GSoC 2025
  • 89.02% are enrolled in an academic program
An infographic titled Google Summer of Code 2025: The numbers are in!. The image provides the following statistics:

Proposals: 23,000+ proposals were received from 15,000+ individual applicants, representing 130 countries.

Applicants: Over 96% of applicants were applying to GSoC for the first time.

Contributors: 89% of GSoC 2025 contributors are enrolled in an academic program.

Mentorship: The program has 2,100+ mentors from 75 countries and involves 185 open-source organizations.

Project Size: A bar chart shows the project size distribution:

Large (~350 hours): 54%

Medium (~175 hours): 42%

Small (~90 hours): 4%

Projects

  • 53.68% of projects were large (~350 hours), 41.54% medium (~175 hours), 4% (~90 hour) projects
  • Currently, 77.9% of projects are the standard 12 weeks in length, with 18.3% extending their projects between 14-22 weeks.
Proposals

We got a whopping 15,240 applicants submitting proposals (an increase of 130% of our previous high - a new record!) from 130 countries. These folks submitted 23,559 proposals, a 159% increase over last year!

96.55% applied to GSoC for the first time in 2025

Registrations

We had a record 98,698 people registering from 172 countries for the 2025 program, an increase of 124.4% over the previous high.

Mentors

This summer, 185 open-source organizations are participating in GSoC. Their projects are supported by over 2,100 mentors from 75 countries. These dedicated volunteers guide new contributors, helping them hone their skills.

Many of these mentors are highly experienced. Almost two-thirds have mentored GSoC contributors for four or more years.

A big thank you for being part of this wonderful community and for helping to spread the word about GSoC, which offers an invaluable opportunity for all the individuals beginning their journey in Open Source. We'll keep you updated with future entries about GSoC 2025, stay tuned!

by Stephanie Taylor, Mary Radomile & Lucila Ortiz, Google Open Source Team

This Week in Open Source #7

Friday, August 8, 2025

This Week in Open Source for 08/08/2025

A look around the world of open source
by Daryl Ducharme, Google Open Source

Upcoming Events

  • August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
  • August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.
  • September 5-7: NixCon 2025 is happening in Switzerland. It is the annual conference for the Nix and NixOS community where Nix enthusiasts learn, share, and connect with others.

Open Source Reads and Links

  • The Asymmetry of Open Source - Open source software projects need funding, but users are not obligated to pay for them. Companies should invest in open source to maintain quality and avoid issues, while hobbyists can contribute without financial pressure. Proper boundaries and mutual responsibility between companies and developers are essential for a healthy open source ecosystem. How do we find and set those boundaries?
  • Linux Foundation Announces Intent to Form Developer Relations Foundation - The Linux Foundation has created the Developer Relations Foundation which aims to unify best practices and enhance the role of developer relations in technology. The DRF will focus on collaboration and shared knowledge. Having an open source organization behind this, helps to make sure DevRel is always of service to developers along with whoever is employing them.
  • 5 tips to get started on accessibility - Not exactly open source and yet super important. So important to the open source community that All Things Open posted it on their site. Accessibility (A11y) is always useful. The more it gets used properly, the more useful it is for everyone.
  • Bringing open source development to Trust and Safety - Ever open source champion, former Googler and now COO at Roost, Anne Bertucio discusses how some teams still have a difficult time understanding open source. The standards that they are used to don't always occur within the transparent world of open source. This means, bringing open source to those teams requires understanding where they are coming from and discussing its limitations as well as its benefits.
  • How we made JSON.stringify more than twice as fast - One of the beautiful things about open source is the transparency in projects. Google's Chromium V8 engine is no exception. This walk through of the technical structuring that led to a faster JSON.stringify is a great way to learn some approaches to solving software bottlenecks that you may not have thought of. With it being open source, you can also visit the repository and follow along with the history of these code changes.

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

What's new in Apache Iceberg v3?

Thursday, August 7, 2025

A Deeper Dive into Apache Iceberg V3: How New Designs Are Solving Core Data Lake Challenges

The Next Chapter for Apache Iceberg: Welcoming the Iceberg V3 Spec
by Talat Uyarer, BigQuery Managed Iceberg & Shane Glass, Google Open Source Programs Office

An infographic illustrating the new features in Apache Iceberg V3. In the center is a logo of an iceberg with V3 written on it. Arrows point from the central logo to four surrounding illustrations, each representing a new feature: Top left: Deletion Vectors, depicted as a tall stack of data blocks. Top right: Variant Data Type, shown as a collection of colorful circles and cubes. Bottom right: Geospatial Data Types, illustrated by a folded world map with location pins. Bottom left: Row Lineage, represented by a grid of various colorful icons.

The data community has long grappled with the challenge of how to bring database-like agility to petabyte-scale datasets stored in open cloud storage. The trade-off has often been between the scalability of data lakes and the performance and ease-of-use of traditional data warehouses. Executing fine-grained updates or evolving table schemas on massive tables often required slow, expensive, and disruptive operations.

The Apache Iceberg project is taking on this challenge. Early versions introduced a revolutionary metadata layer that brought reliability and ACID transactions to data lakes. However, certain operations still presented performance bottlenecks at scale.

With the ratification of the V3 specification, the Apache Iceberg community has introduced new designs that directly address these core issues. These advancements represent a significant leap forward in the mission to build an open and high-performance data lakehouse architecture. Let's explore the technical details of these solutions.

More Efficient Row-Level Transactions with Deletion Vectors

A primary challenge for data lakes has been handling row-level deletes efficiently. Previous approaches, like positional delete files, were a clever solution but could lead to performance degradation at query time when a reader had to reconcile many small delete files against large data files.

The Iceberg V3 spec introduces binary deletion vectors, a more performant and scalable architecture. The core idea is to attach a bitmap to each data file, where each bit corresponds to a row, marking it as deleted or not.

When a query engine reads a data file, it also reads its corresponding deletion vector. As it scans rows, it can check the bitmap with minimal overhead and skip rows marked for deletion. This design is made exceptionally efficient through the use of Roaring bitmaps. This data structure is ideal for this task because it can compress sparse sets of integers—like the positions of deleted rows—into a tiny footprint.

The practical difference is profound:

  • Previous Model (Positional Deletes): A query might involve reading a central log of deletes, like deletes.avro, containing tuples of (file_path, row_position).
  • V3 Model (Deletion Vectors): Each data file (e.g., file_A.parquet) is paired with a small, efficient sidecar file (e.g., file_A.puffin) containing a Roaring bitmap of its deleted rows.

This change localizes delete information, streamlines the read path, and dramatically improves the performance of workloads that rely on frequent Change Data Capture (CDC) or row-level updates.

Simplified Schema Evolution with Default Column Values

Another common operational hurdle in managing large tables has been schema evolution. Adding a column to a table with billions of rows traditionally required a "backfill"—a costly and time-consuming job to rewrite all existing data files to add the new column.

Iceberg V3 eliminates this friction with default column values. This feature allows a default value to be specified directly in the table's metadata when a column is added.

ALTER TABLE events ADD COLUMN version INT DEFAULT 1;

This operation is instantaneous because it only modifies metadata. No data files are touched. When a query engine encounters an older data file without the version column, it consults the table schema, finds the default value, and seamlessly populates it in the query results on the fly. This simple but powerful mechanism makes schema evolution a fast, non-disruptive operation, allowing data models to evolve quickly.

Improved Query Engine Compatibility with Enhanced Data Types and Lineage

Beyond these headline features, V3 broadens the capabilities of Iceberg to support more advanced use cases:

  • Row-Level Lineage: For robust auditing and reliable CDC pipelines, V3 formalizes the tracking of row history. By embedding metadata about when a row was added or last modified, Iceberg tables can now provide a clear lineage, simplifying data governance and enabling more efficient downstream data replication.
  • Rich Data Types: V3 closes the gap with traditional databases by introducing a more expressive type system. This includes a VARIANT type for handling semi-structured data like JSON, native GEOMETRY and GEOGRAPHY types for advanced geospatial analysis, support for nanosecond-precision timestamps with the new timestamp_ns and timestamptz_ns data types, a significant increase from the previous microsecond limit.

Building the Future of the Open Data Lakehouse

These V3 features—deletion vectors, default values, row lineage, and richer types—are more than just individual improvements. Together, they represent a cohesive step toward a new paradigm where the lines between the data lake and the data warehouse are erased. They enable faster, more efficient, and more flexible data operations than previously thought possible.

This progress is a testament to the collaborative spirit of the Apache Iceberg community. At Google, we are proud to contribute to and support open-source projects like Iceberg that are defining the future of data architecture. We are excited to see the innovative applications the community will build on this powerful new foundation.

Want to get started with Iceberg? Check out this blog post to learn more about how Google Cloud's managed Iceberg offering, BigLake tables for Apache Iceberg in BigQuery, makes building Iceberg-native lakehouses easier by maximizing performance without sacrificing governance.


This Week in Open Source #6

Friday, August 1, 2025

This Week in Open Source for 08/01/2025

A look around the world of open source

by Daryl Ducharme & amanda casari, Google Open Source Programs Office

Diving into the open source world this week, we'll cover upcoming events that foster collaboration and innovation, alongside new reads and links that highlight significant advancements and discussions within the open source community. From new Google projects enhancing package ecosystem confidence to thought-provoking articles on open source funding, we hope this keeps you aware of new areas of the ecosystem.

Upcoming Events

  • August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
  • August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.
  • September 5-7: NixCon 2025 is happening in Switzerland. It is the annual conference for the Nix and NixOS community where Nix enthusiasts learn, share, and connect with others.

Open Source Reads and Links

  • [Blog] Google introduced OSS Rebuild, a new project designed to enhance confidence in open source package ecosystems through the reproduction of upstream artifacts.
  • [Story] SF-Based Internet Archive Is Now a Federal Depository Library. What Does That Mean? - The Internet Archive is a foundational reference and repository for open-access information and digital archives.The San Francisco-based digital library now has federal depository status, joining a network of over 1,100 libraries that archive government documents and make them accessible to the public — even as ongoing legal challenges pose an existential threat to the organization.
  • [Video] Keynote: Building community through collaborative datasets - Mago Torres' keynote from csv,conf 8, on her work building collaborative datasets for award-winning data journalism, captures the spirit and focus on where open technology enables communities to accomplish more together.
  • [Paper] Anubis Pilot Project Report - June 2025 - In May and June 2025, Duke University Libraries (DUL) successfully implemented Anubis, a configurable open source web application firewall (WAF), to combat persistent AI-related bot scraping. During this pilot (May 1 - June 10, 2025), aggressive bot scraping caused outages for three critical library platforms (Duke Digital Repository, Archives & Manuscripts, and the Books & Media Catalog); Anubis mitigated the problem in each instance.
  • [Article] Microsoft-owned GitHub says open source needs to be funded - The Register published this editorial which asks whether open source software has reached the point that it should be managed as infrastructure and funded by governments that rely on it? Some studies show impressive numbers in how much it contributes to many economies.
  • [Blog] Open Source Explained Like You're Five (But Smarter) - Explaining open source to people outside the tech world is tough. This article uses some good metaphors along with some details you may not have known to better explain it and spread the word. Or, you could just send them this article and hope they read it. 😜

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

This Week in Open Source #5

Friday, July 25, 2025

This Week in Open Source for July 25, 2025

A look around the world of open source

by Daryl Ducharme & amanda casari, Google Open Source Programs Office

We hope everyone is having a good summer. The world of open source is, with more events and news that caught our attention.

Upcoming Events

  • July 31-August 3: FOSSY (Free and Open Source Software Yearly) will be held in Portland, Oregon and is focused on the creation and impact of free and open source software, uplifting contributors of all experience.
  • August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.
  • August 25-27: Open Source Summit Europe (OSSEU) is happening in Amsterdam, Netherlands. It is the premier event for the open source community to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. Many Googlers will be there giving talks along with so many others.

Open Source Reads and Links

  • [Press Release] Tech Veterans Anne Bertucio and Vinay Rao Join ROOST - A bit of a bittersweet post as our recent, now former Head of Open Source Programs Office, Anne Bertucio, joins ROOST as COO and the previous Head of Safeguards at Anthropic, Vinay Rao, joins as CTO.
  • [Article] An open-source SDK for finding dead code - Maintaining dead code is a waste of resources. So, having good tools for finding dead code in your applications is important. The open sourcing of Reaper for iOS and Android applications might be a worthwhile part of your toolbelt.
  • [Blog] Why I used to prefer permissive licenses and now favor copyleft - Choosing the right license for your open source projects is a very personal choice. A choice that is worth revisiting once in a while to see if your values have shifted and if there are new ideas for what might constitute free software that better align with those new values.
  • [Blog] Announcing FOKS: The Federated Open Key Service - Security and authentication are key to the tech world and open source is a good way to get many eyes on the problems to find solutions. A new federated open key service, FOKS, built from the ground up and based on concepts while working with Keybase is available now.
  • [Article] Kubernetes Surges in Enterprise, But What Can Take It Mainstream? - Different teams in the development work streams have their own ideas about the tech stack. Many teams using Kubernetes have made it quite popular for use in enterprise work, but some are still using systems that have been tried and tested in their own domains. What work needs to be done to get all teams on-board with using Kubernetes?
  • [Blog] Death by a thousand slops - The lead maintainer for the open source project, curl, continues to blog on where low-quality recommendations to curl's Bug Bounty program are increasing the work for the security team.
  • [Article] From A2A to MCP, a look at the protocols that might one day help AI automate you out of a job - Click-bait headline aside, a good overview of where these protocols are at, what they do, and a certain view on whether that's useful or not. We have our opinions, but we are probably biased ;)
  • [Article] How the Free Software Foundation battles the LLM bots - There are many bots out there crawling the web. In the early days of search, the solution was the robots.txt files and bots crawling the web slow enough for the systems to continue to run smoothly. However, many LLM bots are ignoring robots.txt, being greedy with site resources, and that's on top of other bot traffic to deal with. Looking at how a large organization approaches this current trend has some great shared knowledge.

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Stop Leaked Credentials in Their Tracks with Veles, Our New Open-Source Secret Scanner

Tuesday, July 22, 2025

Stop Leaked Credentials in Their Tracks with Veles, Our New Open-Source Secret Scanner

by Kevin Dungs, Charl de Nysschen & Sarah Lucas, Google

In today's complex software supply chain, a single leaked credential—an API key, a service account token, a password—can be all an attacker needs to breach your systems. These secrets can be accidentally committed to a source code repository, embedded in a container image, or attached to a support ticket, creating a critical and often invisible risk.

To help developers and security teams proactively find and fix these exposures, we are excited to announce Veles, a new open-source secret and credential scanner from Google.

Veles is designed to detect unintended exposure of sensitive credentials across your organization's internal systems. It helps you find secrets where they don't belong, so you can prevent them from being abused.

Why Veles? Key Features

Veles is a new, standalone module within our OSV-SCALIBR (Software Composition Analysis LIBRary) ecosystem, but it is built to be used independently. This means you can easily integrate it into your existing security tooling or use it as a standalone scanner.

In its initial release, Veles helps you find high-risk secrets in source code and user-provided artifacts. Our detection library currently identifies:

  • Google Cloud Platform (GCP) API Keys
  • GCP Service Account Keys
  • RubyGems API Keys

This is just the beginning. Veles is built to be extensible, allowing for the rapid addition of new secret types.

Battle-Tested at Google: Powerful Real-World Integration

At Google, we're not just releasing Veles; we're actively using it to protect our own systems and the open-source ecosystem.

  • Internal Protection: Veles is already scanning Google's internal source code repositories and artifacts, helping us find and remediate leaked secrets before they become a problem.
  • Securing the Open Source Ecosystem: The Google Open Source Security Team is incorporating Veles into its pipeline that powers deps.dev, scanning hundreds of millions of open-source artifacts (packages, Docker images, and repositories) to detect and remediate leaked credentials across the community.
  • Integration with Google Cloud Products: Veles is being integrated directly into Google Cloud security services to bring secret scanning to our customers:
    • Artifact Analysis & Artifact Registry: Veles will power secret scanning in Artifact Registry, with findings surfaced through the Container Analysis API and, eventually, in the Artifact Registry UI.
    • Security Command Center (SCC): SCC's integration will provide comprehensive secret detection across the entire cloud lifecycle. This means scanning "left" into the development pipeline (like Infrastructure as Code) and "right" into active runtime environments (like Compute Engine and GKE). SCC will then unify these findings, helping you prioritize the most critical exposures and visualize potential attack paths.

The Road Ahead: What's Next for Veles?

This first release is a foundational step. Our roadmap for Veles includes:

  • Broader Detection: We will continuously expand the library of supported secret and credential types.
  • Automated Validation: We plan to add functionality to intelligently validate if a discovered secret is active.
  • Remediation Workflows: In the future, we aim to help automate the revocation of confirmed, leaked secrets.

Get Started with Veles Today

Veles is open-source and ready for you to use. You can integrate it into your CI/CD pipeline, run it against your existing repositories, or contribute to its development. Protecting your organization from leaked credentials is a critical part of a strong security posture, and Veles is here to help.

Ready to start scanning? Head over to the Veles GitHub repository to get started!

This Week in Open Source #4

Friday, July 18, 2025

This Week in Open Source for July 18, 2025

A look around the world of open source
by Daryl Ducharme & amanda casari, Google Open Source Programs Office

Getting into the middle of July, we've been reading lots of various articles. Here's the upcoming events and some of our favorites.

Upcoming Events

  • July 24-29: GUADEC 2025, the Gnome community's largest conference is in Brescia, Italy.
  • July 31-August 3: FOSSY (Free and Open Source Software Yearly) will be held in Portland, Oregon and is focused on the creation and impact of free and open source software, uplifting contributors of all experience.
  • August 14-16: Open Source Festival 2025 (OSCAFest'25) is happening in Lagos, Nigeria. It uses community to help integrate the act of open source contribution to African developers whilst strongly advocating the movement of free and open source software.

Open Source Reads and Links

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

Unlocking High-Performance AI/ML in Kubernetes with DRANet and RDMA

Tuesday, July 15, 2025

DraNet Enters Beta! High-Performance Networking in Kubernetes

by Antonio Ojea & Federico Bongiovanni, Kubernetes/GKE

We are excited to announce that DraNet has officially entered a beta state! This marks a major leap forward in our mission to streamline and enhance high-performance networking for AI and HPC workloads within Kubernetes. As we progress towards a stable General Availability (GA) release, we are eager to gather your feedback on the current state of the project.

Why DraNet?

DraNet was born from the lessons we learned at Google, observing the challenges end-users faced when running AI and HPC workloads on Kubernetes. The existing networking solutions, often repurposed from traditional networking or bespoke and complex, fell short of providing a good user experience and efficient operational models.
For instance, managing RDMA (Remote Direct Memory Access) interfaces often involved a complex combination of CNI chaining and device plugins. This not only created an unnecessary operational overhead for administrators but also led to coordination issues between different components that needed to work in harmony impacting resilience and scalability.
Another significant pain point we identified was the need for fine-grained interface tuning. AI workloads, for example, are extremely sensitive to latency. The presence of some eBPF programs on network interfaces, or the need to configure specific NIC parameters, could severely impact performance latency and/or throughput. Users were often forced to create custom init containers just to apply these settings, adding another layer of complexity.

Introducing DraNet: A Native and Declarative Solution

DraNet is a native integration with Kubernetes that uses the core Dynamic Resource Allocation (DRA) API to address these challenges by treating high-performance network interfaces as first-class citizens in Kubernetes. Here's how:

  • Simplified RDMA Management: DraNet manages RDMA interfaces natively, handling the different requirements to offer a unified and seamless user experience. No more need for coordinating different components.
  • Declarative Interface Tuning: With DraNet, you can declaratively set interface properties. Need to disable eBPF programs to reduce packet processing overhead or set specific NIC parameters? You can now do this directly in your Kubernetes manifests, eliminating the need for custom scripts or init containers.
  • Standalone and Secure: DraNet is designed as a standalone binary, allowing it to run in a distroless container. This significantly reduces the attack surface and the frequency of security-related updates for the container image. By interacting directly with the kernel via stable APIs like netlink, it avoids dependencies on third-party projects, improving both resilience and performance.
  • Lightweight and Fast: The DraNet container image, with a compressed size of less than 50MB, has a direct impact on node startup times, allowing for faster deployment and scaling of your workloads.

Beta Release and the Road to GA

DraNet is now in a beta state, signifying that it is ready for broader community testing and feedback. This move to beta is aligned with the maturation of the Kubernetes Dynamic Resource Allocation (DRA) KEP (KEP-4381), a foundational technology for DraNet. We are continuing our active development as we work towards a future General Availability release.

We Welcome Your Feedback and Contributions!

DraNet is an open-source project, and we believe that community involvement is key to its success. As we work towards our GA release, we welcome your feedback, whether it's on the design, user experience, or performance.
You can contribute in many ways:

  • Code contributions: We have a fast-paced development cycle and welcome new contributors. Check out our contributing guidelines to get started.
  • Documentation: Help us improve our documentation to make it easier for new users to get started with DraNet.
  • Share your opinion: Your feedback is invaluable. Let us know how you are using DraNet and what we can do to make it better.

To learn more about DraNet and get started, please visit https://dranet.dev/. We look forward to building the future of high-performance networking in Kubernetes with you!

This Week in Open Source #3

Friday, July 11, 2025

This Week in Open Source for July 11, 2025

A look around the world of open source
by Daryl Ducharme, Erin McKean & amanda casari, Google Open Source Programs Office

We took a break as there was a holiday in the US that shortened our work week, but we are back to share what our open source world has to offer.

Upcoming Events

  • July 14-19: The 26th annual Debian Conference (DebConf) for Debian contributors and users interested in improving Debian is in Brest, France.
  • July 24-29: GUADEC 2025, the Gnome community's largest conference is in Brescia, Italy.
  • July 31-August 3: FOSSY (Free and Open Source Software Yearly) will be held in Portland, Oregon and is focused on the creation and impact of free and open source software, uplifting contributors of all experience.

Open Source Reads and Links

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

This Week in Open Source #2

Friday, June 27, 2025

This Week in Open Source for June 27, 2025

A look around the world of open source
By Daryl Ducharme & amanda casari - Google Open Source Programs Office

With Open Source Summit North America (OSSNA) this week, it has been an exciting week.

OSSNA Keynote Announcements and more you may have missed

Upcoming Events

  • July 7-13: The 24th annual SciPy conference will be held in Tacoma, Washington. It brings together attendees from industry, academia, and government to showcase their latest Python projects, learn from skilled users and developers, and collaborate on code development.
  • July 8-9: The Beam Summit is happening in New York City. It is the leading conference for Apache Beam, the unified programming model for batch and stream data processing.
  • July 14-19: The 26th annual Debian Conference (DebConf) for Debian contributors and users interested in improving Debian is in Brest, France.
  • July 24-29: GUADEC 2025, the Gnome community's largest conference is in Brescia, Italy.

Open Source Reads and Links

What exciting open source events and news are you hearing about? Let us know on our @GoogleOSS X account.

.