opensource.google.com

Menu
Showing posts with label PostgreSQL. Show all posts
Showing posts with label PostgreSQL. Show all posts

Google Cloud: PostgreSQL community contribution updates

Friday, July 10, 2026

Group photo of the Google Cloud team smiling and standing at the Google Cloud booth during the PGConf India event.

Google Cloud is deeply committed to the long-term success of the PostgreSQL ecosystem. Our involvement goes beyond providing PostgreSQL managed services; it's also about active participation in the open source communities through technical contributions, leadership in conference committees, and sharing architectural insights that benefit all users. Following is a recap of recent events Google Cloud participated in.

PGConf.dev 2026

Serving as a vital developer-centric hub, PGConf.dev provides a unique opportunity for collaboration with the full assembly of senior PostgreSQL committers. This gathering is essential for aligning technical efforts and shaping the future project roadmap.

Key Highlights

  • Participation focused on strategic coordination with PostgreSQL committers regarding logical replication development, and a consultation on global index architecture.
  • High community interest confirms the Global Index feature solves a vital architectural requirement for enterprises.
  • Established community consensus to pursue a deparsing-based architectural approach for DDL replication.

Google Cloud Sessions

ilip Kumar, a PostgreSQL contributor from Google Cloud, presenting 'Experimenting with a Global Index in PostgreSQL' at pgconf.dev 2026 in Vancouver. He is speaking at a podium next to a presentation slide detailing the Global Index storage architecture and PartitionIdentifier management.
Dilip Kumar, a PostgreSQL contributor from Google Cloud, presenting "Experimenting with a Global Index in PostgreSQL" at pgconf.dev 2026 in Vancouver. He is speaking at a podium next to a presentation slide detailing the Global Index storage architecture and PartitionIdentifier management.
Session Title Session Type Speakers/Led by
Experimenting with a Global Index in PostgreSQL: Design, Implementation, and Challenges Technical Talk Dilip Kumar
Unconference: Global Indexes Unconference Session Dilip Kumar
Unconference: Logical Replication: Warts and Missing Pieces Unconference Session Hannu Krosing

PGConf India 2026

Key Highlights

  • The three-day conference was divided into a training day followed by two days of sessions. More than 580 participants attended the conference over three days.
  • The conference sessions included a mix of keynotes, breakout technical sessions, sponsor sessions, and booth interactions.

Google Cloud Sessions

Session Title Session Type Speakers
Database And GenAI Keynote Paresh Rathod
Experimenting with a Global Index in PostgreSQL Technical Talk Dilip Kumar
GCP - Best home to run PostgreSQL Sponsor Session Trusar Borse, Abhijeet Rajkur
Beyond shared_buffers: On-Demand Memory PostgreSQL Technical Talk Rajeev Rastogi, Vaibhav Popat
Where is my Memory Technical Talk Pushkar Kalidkar
Agentic AI Applications with GCP Databases Keynote Abhijeet Rajkur, Rishi Kapoor, Saurabh Gupta

PGDay Paris & PGDay France 2026

France hosts two distinct flagship PostgreSQL events, and Google Cloud is deeply embedded in both as both organizers and technical contributors. While PGDay Paris serves as an international, English-language hub for the European community, PGDay France is a community-driven, traveling event that focuses on the francophone ecosystem, taking place in Toulouse for 2026.

Key Highlights

  • Matt Cornillon served on the organization committee for PGDay France, while Yves Colin contributed as a member of the program committee.

Google Cloud Sessions

Session Title Session Type Speakers
Creating a "Dungeon Master" with Postgres and MCP Technical Talk Matt Cornillon
Create your first AI agent with PostgreSQL Workshop Matt Cornillon, Yves Colin

PGDay FOSDEM 2026

FOSDEM PGDay is a prominent open source gathering that brings together developers from across the globe to discuss the latest PostgreSQL advancements. It serves as an essential platform for exploring emerging paradigms in database development.

Key Highlights

  • Exploration of how AI-assisted workflows are redefining development beyond standard autocomplete for SQL queries.

Google Cloud Sessions

Session Title Session Type Speakers
Vibe-coding with Postgres: really? Technical Talk Matt Cornillon

PGConf Belgium 2026

PGConf Belgium 2026 took place at the UCLL Campus Proximus in Haasrode, Belgium, serving as an outstanding learning and networking platform for the local PostgreSQL community and students.

Key Highlights

  • The session was selected by faculty as supportive material for a database exam following deep student engagement.

Google Cloud Sessions

Session Title Session Type Speakers
Creating a "Dungeon Master" with Postgres and MCP Technical Talk Matt Cornillon

Nordic PG Day 2026

Nordic PG Day is the largest PostgreSQL event in the Scandinavian countries. The 2026 edition took place in Helsinki, gathering more than 130 PostgreSQL enthusiasts for a day of deep dives.

Key Highlights

  • Google joined as an official Partner-level sponsor for the first time, including a dedicated table booth.

Google Cloud Sessions

Session Title Session Type Speakers
Unlock AI Agents with PostgreSQL Technical Talk Mats Berglin, Miguel Toscano

Swiss PGDay 2026

Swiss PG Day is the annual event organized by the Swiss PostgreSQL User Group in Rapperswil, Switzerland. The ninth edition featured sessions in both English and German.

Key Highlights

  • Demonstration of the physical impact of pushing millions of vectors to PostgreSQL based on a real-world use case.

Google Cloud Sessions

Session Title Session Type Speakers
Surviving pgvector in production: a reality check Technical Talk Miguel Toscano

Postgres Conference: 2026 San Jose

Since its inception in 2007, the Postgres Conference has served as a cornerstone for advancement, fostering a rich environment for learning and professional networking.

Key Highlights

  • Google proudly served as a sponsor for the event.
  • Adapting PostgreSQL for the artificial intelligence era demands a transformation in operational approaches. With the rise of natural language tools and vibe coding speeding up development, Agentic AI places advanced demands on production databases. In their presentation, Vikas and Vishal examine how Google Cloud managed services have evolved to handle these workloads, providing architectural strategies and best practices for contemporary AI deployment.

Google Cloud Sessions

Session Title Session Type Speakers
Postgres and AI - Stronger Together! Technical Talk Vikas Arora and Vishal Bagga

Community Leadership and Committees

Googlers play a vital role in shaping the direction of the most prestigious PostgreSQL developer events. Our leadership in these committees helps ensure that enterprise-grade requirements—such as those needed for large-scale migrations—are part of the global conversation.

  • PGConf.dev 2026: Dilip Kumar served on the Program Committee.
  • PGConf India 2026: Dilip Kumar was a member of the Paper Selection Committee.
  • PGDay France: Matt Cornillon was a member of the organization committee and Yves Colin served as a member of the Program Committee.

Looking Forward

Our commitment remains firm: to turn feedback from these global events into code, reviews, and active community partnerships. We thank the wider PostgreSQL community and the project's committers for their continued collaboration in making PostgreSQL better for everyone.

Acknowledgement

We extend our heartfelt appreciation to our open source community contributors for their outstanding dedication and active participation in making PostgreSQL conferences a great success.

Abhijeet Rajurkar, Darshan Nagarajappa, Dilip Kumar, Hannu Krosing, Mats Berglin, Matt Cornillon, Michael Bautin, Miguel Toscano, Niranjan Shivprasad, Paresh Rathod, Rajeev Rastogi, Vaibhav Popat, Vikas Arora, and Yves Colin

Furthermore, we are deeply grateful to the broader PostgreSQL open-source communities, especially the dedicated conference organizers, committee members, and all supporting sponsors.

Google Cloud: Investing in the future of PostgreSQL

Tuesday, March 31, 2026

At Google Cloud, we are deeply committed to open source, and PostgreSQL is a cornerstone of our managed database offerings, including Cloud SQL & AlloyDB.

Continuing our work with the PostgreSQL community, we've been contributing to the core engine and participating in the patch review process. Below is a summary of that technical activity, highlighting our efforts to enhance the performance, stability, and resilience of the upstream project. By strengthening these core capabilities, we aim to drive innovation that benefits the entire global PostgreSQL ecosystem and its diverse user base.

Our investments in PostgreSQL logical replication aim to unlock critical capabilities for all users. By enhancing conflict detection, we are paving the way for robust active-active replication setups, increasing write scalability and high availability. We are also focused on expanding logical replication to cover missing objects. This is key to enabling major version upgrades with minimal downtime, offering a more flexible alternative to pg_upgrade. Furthermore, our ongoing contributions to bug fixes are dedicated to improving the overall stability and resilience of PostgreSQL for everyone in the community.

Technical contributions: July 2025 – December 2025

The following sections detail technical enhancements and bug fixes contributed to the PostgreSQL open source project between July 2025 and December 2025. Primary engineering efforts were dedicated to advancing logical replication toward active-active capabilities, implementing missing features, optimizing pg_upgrade, and fixing bugs.

Logical Replication Enhancements

Logical replication is a critical feature of PostgreSQL enabling capabilities like near zero down time, major version upgrades, selective replication, active-active replication. We have been working towards closing some of the key gaps.

Automatic Conflict Detection

Active-active replication is a mechanism for increasing PostgreSQL write scalability. One of the most significant hurdles for active-active PostgreSQL setups is handling row-level conflicts when the same data is modified on two different nodes. Historically, these conflicts could stall replication, requiring manual intervention.

In this cycle, the community committed Automatic Conflict Detection which is the first phase of Automatic Conflict Detection and Resolution. This foundation allows the replication worker to automatically detect when an incoming change (Insert, Update, or Delete) conflicts with the local state.

Contributors: Dilip Kumar helped by performing code and design reviews. He is currently advancing the project's second phase, focusing on implementing conflict logging into a dedicated log table.

Logical replication of sequences

Until recently, logical replication in PostgreSQL was primarily limited to table data. Sequences did not synchronize automatically. This meant that during a migration or a major version upgrade, DBAs had to manually sync sequence values to prevent "duplicate key" errors on the new primary node. Since many databases rely on sequences, this was a significant hurdle for logical replication.

Contributors: Dilip Kumar helped by performing code and design reviews.

Drop subscription deadlock

The DROP SUBSCRIPTION command previously held an exclusive lock while connecting to the publisher to delete a replication slot.

If the publisher was a new database on the same server, the connection process would stall while trying to access that same locked catalog.

This conflict created a "self-deadlock," where the command was essentially waiting for itself to finish.

Contributors: Dilip Kumar analyzed and authored the fix.

Upgrade Resilience

Operational ease of use and friction-less upgrades are important to PostgreSQL users. We have been working on improving the upgrade experience.

pg_upgrade optimization for Large Objects

For databases with massive volumes of Large Objects, upgrades could previously span several days. This bottleneck is resolved by exporting the underlying data table directly rather than executing individual Large Object commands, resulting in an upgrade process that is several orders of magnitude faster.

Contributors: Hannu Krosing, Nitin Motiani and, Saurabh Uttam, highlighted the severity of the issue, proposed the initial fix and actively drove it to the resolution.

Prevent logical slot invalidation during upgrade:

Upgrade to PG17 fails if max_slot_wal_keep_size is not set to -1. This fix improves pg_upgrade's resilience, eliminating the need for users to manually set max_slot_wal_keep_size to -1. The server now automatically retains the necessary WAL data for upgrading logical replication slots, simplifying the upgrade process and reducing the risk of errors.

Contributors: Dilip Kumar analyzed and authored the fix.

pg_upgrade NOT NULL constraint related bug fix

A bug in pg_dump previously failed to preserve non-inherited NOT NULL constraints on inherited columns during upgrades from version 17 or older.

The fix updates the underlying query to ensure these specific schema constraints are correctly identified and migrated during the pg_upgrade process.

Contributors: Dilip Kumar analyzed and authored the fix.

Miscellaneous Bug Fixes

We continue to contribute bug fixes to help improve the stability and quality of PostgreSQL.

Make pgstattuple more robust about empty or invalid index pages

pgstattuple is a PostgreSQL extension for analyzing the physical storage of tables and indexes at the row (tuple) level, to determine whether a table is in need of maintenance. However, pgstattuple would raise errors with empty or invalid index pages in hash and gist code. This bug handles the empty and invalid index pages to make pgstattuple more robust.

Contributors: Nitin Motiani and Dilip Kumar, participated as author and reviewer.

Loading extension from different path

A bug incorrectly stripped the prefix from nested module paths when dynamically loading shared library files. This caused libraries in subdirectories to fail to load. The bug fix ensures the prefix is only removed for simple filenames, allowing the dynamic library expander to correctly find nested paths

Contributors: Dilip Kumar, reported and co-authored the fix for this bug.

WAL flush logic hardening

XLogFlush() and XLogNeedsFlush() are internal PostgreSQL functions that ensure log records are written to the WAL to ensure durability. In certain edge cases, like the end-of-recovery checkpoint, the functions relied on inconsistent criteria to decide which code path to follow. This inconsistency posed a risk for upcoming features i.e. Asynchronous I/O for writes that require XLogNeedsFlush() to work reliably.

Contributors: Dilip Kumar, co-authored the fix for this bug.

Major Features in Development

Beyond our recent commits, the team is actively working on several high-impact proposals to further strengthen the PostgreSQL ecosystem.

  • Conflict Log Table for Detection: Dilip Kumar is developing a proposal for a conflict log table designed to offer a queryable, structured record of all logical replication conflicts. This feature would include a configuration option to determine whether conflict details are recorded in the history table, server logs, or both.
  • Adding pg_dump flag for parallel export to pipes: Nitin Motiani is working on this feature. This introduces a flag which allows the user to provide pipe commands while doing parallel export/import from pg_dump/pg_restore (in directory format).

Leadership

Beyond code, our team supports the ecosystem through community leadership. We are pleased to share that Dilip Kumar has been selected for the PGConf.dev 2026 Program Committee to help shape the project's premier developer conference.

Community Roadmap: Your Feedback Matters

We encourage you to utilize the comments area to propose new capabilities or refinements you wish to see in future iterations, and to identify key areas where the PostgreSQL open-source community should focus its investments.

Acknowledgement

We want to thank our open source contributors for their dedication to improving the upstream project.

Dilip Kumar: PostgreSQL significant contributor

Hannu Krosing: PostgreSQL significant contributor

Nitin Motiani: Contributing features and bug fixes

Saurabh Uttam: Contributing bug fixes

We also extend our sincere gratitude to the wider PostgreSQL open source members, especially the committers and reviewers, for their guidance, reviews, and for collaborating with us to make PostgreSQL the most advanced open source database in the world.

Full support of PostgreSQL engine comes to Logica

Tuesday, September 26, 2023

Logica is a logic programming language designed for intuitive and efficient data manipulation, which we open sourced in 2020. It compiles to SQL, providing access to the power of SQL engines with the convenience of a logic programming syntax.

When it was open sourced, Logica's only fully supported engine was BigQuery, a powerful data warehouse, executing queries with high parallelization and processing terabytes of data within seconds.

Modern machines can store and process significant amounts of data, even within a single computer. Thus relational SQL databases are as popular as ever. They contain a lot of data and its analysis is important. Among open source database options, PostgreSQL and SQLite are some of the most popular database engines (example1, example2). Logica added support for SQLite in 2021.

Now we are pleased to announce a new release of Logica that adds support for PostgreSQL.

As Logica compiles to SQL, it is natural to extend the language to use PostgreSQL as the engine. However, there are nuances in the SQL dialect of Postgres which require addressing. The biggest distinction is that PostgreSQL requires types of records to be explicitly spelled out in your query, while BigQuery determines the types automatically.

For example, consider a Logica predicate where for each user we collect a list of records with information about their purchases.

UserPurchases(

    user_id:,

    user_name:,

    purchases? List= {item_name:, item_price:}) distinct :-

  Purchase(purchase_id:, user_id:, item_name:, item_price:),

  UserInfo(user_id:, user_name:);


We can translate this Logica predicate to GoogleSQL to run on BigQuery as follows:

SELECT

  user_id,

  user_name,

  ARRAY_AGG(STRUCT(item_name as item_name, item_price as item_price)) as purchases

FROM

  Purchases INNER JOIN

  UserInfo USING (user_id)

GROUP BY 1, 2;

Logica's record {item_name:, item_price:} simply compiles into GoogleSQL's STRUCT(item_name as item_name, item_price as item_price).

However, in the dialect of PostgreSQL composite types must be explicitly defined and specified. In our example, we need to define the type PurchaseRecord with fields item_name and item_price. We should also specify in the query that the purchases column is aggregating records of type PurchaseRecord. Thus PostgreSQL query for our predicate would be written like so.

CREATE TYPE PurchaseRecord as (item_name text, item_price numeric);


SELECT

  user_id,

  user_name,

  ARRAY_AGG(ROW(item_name,

                item_price)::PurchaseRecord) AS purchases

FROM

  Purchase INNER JOIN

  UserInfo USING (user_id)

GROUP BY UserInfo.user_id, UserInfo.user_name;


Records and lists are also useful as intermediates in calculations, even if the input and output data are normalized. For example, we have a table called ItemSales and want to find a list of most sold items in each of the stores that the table describes. Specifically, we want to assemble a table with information about the top three most sold items among all of the stores. For each of the items, we may want to list the department of the store where the item is being sold. This can be achieved intuitively using the ArgMax3 aggregate function, which accumulates all the information about the items that we need, and no extra join is needed.


# Collecting information of top 3 most sold items for each store.

StoreTopItemsCollection(store) ArgMax3= {item:,

                                         department:} -> sales_volume :-

  ItemSales(store:, item:, department:, sales_volume:);


# Flattening top items collection.

StoreTopItems(store:, item:, department:) :-

  {item:, department:} in StoreTopItemsCollection(store);


To support the PostgreSQL engine, we extended the Logica compiler with type inference. Logica now infers data types for all expressions that a user employs. For records and arrays, Logica specifies their type in the produced SQL, just as PostgreSQL requires. Commands to create necessary types are produced as part of the compiled SQL. In this collab, we show an example of a program that writes a PostgreSQL table, and in this collab, we show how to give type hints when the program does not have enough information for complete inferences.

As a byproduct of type inference, we were able to improve error messages. Now that we know the types, we can point to the user where a mistake is made within the Logica program, rather than the user having to debug the generated SQL statement.

PostgreSQL is a popular and powerful engine. It is easy to start your own instance (maybe just in CoLab!), or use a serverless option. We are excited to provide users of Logica with the option to run on Postgres. If you already use PostgreSQL, we encourage you to give Logica a try, it is a joy to write data analysis with logic programming! If you have any feedback or questions, please share at the discussion section of Logica repository.

By Evgeny Skvortsov, Software Engineer – Google

The API Registry API

Friday, January 8, 2021

We’ve found that many organizations are challenged by the increasing number of APIs that they make and use. APIs become harder to track, which can lead to duplication rather than reuse. Also, as APIs expand to cover an ever-broadening set of topics, they can proliferate different design styles, at times creating frustrating inefficiencies.

To address this, we’ve designed the Registry API, an experimental approach to organizing information about APIs. The Registry API allows teams to upload and share machine-readable descriptions of APIs that are in use and in development. These descriptions include API specifications in standard formats like OpenAPI, the Google API Discovery Service Format, and the Protocol Buffers Language.

An organized collection of API descriptions can be the foundation for a wide range of tools and services that make APIs better and easier to use.
  • Linters verify that APIs follow standard patterns
  • Documentation generators provide documentation in consistent, easy-to-read, accessible formats
  • Code generators produce API clients and server scaffolding
  • Searchable online catalogs make everything easier to find
But perhaps most importantly, bringing everything about APIs together into one place can accelerate the consistency of an API portfolio. With organization-wide visibility, many find they need less explicit governance even as their APIs become more standardized and easy to use.

The Registry API is a gRPC service that is formally described by Protocol Buffers and that closely follows the Google API Design Guidelines at aip.dev. The Registry API description is annotated to support gRPC HTTP/JSON transcoding, which allows it to be automatically published as a JSON REST API using a proxy. Proxies also enable gRPC web, which allows gRPC calls to be directly made from browser-based applications, and the project includes an experimental GraphQL interface.

We’ve released a reference implementation that can be run locally or deployed in a container with Google Cloud Run or other container-based services. It stores data using the Google Cloud Datastore API or a configurable relational interface layer that currently supports PostgreSQL and SQLite.

Following AIP-181, we’ve set the Registry API’s stability level as "alpha," but our aim is to make it a stable base for API lifecycle applications. We’ve open-sourced our implementation to share progress and gather feedback. Please tell us about your experience if you use it.

By Tim Burks, Tech Lead – Apigee API Lifecycle and Governance

HarbourBridge: From PostgreSQL to Cloud Spanner

Wednesday, February 12, 2020

Would you like to try out Cloud Spanner with data from an existing PostgreSQL database? Maybe you’ve wanted to ‘kick the tires’ on Spanner, but have been discouraged by the effort involved?

Today, we’re announcing a tool that makes trying out Cloud Spanner using PostgreSQL data simple and easy.

HarbourBridge is a tool that loads Spanner with the contents of an existing PostgreSQL database. It requires zero configuration—no manifests or data maps to write. Instead, it ingests pg_dump output, automatically builds a Spanner schema, and creates a new Spanner database populated with data from pg_dump.

HarbourBridge is part of the Cloud Spanner Ecosystem, a collection of public, open source repositories contributed to, owned, and maintained by the Cloud Spanner user community. None of these repositories are officially supported by Google as part of Cloud Spanner.

Get up and running fast

HarbourBridge is designed to simplify Spanner evaluation, and in particular to bootstrap the process by getting moderate-size PostgreSQL datasets into Spanner (up to a few GB). Many features of PostgreSQL, especially those that don't map directly to Spanner features, are ignored, e.g. (non-primary) indexes, functions and sequences.

View HarbourBridge as a way to get up and running fast, so you can focus on critical things like tuning performance and getting the most out of Spanner. Expect that you'll need to tweak and enhance what HarbourBridge produces—More on this later.

Quick-start guide

The HarbourBridge README contains a step-by-step quick-start guide. We’ll quickly review the main steps. Before you begin, you'll need a Cloud Spanner instance, Cloud Spanner API enabled for your Google Cloud project, authentication credentials configured to use the Cloud API, and Go installed on your development machine.

To download HarbourBridge and install it, run
go get -u github.com/cloudspannerecosystem/harbourbridge
The tool should now be installed as $GOPATH/bin/harbourbridge. To use HarbourBridge on a PostgreSQL database called mydb, run
pg_dump mydb | $GOPATH/bin/harbourbridge
The tool will use the cloud project specified by the GCLOUD_PROJECT environment variable, automatically determine the Cloud Spanner instance associated with this project, convert the PostgreSQL schema for mydb to a Spanner schema, create a new Cloud Spanner database with this schema, and finally, populate this new database with the data from mydb. HarbourBridge also generates several files when it runs: a schema file, a report file (with details of the conversion), and a bad data file (if any data is dropped). See Files Generated by HarbourBridge.

Take care with ACLs

Note that PostgreSQL table-level and row-level ACLs are dropped during conversion since they are not supported by Spanner (Spanner manages access control at the database level). All data written to Spanner will be visible to anyone who can access the database created by HarbourBridge (which inherits default permissions from your Cloud Spanner instance).

Next steps

The tables created by HarbourBridge provide a starting point for evaluation of Spanner. While they preserve much of the core structure of your PostgreSQL schema and data, many important PostgreSQL features have been dropped.

In particular, HarbourBridge preserves primary keys but drops all other indexes. This means that the out-of-the-box performance you get from the tables created by HarbourBridge can be significantly slower than PostgreSQL performance. If HarbourBridge has dropped indexes that are important to the performance of your SQL queries, consider adding Secondary Indexes to the tables created by HarbourBridge. Use the existing PostgreSQL indexes as a guide. In addition, Spanner's Interleaved Tables can provide a significant performance boost.

Other dropped features include functions, sequences, procedures, triggers, and views. In addition, types have been mapped based on the types supported by Spanner. Types such as integers, floats, char/text, bools, timestamps and (some) array types map fairly directly to Spanner, but many other types do not and instead are mapped to Spanner's STRING(MAX). See Schema Conversion for details of the type conversions and their tradeoffs.

Recap

HarbourBridge automates much of the manual work of trying out Cloud Spanner using PostgreSQL data. The goal is to bootstrap your evaluation and help get you to the meaty issues as quickly as possible. The tables generated by HarbourBridge provide a starting point, but they will likely need to be tweaked and enhanced to support a full evaluation.

We encourage you to try out the tool, send feedback, file issues, fork and modify the codebase, and send PRs for fixes and new functionality. Our plans and aspirations for developing HarbourBridge further are outlined in the HarbourBridge Whitepaper. HarbourBridge is part of the Cloud Spanner Ecosystem, owned and maintained by the Cloud Spanner user community. It is not officially supported by Google as part of Cloud Spanner.

By Nevin Heintze, Cloud Spanner
.