How much should you trust your OSS data?

 by Sophia Vargas, Google Open Source & Andrew Nesbitt, Ecosyste.ms

Every second, open source contribution quietly shapes the software we rely on, and yet our view of this open ecosystem is surprisingly opaque. Open source development is performed in public spaces — we can see the commits, issues and comments, the APIs and endpoints are free to use — the logs are just sitting there, so why can’t we just collect all of the data?

…Said every researcher, everywhere. However in most cases of open source related data, we are only looking at part of the whole. Why am I writing this post? Because many of us (including many business decision-makers) are too comfortable with unsubstantiated data. We’ve gotten used to it. Our models assume that it's smelly and we adjust the logic and weights to compromise. When it comes to open source, our confidence is even lower, even though our resulting decisions can directly impact individuals whom we collectively depend on.

Let’s consider one of my favorite datasets: GHarchive. Started as a hobby project in 2011, this crawler has amassed more than 15 years of event data from GitHub. While this source provides a historical record of open source development on GitHub, as a real-time or comprehensive source of metrics, it's unreliable and should not be a source for volume-based metrics. 

In 2025, GHarchive captured 14% fewer events than in 2024, despite steady growth in platform adoption.  Since 2025, we estimate that data retention in GHarchive has fallen to ~50% and in 2026 it may be as low as 20% for some event types (see figure below). Prior to 2025, you could make the general assumption that the majority of events would be represented in this pipeline. Since 2025, we must now assume we may be missing at least half of events and possibly more — not to mention all of the additional activity that’s left out of the event API (see GitHub’s GraphQL API.) 

The crawler logic behind this dataset is simple: give me all the events from the GitHub Event stream (e.g. opening pull requests, commenting on issues etc). However, the GitHub API has limitations on the number of calls per hour as well as the number of events listed, so for days with a lot of spiky activity, the crawler will miss some. Although we never assumed that this dataset was collecting 100% of events, the current architecture is showing signs of strain. We suspect that this is due, in part, to the rate of repository growth and adoption of automated tooling on GitHub. In 2011, GitHub announced it reached 2 million public repositories, and by 2026, that figure surpassed 400 million.  

I want to acknowledge that building and sharing comprehensive open datasets at scale is hard. Have you ever built a pipeline only to discover that the variables changed mid year, the payload for one output is getting truncated, all your joins broke because one side of the dataset is case sensitive … I could go on. And these examples are just ordinary data issues. Building a dataset at the scale of GitHub where “Every second, more than one new developer on average joined GitHub—over 36 million in the past year”—you start running into a new set of challenges.

My own journey with open source related data began when I repeatedly found myself questioning how much we could trust our own metrics. To expand my understanding of the nuances and the limitations of open source related datasets, I reached out to Andrew Nesbitt, who has spent years digging in data trenches for the benefit of the community. Together we converged on the following issues that we wanted to highlight for the broader community.

Assembling: Assume there will be problems

When I asked Andrew ‘can you summarize the challenges you have faced assembling comprehensive datasets?’ —“I just assume I'm going to have a terrible time anyway, so I start with my best effort and fill in the gaps”. While disappointing, this aligned with most data aggregation methods I’ve reviewed—tools such as Grimoire labs and OSS insights also require multiple processes for collection, combination and reconciliation. Even with these approaches, many sources have missing, incomplete, or inconsistent information.

One source is probably not enough. If you are considering the use of an open source project, you may want to know how many maintainers work on this project, what versions are available, what their dependencies are and any active vulnerabilities or known issues. Each of these queries requires a distinct source—the development history, the dependency graph, the CVE database, etc. Ecosyste.ms strives to pull this information together into one place, but combining data from 1000+ datasets has its own unique set of challenges.

For example, my index is probably not your index. One perennial issue is inconsistent naming conventions across sources. Beyond variable type and format, repository names, versions, packages, tags, licenses, urls, etc. tend to be unique across platforms. Some are case sensitive, there are often duplicates, and anyone can change a name at any time… I’ve been keenly following the adoption of purl and SWHID, but so far I have not found one name to rule them all.

Now we have to keep this up to date: At the moment, there is no consistent way of sharing updates across platforms. Changes to names, APIs, deletions, etc. are more often discovered by errors and breakage than by scouring release notes. To keep Ecosyste.ms up to date, Andrew has written multiple syncing processes that identify or infer updates that need to be accounted for. I asked Andrew ‘If you could ask a platform/data source to change one thing, what would it be?’, “Can I crawl an endpoint that's just NEW stuff?’

Consuming: Design your pipeline for your use case

Because of LLMs, “it's now easier for anyone to try to access and build reports”. But those building quick reports are likely not going to go through the pain of being comprehensive. This is where aggregated sources like GHarchive and Ecosyste.ms thrive. As data providers, we’d love if data consumers knew that:

How you collect data matters. If everyone wanted the same dataset, in the same format, at the same time, it would be simple. Depending on how the data is stored—centralized vs distributed and cached, relational vs graph, etc. —queries could be more efficient (in cost and computation) than exports or bulk requests faster than individual requests. This all depends on the topology of the infrastructure and the dataset. In a perfect world, data producers would design their architecture for their top user journeys. However open source related datasets serve a wide variety of user personas from corporations to non-profits, researchers to individual users, maintainers, funders, and many more, with a variety of demands from historical deep dives to realtime feedback. Data producers can’t design for all of these cases, so my challenge to them is to be more open about the best way to access this information. 

At the end of the day, we have to respect the human infrastructure: Open source-related datasets are riddled with personally identifiable information (PII). Some individuals may be comfortable sharing their information with fellow contributors, but seeing it aggregated across platforms can be uncomfortable. Any source with PII should be handled with care: anonymize when you can and ensure you are in alignment with policies and regulations. Open source communities are real people so please, consume their data responsibly.

Interpreting: Never stop asking questions

While many have moved on from ‘data-driven’ to ‘AI-enabled’, the fact remains that ALL AI SYSTEMS DEPEND ON DATA. Our data about open source will continue to be incomplete and imperfect, but by asking questions about our sources, acknowledging the gaps, and considering both the technical and human processes behind open source development, we can refine and improve on how we interpret our insights and models even if they don’t completely reflect reality.

Securing the agentic era: Introducing formal verification for CEL

by Sean Huh, Common Expression Language Team

We are rapidly entering an era where AI agents can autonomously draft, refactor, and deploy policies that protect our users and our systems. But this velocity introduces a vital question: How do we trust AI-generated policies?

Unit tests may fail to cover the infinite set of possible inputs that occur in production; thus, an AI agent that overfits its policy to existing tests may fail spectacularly in production. To secure automated policy authoring, we must combine heuristic testing with mathematical proofs.

We are thrilled to announce the Common Expression Language (CEL) Formal Verification Framework is now available. Powered by the Z3 theorem prover, this framework allows you to prove the correctness of your CEL expressions and policies, serving as the ultimate safety net for the agentic policy.

Automated reasoning definitively answers questions like:

• “Is there any combination of inputs that allows an unapproved request into production?”
• “Are we absolutely certain this AI-refactored policy matches the original behavior?”
• “Can a bad actor manipulate this rule to force an evaluation error?”

Formal verification establishes mathematical certainty across the infinite spectrum of inputs. Proven policies protect your users and system while giving auditors clear proof of compliance.

To see these capabilities in action, watch our video demonstrating how the CEL Verifier REPL catches subtle logic flaws in seconds:

Proving rules from the ground up

Getting started with formal verification doesn’t require learning complex architectures right away. You can evaluate simple standalone CEL expressions to catch edge cases that tests easily miss.

(Note: The examples below use our interactive REPL syntax—check out the REPL documentation to follow along!)

1. Catching logic bugs in simple expressions (Equivalence)

How do you guarantee a refactored rule behaves identically to the original? Suppose we have a policy that allows ports 80 or 443 in production. An agent might factor the is_prod check like so:

equiv
  (is_prod && port == 80) || (is_prod && port == 443) 
  <=>
  is_prod && port == 80 || port == 443

Because logical AND has a higher operator precedence than OR, the verifier immediately flags Violated, and outputs the exact exploit: in a non-production environment (is_prod = false), the rule mistakenly allows port 443. Fixing the grouping parentheses returns Verified.

2. Enforcing exhaustive guardrails (Validity)

This capability scales directly to use cases like Kubernetes Validating Admission Policies. Suppose an engineer writes a guardrail expression that assumes every request will either be on a low port (under 80) or a high port (over 1024):

valid request.port > 1024 || request.port <= 80

When we check validity (whether an expression holds true for all inputs), the verifier exhaustively searches the entire integer space, flags Violated, and outputs the exact counterexample:

[VIOLATED] Condition is not always true. Counterexample input:
  request.port = 81

3. Guaranteeing security invariants with CEL Policy

While the verifier works perfectly with standalone CEL expressions, complex environments compose multiple rules and variables. Here, the CEL policy format shines. Using assume and assert blocks, the verifier proves a mathematical implication: if the assumptions hold, the assertions must also hold.

name: workload_admission
rule:
  variables:
    - is_admin: 'request.auth.claims.groups.exists(g, g == "admin")'
  match:
    # A subtle flaw introduced during authoring:
    - condition: 'request.is_privileged && request.is_prod'
      output: 'true'
    - condition: 'variables.is_admin || request.has_approval'
      output: 'true'
    - output: 'false'
verification:
  invariants:
    - id: universal_no_unapproved_privileged_prod
      assume:
        - 'request.has_approval == false'
        - 'variables.is_admin == false'
      assert:
        - 'rule.result == false'

The first condition admits privileged workloads into production without checking for approval or admin status. The verifier flags this and provides an example that exploits the issue:

Invariant 'universal_no_unapproved_privileged_prod' violation detected. Counterexample input:
  request.is_privileged = true
  request.is_prod = true
  request.has_approval = false
  request.auth.claims.groups = []

Assertions and assumptions define the boundaries of acceptable agent behavior, allowing developers to configure CI/CD pipelines to validate AI-generated changes simply and securely.

Under the hood: High-fidelity mathematical modeling

Translating a dynamic language into the Satisfiability Modulo Theories (SMT) domain requires immense engineering rigor to prevent the solver from hanging or hallucinating bugs. Our engine provides:

Zero false positives via three-pass taint tracking

Traditional verification tools are prone to “solver hallucinations”—reporting fake bugs when encountering custom domain-specific functions or external variables they don’t fully understand. To eliminate this noise, if a potential issue relies on an unmapped custom function, the verifier isolates and flags it as Inconclusive rather than breaking your CI pipeline with a false alarm. This guarantees every Violation report is a 100% real, reproducible bug.

Deep structural extensionality

The Formal Verification Framework offers configurable-depth bounded-model checking to prevent infinite loops within SMT quantifiers. These configurable limits allow you to control the cost of verification when analyzing deep structure equivalence in expressions like [[1], [2]] == [[1], [2]].

The mandatory bridge of trust

In the agentic era, code writes code. Mathematical proof isn’t just a nice-to-have; it is the fundamental bridge of trust developers require to let AI operate autonomously in their most sensitive systems. Get started with the CEL Formal Verification Framework, to take the next step toward a more secure agentic future today!

Let us know what you think—issues, pull requests, and feedback are always welcome!

Google joins the OpenROAD Initiative as principal member to accelerate open source silicon innovation

by Ethan Mahintorabi & Aaron Cunningham, Hardware Toolchains Team

Google is committed to advancing open source silicon innovation. We are excited to share that we have formally joined the OpenROAD Initiative (ORI), Inc. as a principal member. ORI is a nonprofit public benefit corporation dedicated to the open source electronic design automation (EDA) ecosystem. As part of this commitment, Aaron Cunningham has been appointed to the ORI Governing Board to represent Google and help drive the foundation’s strategic direction, financial sustainability, and technical stewardship.

Driving long-term open source sustainability

The OpenROAD Initiative’s mission is to advance and sustain the open source EDA ecosystem by fostering collaborative innovation across research, education, and industry—transforming ideas into silicon. Google’s membership aligns directly with ORI’s multi-year sustainability goals, supported by the US National Science Foundation’s (NSF) Pathways to Enable Open-Source Ecosystems (POSE) program.

With Google’s participation and membership commitment, ORI will continue to strengthen, grow, and sustain its open source ecosystem through key vectors:

• Neutral Stewardship: Fostering transparent governance where no single company has outsized control over the code, ensuring the project remains inspectable, accessible, and community-driven.
• Ecosystem Growth: Supporting open and reproducible silicon research, developing robust design flows, and hosting global design contests.
• Workforce Development: Supporting global silicon skilling initiatives by expanding open source chip design curricula and collaborating with academic institutions and industrial training networks.
• Technical Strengthening: Enhancing continuous integration and deployment (CI/CD) pipelines, expanding PDK enablement, and improving user experience.

Leadership perspectives

“The OpenROAD Initiative is built on the vision of making chip design open and accessible to all—building a collaborative ecosystem driven by transparency and shared innovation,” said Andrew Kahng, board member of the OpenROAD Initiative. “Google’s deep commitment to open source software and hardware makes them an ideal partner. By joining at our highest membership tier, Google is helping to ensure that the open source EDA ecosystem has the stable, long-term governance and financial foundation required to grow.”

“Cutting-edge silicon research requires robust, inspectable, and reproducible toolchains,” said Drew Wingard, Director of Silicon Infrastructure, Tools and Methodology at Google. “OpenROAD has already made an incredible impact across academia and the broader industry, enabling many successful tapeouts. Google is proud to support the OpenROAD Initiative’s mission to scale this open infrastructure for the next generation of developers.”

About the OpenROAD Initiative and OpenROAD project

The OpenROAD Initiative, Inc. is a California-based 501(c)(3) nonprofit organization that provides governance, stewardship, and coordination for the OpenROAD ecosystem. The OpenROAD Project is an open source, autonomous digital chip design toolchain that democratizes semiconductor design, enabling a complete RTL-to-GDSII flow in less than 24 hours with no human in the loop. Grounded in academic research and referenced in over 500 peer-reviewed publications, OpenROAD has lowered the barriers to hardware innovation, enabling thousands of students, researchers, and startups worldwide to design and manufacture chips.

To learn more about the OpenROAD Project and install the toolchain, visit the new OpenROAD website.

For more information about membership tiers and the foundation’s governance, visit the OpenROAD Initiative website at www.openroadinitiative.org or contact membership@openroadinitiative.org.

Adapting open source practices to an AI-first world: A retrospective on 2025

by Sophia Vargas, Google Open Source

Even as AI adoption accelerates and transforms the global technology landscape, open source remains foundational to how Alphabet builds, uses, and collaborates on products for billions of users. Our commitment to open source remains broad and consistent, including sharing our work year-over-year, and reflecting on what we've learned.

In 2025:

• Roughly 10% of Alphabet's full-time workforce actively contributed to open source projects. This contribution ratio has remained steady over the past five years, scaling to match our growth.
• These open source contributions are not just solely focused on Google. Our top projects by unique contributors at Alphabet include community-led projects such as LLVM, vLLM, Envoy, and Rust, as well as Google-initiated projects like Kubernetes, Apache Beam, and gRPC.
• In addition, Alphabet projects received commits from more than 20,000 non-Alphabet affiliated user accounts.

Working together on emerging standards

Open source communities continue to provide vital collaborative spaces to define emerging standards, ensuring the interoperability and extensibility for the next generation of technologies. In 2025, we worked with more than 50 partners on the Agent2Agent (A2A) protocol to enable AI agents to communicate with each other, securely exchange information, and coordinate actions on top of various enterprise platforms and applications. Within weeks of our initial announcement, Google donated the A2A project to the Linux Foundation as part of our long-standing commitment to develop "open, collaborative ecosystem – offering greater autonomy and multiplying productivity."

Launching tools with transparency

Open source licenses provide a framework for anyone to explore, test, fork and expand on our technologies. Over the last 15 years, Google has created more than 15,000 public repositories on GitHub. Today, Google continues to maintain more than 5,000 public repositories on GitHub, and more than 1,500 public repositories on Git-on-Borg. A quick look back to some highlights from our 2025 launches includes:

• Gemma 3 – a collection of lightweight open models built from the same research and technology that powers our Gemini 2.0 models. They are designed to run directly on devices — from phones and laptops to workstations — helping developers create AI applications, wherever people need them. Since Gemma’s release in 2024, the Gemmaverse community has created more than 60,000 Gemma variants. We’ve continued work on this collection and in 2026 we released Gemma 4 under an Apache 2.0 license.
• Agent Development Kit – an open-source framework and SDK designed to help developers build, compose, and run both conversational and non-conversational AI agents.

Adjusting our collective security practices

As the threat landscape evolves and accelerates in the AI era, we are working on novel solutions to mitigate vulnerabilities at scale and remove some of the burden from overloaded maintainers. In 2025, we introduced several critical security initiatives to support upstream open source projects:

• CodeMender: An agent designed to be both reactive, instantly patching new vulnerabilities, and proactive, rewriting and securing existing code and eliminating entire classes of vulnerabilities in the process. In its first six months, we were able to upstream 72 security fixes to open source projects, including some as large as 4.5 million lines of code.
• OSS Rebuild: A security initiative to prevent software supply chain attacks by verifying build provenance. It automates package rebuilding and semantically compares results to upstream artifacts to detect tampered code on registries like PyPI, npm, and Crates.io.
• OSV-Scalibr: (Software Composition Analysis Library) the core engine for vulnerability scanning used internally and in OSV-Scanner.
• Credential Scanning on deps.dev: A service (using Veles) that actively scans open-source packages on deps.dev for leaked GCP credentials to prevent account compromise.

Working together to sustain the contributor community

Beyond security contributions, we remain committed to providing financial support directly to projects and maintainers. In 2025, the Open Source Programs Office (OSPO) directed $2M in sponsorships and investments to more than 40 open source projects.

Google Summer of Code celebrated its 21st year of enabling open source organizations to find, mentor, and onboard new contributors, directly supporting 1,280 individuals to contribute to 185 organizations. Over its lifetime, the global program has connected more than 23,000 participants from 125 countries with over 1,000 open source organizations globally. Moving forward, we recognize the need to evolve our program structures to ensure they continue to effectively support maintainers in the AI era.

Evolving the way we work in open spaces

Respecting community norms and practices is fundamental to establishing and maintaining trust in open source communities. To ensure we remain supportive members of your community, we encourage projects to document preferred practices and policies in the wake of new technologies. As AI changes the way we work, we are continually evaluating how best to evolve the way we engage in open spaces to preserve, prepare and bolster the communities we depend on.

We are deeply grateful for the many individuals and organizations that have worked with us to create global technologies from which everyone can benefit. You can continue to learn more about our open source initiatives, ongoing programs, and new projects at opensource.google.

Appendix: About this data

This report features metrics provided by many teams and programs across Alphabet. In regards to the code and code-adjacent activities data, we wanted to share more details about the derivation of those metrics.

• Data sources: These data represent the activities of Alphabet employees on public repositories hosted on GitHub and our internal production Git service Git-on-Borg. These sources represent a subset of open source activity currently tracked by Google OSPO.
• Business and personal: Activity on GitHub reflects a mixture of Alphabet projects, third-party projects, experimental efforts, and personal projects. Our metrics report on all of the above unless otherwise specified.
• Alphabet contributors: Please note that unless additional detail is specified, activity counts attributed to Alphabet open source contributors will include our full-time employees as well as our extended Alphabet community (temps, vendors, contractors, and interns). In 2025, full time employees at Alphabet represented more than 95% of our open source contributors.
• GitHub Accounts: For counts of GitHub accounts not affiliated with Alphabet, we cannot assume that one account is equivalent to one person, as multiple accounts could be tied to one individual or bot account.
• Active counts: Where possible, we will show ‘active users’ defined by logged activity (excluding ‘WatchEvent’) within a specified timeframe (a month, year, etc.) and ‘active repositories’ and ‘active projects’ as those that have enough activity to meet our internal active-project criteria and have not been archived.

This Week in Open Source for July 16, 2026

by Daryl Ducharme, Open Source Programs Office

This Week in Open Source for July 16, 2026

A look around the world of open source

We’re diving into another week of open source news and discussions. As the ecosystem continues to evolve, we're seeing more conversations around the intersection of AI workflows, security at scale, and the infrastructure that supports the open web.

This week, we’re highlighting a few "Open Source Reads" that tackle some of the biggest questions facing our ecosystem today—from the complex ethics of AI-generated workflows to the future of federated social networks. We hope these links provide valuable context as we work together to sustain the critical infrastructure we all rely on.

Upcoming Events

• ASWF Open Source Days (July 19–20) — Los Angeles, CA. Hosted by the Academy Software Foundation, focusing on open source software in visual effects (VFX) and animation.
• KubeCon + CloudNativeCon Japan 2026 (July 28–30) — Yokohama, Japan. The Cloud Native Computing Foundation's flagship conference in Japan.
• Black Hat & DEF CON 2026 (August 1–9) — Las Vegas, NV. While primarily cybersecurity, both heavily feature open source hacking tools. The Open Source Security Foundation (OpenSSF) also has a major presence here.
• FOSSY (August 6–9) — Vancouver, Canada. A highly community-focused open source conference emphasizing grassroots participation and collaboration.
• Community over Code Asia (August 7–9) — Beijing, China. The Asian edition of the Apache Software Foundation's (ASF) community-first conference.
• Open Source Summit Korea (August 11–12) — Seoul, South Korea. Hosted by the Linux Foundation, bringing together open source maintainers and enterprises.
• KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China (September 7–9) — Shanghai, China. A massive collaborative mega-event uniting open infrastructure, cloud, and AI communities.
• OSPOlogy + OSPO Summit (September 7) — Shanghai, China. Focused on Open Source Program Offices (OSPOs), governance, and corporate open source sustainability.

Open Source Reads and Links

• [Blog] Elephants, Goldfish and the New Golden Age of Software Engineering - This isn't directly open source, but with the open source ecosystem trying to find good AI workflows I thought the Elephant-Goldfish model was an interesting take. It makes the distinction between using AI as a toy and using it as a tool.
• [Article] Open-source AI and the Choice Before Us - The communities with minimal influence on AI development will likely be the most affected by it. This article looks at how strong rules and institutions are needed to guide AI's safe and fair use. The future of AI depends on whether we build systems that include everyone or just deepen existing divides.
• [Article] Open source carries the world. Patching it at Mythos-scale can't fall to maintainers alone. - The past year has shown that AI is speeding up the discovery of vulnerabilities in open source software. Endor Labs is offering a solution to utilize AI to create patching stop gaps for users of the libraries so that maintainers and contributors have the time to create good quality fixes to these vulnerabilities.
• [Blog] The AT-URI Syntax Mess - I am very excited about the open social web, especially things happening around the AT protocol. Early misunderstanding led to the current AT URI syntax being invalid. This article looks at the possible ways forward.
• [Article] The Open Social Web Needs Section 230 to Survive - The open web has always been a promise of free speech. When walled garden social media got involved it created some questions around whether free speech applies to the content served by the corporations running those networks. However, the open social web is not run by corporations. It can be, and it can also be run by individuals federating instances or running their own PDS. So, how do we maintain the free speech promise of the internet with this in mind?

Which of these stories will you be chatting about at your next meetup or conference? Let us know! Share with us on our @GoogleOSS X account or our @opensource.google Bluesky account.