opensource.google.com

Menu

Open Source in the 2021 Accelerate State of DevOps Report

Wednesday, September 22, 2021

To truly thrive, organizations need to adopt practices and capabilities that will lead them to performance improvements. Therefore, having access to data-driven insights and recommendations about the most effective and efficient ways to develop and deliver technology is critical. Over the past seven years, the DevOps Research and Assessment (DORA) has collected data from more than 32,000 industry professionals and used rigorous statistical analysis to deepen our understanding of the practices that lead to excellence in technology delivery and to powerful business outcomes.
 
One of the most valuable insights that has come from this research is the categorization of organizations on four different performance profiles (Elite, High, Medium, and Low) based on their performance on four software delivery metrics centered around throughput and stability - Deployment Frequency, Lead Time for Changes, Time to Restore Service and Change Failure Rate. We found that organizations that excel at these four metrics can be classified as elite performers while those that do not can be classified as low performers. See DevOps Research and Assessment (DORA) for a detailed description of these metrics and the different levels of organizational performance.

DevOps Research and Assessment (DORA) showing a detailed description of these metrics and the different levels of organizational performance

We have found that a number of technical capabilities are associated with improved continuous delivery performance. Our findings indicate that organizations that have incorporated loosely coupled architecture, continuous testing and integration, truck-based development, deployment automation, database change management, monitoring and observability and have leveraged open source technologies perform better than organizations that have not adopted these capabilities.

Now that you know a little bit about what DORA is and some of its key findings, let’s dive into whether the use of open source technologies within organizations impacts performance.

A quick Google search will yield hundreds (if not, thousands) of articles describing the myriad of ways organizations benefit from using open source software—faster innovation, higher quality products, stronger security, flexibility, ease of customization, etc. We know using open source software is the way to go, but until recently, we still had little empirical evidence demonstrating that its use is associated with improved organizational performance – until today.

This year, we surveyed 1,200 working professionals from a variety of industries around the globe about the factors that drive higher performance, including the use of open source software. Research from this year’s DORA report illustrates that low performing organizations have the highest use of proprietary software. In contrast, elite performers are 1.75 times more likely to make extensive use of open source components, libraries, and platforms. We also find that elite performers are 1.5 times more likely to have plans to expand their use of open source software compared to their low-performing counterparts. But, the question remains—does leveraging open source software impact an organization’s performance? Turns out the answer is, yes!

Our research also found that elite performers who meet their reliability targets are 2.4 times more likely to leverage open source technologies. We suspect that the original tenets of the open source movement of transparency and collaboration play a big role. Developers are less likely to waste time reinventing the wheel which allows them to spend more time innovating, they are able to leverage global talent instead of relying on the few people in their team or organization.

Technology transformations take time, effort, and resources. They also require organizations to make significant mental shifts. These shifts are easier when there is empirical evidence backing recommendations—organizations don’t have to take someone’s word for it, they can look at the data, look at the consistency of findings to know that success and improvement are in fact possible.

In addition to open source software, the 2021 Accelerate State of DevOps Report discusses a variety of capabilities and practices that drive performance. In the 2021 report, we also examined the effects of SRE best practices, the pandemic and burnout, the importance of quality documentation, and we revisited our exploration of leveraging the cloud. If you’d like to read the full report or any previous report, you can visit cloud.google.com/devops.

Learn Kubernetes with Google: Join us live on October 6!

Tuesday, September 21, 2021

 

Graphic describing the Multi-cluster Services API functionalities

Kubernetes hasn’t stopped growing since it was released by Google as an open source project back in June 2014: from July 7, 2020 to a year later in 2021, there were 2,284 new contributors to the project1. And that’s not all: in 2020 alone, the Kubernetes project had 35 stable graduations2. These are 35 new features that are ready for production use in a Kubernetes environment. Looking at the CNCF Survey 2020, use of Kubernetes has increased to 83%, up from 78% in 2019. With these many new people joining the community, and the project gaining so much complexity: how can we make sure that Kubernetes remains accessible to everyone, including newcomers?

This is the question that inspired the creation of Learn Kubernetes with Google, a content program where we develop resources that explain how to make Kubernetes work best for you. At the Google Open Source Programs Office, we believe that increasing access for everyone starts by democratizing knowledge. This is why we started with a series of short videos that focus on specific Kubernetes topics, like the Gateway API, Migrating from Dockershim to Containerd, the Horizontal Pod Autoscaler, and many more topics!

Join us live

On October 6, 2021, we are launching a series of live events where you can interact live with Kubernetes experts from across the industry and ask questions—register now and join for free! “Think beyond the cluster: Multi-cluster support on Kubernetes” is a live panel that brings together the following experts:
  • Laura Lorenz - Software Engineer (Google) / Member of SIG Multicluster in the Kubernetes project
  • Tim Hockin - Software Engineer (Google) / Co-Chair of SIG Network in the Kubernetes project
  • Jeremy Olmsted-Thompson - Sr Staff software Engineer (Google) / Co-Chair of the SIG Multicluster in the Kubernetes project
  • Ricardo Rocha - Computing Engineer (CERN) / TOC Member at the CNCF
  • Paul Morie - Software Engineer (Apple) / Co-Chair of the SIG Multicluster in the Kubernetes project
Why is Multi-cluster support in Kubernetes important? Kubernetes has brought a unified method of managing applications and their infrastructure. Engineering your application to be a global service requires that you start thinking beyond a single cluster; yet, there are many challenges when deploying multiple clusters at a global scale. Multi-cluster has many advantages, it lets you minimize the latency and optimize it for the people consuming your application.

In this panel, we will review the history behind multi-cluster, why you should use it, how companies are deploying multi-cluster, and what are some efforts in upstream Kubernetes that are enabling it today. Check out the “Resources” tab on the event page to learn more about the Kubernetes MCS API and Join us on Oct 6!

By María Cruz, Program Manager – Google Open Source Programs Office

1 According to devstats

Kubernetes Community Annual Report 2020

Announcing HIBA: Host Identity Based Authorization for SSH

Monday, September 20, 2021

How do you manage SSH access to a fleet of hosts? While OpenSSH provides many methods from a simple password to the use of certificates, each of these on its own still presents challenges.

Let's start by clarifying the difference between Authentication and Authorization. The former is a way to prove you are the entity you claim to be. This is usually achieved by providing the secret password associated with your account or by signing a challenge proving you own the private key corresponding to a public key. Authorization is a way to decide whether an entity is allowed or not to access a resource—usually performed after Authentication happens.

Request ⇒ AuthenticateAuthorize ⇒ Result

Authorization with OpenSSH is typically done using one of the following:
  • The principal knows the password of the target role account on the host.
  • The principal's public key is present in the authorized_keys of the target role account on the host.
  • The principal's username is present in the authorized_users of the target role account on the host.
Transposing this into a real life example: the bouncer of the Foo bar asks for your ID and checks if the photo matches your face (authentication), then decides, based on whether your name is on the guest list, if you are allowed in or not (authorization).

All these methods require accessing the target host in order to update authorizations (the guest list) by either updating the password, or adding/removing authorized_(keys|users).

Note: OpenSSH's use of certificates provides an extra layer of authorization by requiring a Certificate Authority (CA) to trust the incoming public key. Short lived certificates provide a flexible on/off switch for globally authorizing an entity access to the targeted hosts, but they still don't solve the problem of per hosts authorization policies.

Introducing Hiba image
Introducing HIBA

HIBA is a layer on top of OpenSSH that aims at solving this problem:
  • Centralized policy for controlling per host authorization.
  • Hermetic authorization mechanism making it suitable for deep embedded applications or emergency situations.
Going back to our previous nightclub example: the certificate is your ID, and the signed challenge is your photo/face matching. HIBA would be the bouncer, except its decision is not based on a guest list, but rather on you presenting a valid access ticket.

HIBA achieves this by relying on OpenSSH and the certificate infrastructure.
  • Authorization decisions are made centrally by the CA.
  • Authorization is stored as a certificate extension signed by the CA.
  • Hosts trust the CA thanks to TrustedUserCAKeys sshd configuration option.
  • Hosts enforce the authorization policy presented with the certificate by relying on HIBA via the AuthorizedPrincipalsCommand sshd configuration option.

In more details

HIBA defines two extensions to SSH certificates:
  • The HIBA identity, attached to host certificates, lists properties defining this host. They will be used as criteria for granting access.
  • The HIBA grant, attached to user certificates, lists constraints that a host must match for access to be granted.
On the host side, the hiba-chk helper is responsible for decoding extensions from both the host and user certificates and deciding whether or not to grant access.

On the CA side, the HIBA policy defines a set of grants and a list of users allowed to request them. Users can now ask the CA for HIBA grants to be attached to their certificates. The CA will confirm eligibility based on the policy and sign the certificate after adding the requested HIBA grants.

The HIBA extension scheme is flexible by nature, and any type of constraint can be assigned to grants as long as they are also defined in the identity. The exact specifications can be found in the PROTOCOL.authorizations file. HIBA only defines a handful of reserved constraints that are described in the PROTOCOL.extensions file.

The HIBA source code implements hiba-chk for processing authorizations as well as a library (libhiba) and a CLI (hiba-gen) for generating HIBA extensions. It also provides a simple shell based CA implementation hiba-ca.sh.

Practical example

Hosts identities

Host identities example

Grants

Grants example

In this example:
  • A user with an Employee SEA grant can access any foo.bar hosts located in Seattle.
  • A user with the Guests grant can access foo.bar hosts everywhere, but only if they are Dance floor, while Celebrities can access both VIP lounge and Dance floor hosts.
  • The owner grant gives unrestricted access to any foo.bar hosts.
By Tristan Lelong – Site Reliability Enthusiast, Technical Infrastructure
.