opensource.google.com

Menu

Google Season of Docs announces results of 2023 program

Tuesday, December 5, 2023

Google Season of Docs is happy to announce the 2023 program results, including the project case studies.

Google Season of Docs is a grant-based program where open source organizations apply for US$5-15,000 to hire technical writers to complete documentation projects. At the end of the six-month documentation development phase, organizations submit a case study to outline the problems their documentation project was intended to solve, how they are measuring the success of their documentation project, and what they learned during the project. The case studies are publicly available and are intended to help other open source organizations learn best practices in open source documentation.

The 2023 Google Season of Docs documentation development phase began on March 31 and ended November 21, 2023 for all projects. Participants in the 2023 program will also answer three followup surveys in 2024, in order to better track the impact of these documentation projects over time.

Feedback from the 2023 participating projects was extremely positive:

“I would strongly recommend engaging with a technical writer who is genuinely passionate about open-source initiatives. A writer who asks probing questions, encourages leaders to think innovatively, and is eager to learn in unfamiliar domains can be incredibly beneficial."
      – Digital Biomarker Discovery Project
“Having a dedicated resource under the banner of GSoD helped as it allowed the team to focus on core activities while leaving out the worries related to the stacking documentation challenges behind, to be taken care of by the writer."
      – Flux
“We made significant improvements to nearly half of the p5.js reference and laid the groundwork for a team of writers currently working on documentation. Along the way, we engaged a broad cross-section of the community and strengthened bonds among core contributors. "
      – p5.js

Take a look at the participant list to see the initial project plans and case studies!


What’s next?

Stay tuned for information about Google Season of Docs 2024—watch for posts on this blog and sign up for the announcements email list. We’ll also be publishing the 2023 case study summary report in early 2024.

If you were excited about participating in the 2023 Google Season of Docs program, please do write social media posts. See the promotion and press page for images and other promotional materials you can include, and be sure to use the tag #SeasonOfDocs when promoting your project on social media. To include the tech writing and open source communities, add #WriteTheDocs, #techcomm, #TechnicalWriting, and #OpenSource to your posts.

By Erin McKean, Google Open Source Programs Office

Google Summer of Code 2023 Final Results!

Monday, December 4, 2023

On November 17th, we wrapped up our 2023 Google Summer of Code program where 903 contributors completed open source projects for 168 OSS organizations. This year 70% (628) of the successful 2023 contributors opted for a 12-week project, while the remaining 30% (275) completed their GSoC work over the past few weeks. That being said, many contributors choose to continue involvement in the OSS community after finishing their GSoC projects. GSoC is typically just one small chapter in a contributor's lifetime open source journey.

This certainly was one of our most enthusiastic groups of mentors & GSoC contributors yet. We were able to host multiple virtual check-ins where contributors had the chance to ask GSoC Administrators questions and get live reminders and advice regarding the program and its milestones, the response to these sessions was overwhelming with one session having over 60% of 2023 GSoC contributors attending. Our final virtual event as part of this series was a multi-day ‘Contributor Talks’ Series where 43 participants had the chance to give three minute Lightning Talks about their GSoC projects.

Quick GSOC 2023 by the Numbers: 99% of 2023 orgs say that GSoC has value for their organization, 83% of 2023 contributors said they would consider being a mentor, 30% of 2023 contributors said that GSoC has already helped them with a job offer, 99.8% of 2023 contributors plan to continue working on open source, 88%
of 2023 contributors were first-time GSoC contributors meaning 12% had participated in GSOC before, 96.25% of contributors said they would apply to GSoC again.
In 2023, GSoC contributors rated their GSOC experience as 3.79/4. Mentors gave their GSoC contributor's overall performance a 4.41/5.

Our mentors and GSoC contributors spent a lot of time giving us invaluable feedback from the program so we wanted to share a few top insights below. Their comments help us to keep the program relevant and to continue to meet the needs of open source communities and new open source contributors.


Advice for future contributors

As we head into our 20th year of GSoC, we wanted to highlight some of the advice that the 2023 GSoC contributors offered to future contributors. Much of the advice falls into the themes of:

    • Communicate early and often with mentors.
    • Take the time in February as soon as orgs are announced to find the right org and choose a project you are excited about, it will make the program much more enjoyable.
    • Set realistic goals and break tasks into milestones.
    • Be open to learning! Open source can seem intimidating but you have amazing mentors and the community there encouraging and supporting you.

We welcomed 18 new mentoring organizations this year, many of which were able to attend our Mentor Summit on Google’s campus a few weeks ago.

In 2023, 10.15% of GSoC contributors were non-students. This was the second year since we opened up the program to non-students. We hope to continue to have more potential GSoC contributors who are changing careers or not currently enrolled in academic programs join the program.

“My advice is to just go for it. I'm a thirty-something career-changer who doesn't have a technical background; at times, I doubt myself and my ability to transition into a more technical field. During GSoC, I was paired with knowledgeable, friendly, engaging mentors who trusted me to get the work done. It was empowering, and I did work that I'm extremely proud of. To anyone in my shoes who may be afraid to take the plunge, I highly encourage them to do so. Seriously - if I can do it, anyone can."
      – brittneyjuliet, GSoC’23 Contributor

Favorite part of GSoC

GSoC contributors have shared their favorite parts of GSoC with some very common themes:

    • Working on real-world projects that thousands/millions of people actually use and rely on
    • Interacting with experienced developers and truly being part of an enthusiastic, welcoming community
    • Making a difference
    • Gaining overall skills and confidence to boost their careers that can’t be obtained from classrooms alone

How GSoC improved their programming skills

95.5% of contributors believe GSoC improved their programming skills. The most common responses to how GSoC improved their skills were:

    • Practical experience. Applying programming concepts and techniques to real projects.
    • Learning to write cleaner, more maintainable code.
    • Enhanced problem solving skills.
    • Project management - learned how to break large, complex problems into smaller, organized tasks.
    • Learning to understand complex codebases.
    • Learning new concepts and technologies.
    • Engaging in code reviews with mentors regularly helped to grasp industry best practices.

We want to thank all of our mentors, organization administrators, and GSoC contributors for a rewarding and smooth GSoC 2023. The excitement from our GSoC contributors throughout the program and our mentors at the recent Mentor Summit was palpable. Thank you all for the time and energy you put in to make open source communities stronger and sustainable.


GSoC 2024 will be open for organization applications from January 22–February 6, 2024. We will announce the 2024 accepted GSoC organizations February 21 on the program site: g.co/gsoc. GSoC contributor applications will be open March 18–April 2, 2024.

By Stephanie Taylor, Program Manager, and Perry Burnham, Associate Program Manager for the Google Open Source Programs Office

Kubernetes CRD Validation Using CEL

Thursday, November 30, 2023

Motivation

CRDs was used to support two major categories of built-in validation:

  • CRD structural schemas: Provide type checking of custom resources against schemas.
  • OpenAPIv3 validation rules: Provide regex ('pattern' property), range limits ('minimum' and 'maximum' properties) on individual fields and size limits on maps and lists ('minItems', 'maxItems').

For use cases that cannot be covered by build-in validation support:

  • Admission Webhooks: have validating admission webhook for further validation
  • Custom validators: write custom checks in several languages such as Rego

While admission webhooks do support CRDs validation, they significantly complicate the development and operability of CRDs.

To provide an self-contained, in-process validation, an inline expression language - Common Expression Language (CEL), is introduced into CRDs such that a much larger portion of validation use cases can be solved without the use of webhooks.

It is sufficiently lightweight and safe to be run directly in the kube-apiserver, has a straight-forward and unsurprising grammar, and supports pre-parsing and typechecking of expressions, allowing syntax and type errors to be caught at CRD registration time.


CRD validation rule

CRD validation rules are promoted to GA in Kubernetes 1.29 to validate custom resources based on validation rules.

Validation rules use the Common Expression Language (CEL) to validate custom resource values. Validation rules are included in CustomResourceDefinition schemas using the x-kubernetes-validations extension.

The Rule is scoped to the location of the x-kubernetes-validations extension in the schema. And self variable in the CEL expression is bound to the scoped value.

All validation rules are scoped to the current object: no cross-object or stateful validation rules are supported.

For example:

...
  openAPIV3Schema:
    type: object
    properties:
      spec:
        type: object
        x-kubernetes-validations:
          - rule: "self.minReplicas <= self.replicas"
            message: "replicas should be greater than or equal to minReplicas."
          - rule: "self.replicas <= self.maxReplicas"
            message: "replicas should be smaller than or equal to maxReplicas."
        properties:
          ...
          minReplicas:
            type: integer
          replicas:
            type: integer
          maxReplicas:
            type: integer
        required:
          - minReplicas
          - replicas
          - maxReplicas

will reject a request to create this custom resource:

apiVersion: "stable.example.com/v1"


kind: CronTab


metadata:


 name: my-new-cron-object


spec:


 minReplicas: 0


 replicas: 20


 maxReplicas: 10

with the response:

The CronTab "my-new-cron-object" is invalid:
* spec: Invalid value: map[string]interface {}{"maxReplicas":10, "minReplicas":0, "replicas":20}: replicas should be smaller than or equal to maxReplicas.

x-kubernetes-validations could have multiple rules. The rule under x-kubernetes-validations represents the expression which will be evaluated by CEL. The message represents the message displayed when validation fails.

Note: You can quickly test CEL expressions in CEL Playground.

Validation rules are compiled when CRDs are created/updated. The request of CRDs create/update will fail if compilation of validation rules fail. Compilation process includes type checking as well.

Validation rules support a wide range of use cases. To get a sense of some of the capabilities, let's look at a few examples:

Validation Rule

Purpose

self.minReplicas <= self.replicas

Validate an integer field is less than or equal to another integer field

'Available' in self.stateCounts

Validate an entry with the 'Available' key exists in a map

self.set1.all(e, !(e in self.set2))

Validate that the elements of two sets are disjoint

self == oldSelf

Validate that a required field is immutable once it is set

self.created + self.ttl < self.expired

Validate that 'expired' date is after a 'create' date plus a 'ttl' duration

Validation rules are expressive and flexible. See the Validation Rules documentation to learn more about what validation rules are capable of.


CRD transition rules

Transition Rules make it possible to compare the new state against the old state of a resource in validation rules. You use transition rules to make sure that the cluster's API server does not accept invalid state transitions. A transition rule is a validation rule that references 'oldSelf'. The API server only evaluates transition rules when both an old value and new value exist.

Transition rule examples:

Transition Rule

Purpose

self == oldSelf

For a required field, make that field immutable once it is set. For an optional field, only allow transitioning from unset to set, or from set to unset.

(on parent of field) has(self.field) == has(oldSelf.field)

on field: self == oldSelf

Make a field immutable: validate that a field, even if optional, never changes after the resource is created (for a required field, the previous rule is simpler).

self.all(x, x in oldSelf)

Only allow adding items to a field that represents a set (prevent removals).

self >= oldSelf

Validate that a number is monotonically increasing.

Using the Functions Libraries

Validation rules have access to a couple different function libraries:

Examples of function libraries in use:

Validation Rule

Purpose

!(self.getDayOfWeek() in [0, 6]

Validate that a date is not a Sunday or Saturday.

isUrl(self) && url(self).getHostname() in [a.example.com', 'b.example.com']

Validate that a URL has an allowed hostname.

self.map(x, x.weight).sum() == 1

Validate that the weights of a list of objects sum to 1.

int(self.find('^[0-9]*')) < 100

Validate that a string starts with a number less than 100

self.isSorted()

Validate that a list is sorted

Resource use and limits

To prevent CEL evaluation from consuming excessive compute resources, validation rules impose some limits. These limits are based on CEL cost units, a platform and machine independent measure of execution cost. As a result, the limits are the same regardless of where they are enforced.


Estimated cost limit

CEL is, by design, non-Turing-complete in such a way that the halting problem isn’t a concern. CEL takes advantage of this design choice to include an "estimated cost" subsystem that can statically compute the worst case run time cost of any CEL expression. Validation rules are integrated with the estimated cost system and disallow CEL expressions from being included in CRDs if they have a sufficiently poor (high) estimated cost. The estimated cost limit is set quite high and typically requires an O(n2) or worse operation, across something of unbounded size, to be exceeded. Fortunately the fix is usually quite simple: because the cost system is aware of size limits declared in the CRD's schema, CRD authors can add size limits to the CRD's schema (maxItems for arrays, maxProperties for maps, maxLength for strings) to reduce the estimated cost.

Good practice:

Set maxItems, maxProperties and maxLength on all array, map (object with additionalProperties) and string types in CRD schemas! This results in lower and more accurate estimated costs and generally makes a CRD safer to use.


Runtime cost limits for CRD validation rules

In addition to the estimated cost limit, CEL keeps track of actual cost while evaluating a CEL expression and will halt execution of the expression if a limit is exceeded.

With the estimated cost limit already in place, the runtime cost limit is rarely encountered. But it is possible. For example, it might be encountered for a large resource composed entirely of a single large list and a validation rule that is either evaluated on each element in the list, or traverses the entire list.

CRD authors can ensure the runtime cost limit will not be exceeded in much the same way the estimated cost limit is avoided: by setting maxItems, maxProperties and maxLength on array, map and string types.


Adoption and Related work

This feature has been turned on by default since Kubernetes 1.25 and finally graduated to GA in Kubernetes 1.29. It raised a lot of interest and is now widely used in the Kubernetes ecosystem. We are excited to share that the Gateway API was able to replace all the validating webhook previously used with this feature.

After CEL was introduced into Kubernetes, we are excited to expand the power to multiple areas including the Admission Chain and authorization config. We will have a separate blog to introduce further.

We look forward to working with the community on the adoption of CRD Validation Rules, and hope to see this feature promoted to general availability in upcoming Kubernetes releases.


Acknowledgements

Special thanks to Joe Betz, Kermit Alexander, Ben Luddy, Jordan Liggitt, David Eads, Daniel Smith, Dr. Stefan Schimanski, Leila Jalali and everyone who contributed to CRD Validation Rules!

By Cici Huang – Software Engineer

.