How Google’s 2020 summer interns became the newest contributors in open source

Our internship program changed in structure this year to accommodate a virtual environment, and we enjoyed seeing the intern involvement in our open source teams. Now, as the Summer 2020 Interns have departed Google, we’ve seen widespread impact across these OSS projects. Some accomplishments from the intern community included:

• Mohamed Ibrahim, a Software Engineering major at the University of Ontario Institute of Technology, interned on the Earth Engine team in Geo. He built a web app from scratch that allows Earth Engine developers, who are primarily climate and remote-sensing researchers, to build rich UIs for their Earth Engine Apps without needing to write any code. Mohamed also learned two coding languages unfamiliar to him, enabling him to write over 10,000 lines of TypeScript, 480 lines of Go, and merge over 30 PRs during one internship.

Web app demo

• Vismita Uppalli, a Cloud intern and Computer Science major at the University of Virginia, wrote a tutorial showing how to use AI Platform Operators on Apache Airflow, which is now published in the official Airflow docs.
• Colin Marsch interned with the Android team and published a blog post for Android developers, "Re-writing the AOSP DeskClock App in Kotlin," which has reached over 1,600 viewers! He is scheduled to graduate from the University of Waterloo with a major in Computer Science in Spring 2021.
• Satyam Ralhan worked in the MyHeart team in Research to build a first-of-its-kind Android app that engages users in conversations to encourage healthy habits. He created a demo, which explores the different phases of the app and how it learns to personalize lifestyle suggestions for various kinds of users. He is in his fourth year at the Indian Institute of Technology, Kanpur, studying Computer Science and Engineering.

  

  MyHeart app demo
  

• An Apigee intern, Nicole Gizzo, presented her work analyzing API vocabularies at the API Specifications Conference. She is majoring in Computer Science and Cognitive Science at Rensselaer Polytechnic Institute, and will graduate in May 2021.
• The OSS Fuzzing Interns have found and reported over 600 bugs to critical open source projects like the Linux kernel and Nginx, over 100 of which were security vulnerabilities.
• Madelyn Dubuk, a SWE Intern on the Cloud DPE team and a Computer Science major at USC, worked with three other interns to create a full stack web app to help better understand test flakiness, and enjoyed working directly with other interns.

Initial feedback from our interns indicates that their OSS contributions won’t stop when their internships end. Of the interns who worked on OSS projects, 69% plan to continue contributing to OSS, enjoying the ability to talk about their work and have a broader impact. Beyond the impact on OSS, we’ve seen tremendous professional growth for our interns. Lucia Cantu-Miller, an intern on the Chrome team and Computer Science major at ITESM Monterrey, reflected she was, “proud of seeing how I’ve grown during the internship. As the days passed I became more confident in my work and in asking questions, I have grown a lot as a person and as a professional student.” Lucia wasn’t the only intern to experience this as 98% of interns who worked on OSS feel that Google is a good place to start a career. The success of this summer’s Internship is due in large part to the many contributions of Google’s OSS community—from the intern hosts to the project champions and mentors—we can’t thank them enough for their support. 

By Emma Stamp, Google Engineering Education

Kubernetes: Efficient Multi-Zone Networking with Topology Aware Routing

Topology Aware Routing of Services, a feature that was first introduced as alpha in the Kubernetes 1.17 release, aims to solve an often overlooked issue with Kubernetes Services; that they are not region aware.

Kubernetes services provide a uniform, durable, and easy to use method of accessing a variety of different backend applications. These backend applications are most commonly an exposed app running within your pods. Kubernetes does this by reserving a static virtual IP and DNS name, unique to it throughout the cluster and turning them into simple load balancers.

While this model is great for small clusters or applications, if you have thousands of nodes, your cluster spans multiple regions, or your application is latency sensitive then the service model can start to break down a bit. By default, each endpoint in a service has an equal opportunity to be selected as the destination. If you’re accessing a service with a backend hosted in the same zone, there’s a high probability that you’d be directed to a pod in a completely separate zone—likely in a completely separate region—and is what Topology Awareness intends to solve.

The Topology Aware Routing of Services feature added the concept of topologyKeys as an additional field in service objects. It allows you to define a set of node labels that could be used to route traffic closer to where it originated from.

Example Service with topologyKeys

apiVersion: v1 kind: Service metadata:   Name: my-app-web spec:   selector:     app: my-app   ports:     - protocol: TCP       port: 80       targetPort: 8080   topologyKeys:     - "topology.kubernetes.io/zone"     - "topology.kubernetes.io/region"

In this example, the service makes use of some commonly used labels for its topology preferences. It signals that when kube-proxy is routing traffic for that service, it should only route to pods within the same zone or region the traffic is originating from. This is great! Traffic should remain “close” to where it originated and remove unnecessary latency. While topologyKeys is available as alpha in 1.17, it hasn’t yet graduated to the next stage because the first pass at building topology-aware routing surfaced many challenges and scalability issues.

Each node in the cluster now has to manage a potentially complex ruleset for each service that would require more frequent updating. In clusters with thousands of pods or thousands of nodes, this solution quickly becomes untenable.

Another pain point with this implementation depends on how your application was distributed across a zone or region, as it's quite possible that a singular pod would be receiving ALL traffic for that zone or region. The preference list doesn’t take into account the performance of the pod on the receiving end and could potentially cause an outage.

These problems have led the Kubernetes Network Special Interest Group (SIG) to do a full re-evaluation of how to approach the Topology Awareness implementation.

What’s Planned for Topology Aware Routing?

The new design is intended to automatically handle the routing of services so that they will be load-balanced across a minimum number of the closest possible endpoints. It does this by applying an algorithm using two of the topology keys to signal affinity for service routing: topology.kubernetes.io/region and topology.kubernetes.io/zone without having to specify them via topologyKeys at the service level.

This algorithm works by establishing a dynamic threshold for a service where it calculates an expected number of endpoints per zone. It then builds a list of available endpoints for that service, prioritizing the ones that are in the same zone. If there are not enough endpoints to meet that expected number, it adds them from other zones until it reaches its expected number of endpoints. This list of expected endpoints, or a subset of endpoints are then passed to the nodes within that zone.

These nodes no longer have to maintain the complex set of rules like they had in the first iteration, and now just manage the small subset of endpoints for each service. This is less flexible than its predecessor, but it drastically reduces the performance overhead when compared to the previous method, while also covering the majority of use-cases. A big win for everyone.

These features are slated to graduate to alpha in the 1.21 release in the first part of 2021. If Topology Aware Routing would be of value to you, please consider taking the time to test it when it becomes available. Early feedback is highly appreciated and helps shape the direction of the feature.

Until then, if you’d like to learn more about Service Topology, Endpoint Slice, and the various algorithms that have been evaluated for service routing, check out Rob Scott’s presentation: Improving Network Efficiency with Topology Aware Routing, on November 19th, at KubeCon + CloudNativeCon North America.



By Bob Killen, Program Manager – Google Open Source Programs Office

Welcome Android Open Source Project (AOSP) to the Bazel ecosystem

After significant investment in understanding how best to build the Android Platform correctly and quickly, we are pleased to announce that the Android Platform is migrating from its current build systems (Soong and Make) to Bazel. While components of Bazel have been already checked into the Android Open Source Project (AOSP) source tree, this will be a phased migration over the next few Android releases which includes many concrete and digestible milestones to make the transformation as seamless and easy as possible. There will be no immediate impact to the Android Platform build workflow or the existing supported Android Platform Build tools in 2020 or 2021. Some of the changesto support Android Platform builds are already in Bazel, such as Bazel’s ability to parse and execute Ninja files to support a gradual migration.

Migrating to Bazel will enable AOSP to:

• Provide more flexibility for configuring the AOSP build (better support for conditionals)
• Allow for greater introspection into the AOSP build progress and dependencies
• Enable correct and reproducible (hermetic) AOSP builds
• Introduce a configuration mechanism that will reduce complexity of AOSP builds
• Allow for greater integration of build and test activities
• Combine all of these to drive significant improvements in build time and experience

The benefits of this migration to the Bazel community are:

• Significant ongoing investment in Bazel to support Android Platform builds
• Expansion of the Bazel ecosystem and community to include, initially, tens of thousands of Android Platform developers and Android handset OEMs and chipset vendors.
• Google’s Bazel rules for building Android apps will be open sourced, used in AOSP, and maintained by Google in partnership with the Android / Bazel community
• Better Bazel support for building Android Apps
• Better rules support for other languages used to build Android Platform (Rust, Java, Python, Go, etc)
• Strong support for Bazel Long Term Support (LTS) releases, which benefits the expanded Bazel community
• Improved documentation (tutorials and reference)

The recent check-in of Bazel to AOSP begins an initial pilot phase, enabling Bazel to be used in place of Ninja as the execution engine to build AOSP. Bazel can also explore the AOSP build graph. We're pleased to be developing this functionality directly in the Bazel and AOSP codebases. As with most initial development efforts, this work is experimental in nature. Remember to use the currently supported Android Platform Build System for all production work.

We believe that these updates to the Android Platform Build System enable greater developer velocity, productivity, and happiness across the entire Android Platform ecosystem.

By Joe Hicks on behalf of the Bazel and AOSP infrastructure teams

Get ready for BazelCon 2020

With only 24 hours to go, BazelCon 2020 is shaping up to be a much anticipated gathering for the Bazel community and broader Build ecosystem. With over 1000 attendees, presentations by Googlers, as well as talks from industry Build leaders from Twitter, Dropbox, Uber, Pinterest, GrabTaxi, and more, we hope BazelCon 2020 will provide an opportunity for knowledge sharing, networking, and community building.

I am very excited by the keynote announcements, the migration stories at Twitter, Pinterest, and CarGurus, as well as technical deep dives on Bazel persistent workers, incompatible target skipping, migrating from Gradle to Bazel, and more. The “sold out” Birds of a Feather sessions and the Live Q&A with the Bazel team will bring the community together to discuss design docs, look at landings, and provide feedback on the direction of Bazel and the entire ecosystem.

We are also pleased to announce that, starting with the next major release (4.0), Bazel will support Long Term Support (LTS) releases as well as regular Rolling releases.

Some benefits of this new release cadence are:

• Bazel will release stable, supported LTS releases on a predictable schedule with a long window without breaking changes
• Bazel contributors / rules owners can prepare to support future LTS releases via rolling releases.
• Bazel users can choose the release cadence that works best for them, since we will offer both LTS releases and rolling releases.

Long Term Support (LTS) releases:

• We will create an LTS release every ~9 months => new LTS release branch, increment major version number.
• Each LTS release will include all new features, bug fixes and (breaking) changes since the last major version.
• Bazel will actively support each LTS branch for 9 months with critical bug fixes, but no new features.
• Thereafter, Bazel will provide maintenance for two additional years with only security and OS compatibility fixes.
• Bazel Federation reboot: Bazel will provide strong guidance about the ruleset versions that should be used with each Bazel release so that each user will not have to manage interoperability themselves.

Make sure that you register at http://goo.gle/bazelcon to be a part of the excitement of the premier build conference!

See you all at BazelCon 2020!

By Joe Hicks and the entire Bazel Team at Google

Google’s initiative for more inclusive language in open source projects

Certain terms in open source projects reinforce negative associations and unconscious biases. At Google, we want our language to be inclusive. The Google Open Source Programs Office (OSPO) created and posted a policy for new Google-run projects to remove the terms “slave,” “whitelist,” and “blacklist,” and replace them with more inclusive alternatives, such as “replica,” “allowlist,” and “blocklist.” OSPO required that new projects follow this policy beginning October 2020, and has plans to enforce these changes on more complex, established projects beginning in 2021. 

To ensure this policy was implemented in a timely manner, a small team within OSPO and Developer Relations orchestrated tool and policy updates and an open-source specific fix-it, a virtual event where Google engineers dedicate time to fixing a project. The fix-it focused on existing projects and non-breaking changes, but also served as a reminder that inclusivity is an important part of our daily work. Now that the original fix-it is over, the policy remains and the projects continue.

For more information on why inclusive language matters to us, you can check out Google Developer Documentation Style Guide which contains a section on word-choice with useful, clearer alternatives. Regardless of the phrases used, it is necessary to understand that certain terms reinforce biases and that replacing them is a positive step, both in creating a more welcoming atmosphere for everyone and in being more technically accurate. In short, words matter.


By Erin Balabanian, Open Source Compliance.

Security scorecards for open source projects

When developers or organizations introduce a new open source dependency into their production software, there’s no easy indication of how secure that package is.

Some organizations—including Google—have systems and processes in place that engineers must follow when introducing a new open source dependency, but that process can be tedious, manual, and error-prone. Furthermore, many of these projects and developers are resource constrained and security often ends up a low priority on the task list. This leads to critical projects not following good security best practices and becoming vulnerable to exploits. These issues are what inspired us to work on a new project called “Scorecards” announced last week by the Open Source Security Foundation (OpenSSF). 

Scorecards is one of the first projects being released under the OpenSSF since its inception in August, 2020. The goal of the Scorecards project is to auto-generate a “security score” for open source projects to help users as they decide the trust, risk, and security posture for their use case. Scorecards defines an initial evaluation criteria that will be used to generate a scorecard for an open source project in a fully automated way. Every scorecard check is actionable. Some of the evaluation metrics used include a well-defined security policy, code review process, and continuous test coverage with fuzzing and static code analysis tools. A boolean is returned as well as a confidence score for each security check. Over time, Google will be improving upon these metrics with community contributions through the OpenSSF.

Check out the Security Scorecards project on GitHub and provide feedback. This is just the first step of many, and we look forward to continuing to improve open source security with the community.

By Kim Lewandowski, Dan Lorenc, and Abhishek Arya, Google Security team