Kpt: Packaging up your Kubernetes configuration with git and YAML since 2014

Kubernetes configuration manifests have become an industry standard for deploying both custom and off-the-shelf applications (as well as for infrastructure). Manifests are combined into bundles to create higher-level deployable systems as well as reusable blueprints (such as a product offering, off the shelf software, or customizable starting point for a new application).

However, most teams lack the expertise or desire to create bespoke bundles of configuration from scratch and instead: 1) either fork them from another bundle, or 2) use some packaging solution which generates manifests from code.

Teams quickly discover they need to customize, validate, audit and re-publish their forked/ generated bundles for their environment. Most packaging solutions to date are tightly coupled to some format written as code (e.g. templates, DSLs, etc). This introduces a number of challenges when trying to extend, build on top of, or integrate them with other systems. For example, how does one update a forked template from upstream, or how does one apply custom validation?

Packaging is the foundation of building reusable components, but it also incurs a productivity tax on the users of those components.

Today we’d like to introduce kpt, an OSS tool for Kubernetes packaging, which uses a standard format to bundle, publish, customize, update, and apply configuration manifests.

Kpt is built around an “as data” architecture bundling Kubernetes resource configuration, a format for both humans and machines. The ability for tools to read and write the package contents using standardized data structures enables powerful new capabilities:

• Any existing directory in a Git repo with configuration files can be used as a kpt package.
• $ kpt pkg get
• Packages can be arbitrarily customized and later pull in updates from upstream by merging them.
• $ kpt pkg update
• Tools and automation can perform high-level operations by transforming and validating package data on behalf of users or systems.
• $ kpt cfg set
• Organizations can develop their own tools and automation which operate against the package data.
• $ kpt fn
• Existing tools and automation that work with resource configuration “just work” with kpt.
• $ kustomize build & kubectl apply
• Existing solutions that generate configuration (e.g. from templates or DSLs) can emit kpt packages which enable the above capabilities for them.
• $ helm template

Example workflow with kpt

Now that we’ve established the benefits of using kpt for managing your packages of Kubernetes config, lets walk through how an enterprise might leverage kpt to package, share and use their best practices for Kubernetes across the organization.


First, a team within the organization may build and contribute to a repository of best practices (pictured in blue) for managing a certain type of application, for example a microservice (called “app”). As the best practices are developed within an organization, downstream teams will want to consume and modify configuration blueprints based on them. These blueprints provide a blessed starting point which adheres to organization policies and conventions.

The downstream team will get their own copy of a package by downloading it to their local filesystem (pictured in red) using kpt pkg get. This clones the git subdirectory, recording upstream metadata so that it can be updated later.

They may decide to update the number of replicas to fit their scaling requirements or may need to alter part of the image field to be the image name for their app. They can directly modify the configuration using a text editor (as would be done before). Alternatively, the package may define setters, allowing fields to be set programmatically using kpt cfg set. Setters streamline workflows by providing user and automation friendly commands to perform common operations.

Once the modifications have been made to the local filesystem, the team will commit and push their package to an app repository owned by them. From there, a CI/CD pipeline will kick off and the deployment process will begin. As a final customization before the package is deployed to the cluster, the CI/CD pipeline will inject the digest of the image it just built into the image field (using kpt cfg set). When the image digest has been set, the CI/CD pipeline can send the manifests to the cluster using kpt live apply. Kpt live operates like kubectl apply, providing additional functionality to prune resources deleted from the configuration and block on rollout completion (reporting status of the rollout back to the user).

Now that we’ve walked through how you might use kpt in your organization, we’d love it if you’d try it out, read the docs, or contribute.

One more thing

There’s still a lot to the story we didn’t cover here. Expect to hear more from us about:

• Using kpt with GitOps
• Building custom logic with functions
• Writing effective blueprints with kpt and kustomize

By Phillip Wittrock, Software Engineer and Vic Iglesias, Cloud Solutions Architect

OpenTelemetry is now beta!

OpenTelemetry and OpenCensus have been a critical part of our goal of making platforms like Kubernetes more observable and more manageable. This has been a multi-year journey for us, from creating OpenCensus and growing it into a core part of major web services’ observability stack, to our announcement of OpenTelemetry last year and the rapid growth of the OpenTelemetry community.

Beta is a big milestone for OpenTelemetry, as developers can now use the SDKs, integrations, and Collector to capture distributed traces and metrics from their applications and send them to backends like Prometheus, Jaeger, Cloud Monitoring, Cloud Trace, and others for analysis. This is a great time to try out OpenTelemetry and get involved in the observability community— whether you’re looking to improve your visibility into production services, giving your users performance data from client libraries that you maintain—or want to join a rapidly-growing open source project!

To learn more, please read our official community announcement, which copied below:

Co-authored by maintainers, community contributors, and members of the OpenTelemetry governance committee.

OpenTelemetry has just begun its first wave of beta releases, starting with the Collector and the Erlang, Go, Java, JavaScript, and Python SDKs, followed by the .Net SDK and Java auto-instrumentation agent. This means that you can begin integrating OpenTelemetry into your applications and client libraries to capture app-level metrics and distributed traces.

If you’re not already familiar with OpenTelemetry, the project provides a single set of language-specific APIs, SDKs, agents, and other components that you can use to collect distributed traces, metrics, and related metadata from your applications. In addition to its core capabilities, much of OpenTelemetry’s utility comes from integrations for HTTP and RPC libraries, storage clients, etc. that allow developers to capture critical observability data from their applications with almost zero effort. After capturing these signals, each OpenTelemetry component can export them to your backends of choice, including Prometheus, Jaeger, Zipkin, Azure Monitor, Dynatrace, Google Cloud Monitoring + Trace, Honeycomb, Lightstep, New Relic, and Splunk.

This first beta release includes:

• APIs and SDKs for Erlang, Go, Java, JavaScript, and Python, which include the interfaces and implementations that you need to define and create distributed traces and metrics, manage sampling and context propagation, etc. The .Net API + SDK will follow shortly.
• Language-specific API integrations for at least one popular HTTP framework, gRPC, and at least one popular storage client, which can be enabled with one line of code, and will automatically capture relevant traces and metrics and handle context propagation.
• Language-specific exporters that allow SDKs to send captured traces and metrics to any supported backends.
• The OpenTelemetry Collector, which can receive data from OpenTelemetry SDKs and other sources, and then export this telemetry to any supported backend.
• Auto-Instrumentation for Java that captures telemetry from 47 Java libraries and frameworks without requiring any modification to your application.
• Documentation for each component including getting started guides.

As these and subsequent OpenTelemetry components enter beta (requirements and release plan), we are declaring that they are ready to start integrating with. This means that service developers can begin to include OpenTelemetry in their applications and that maintainers of storage, RPC, etc. clients should start testing the OpenTelemetry APIs to provide better observability of their users.

However, this does come with some caveats:

• Each OpenTelemetry component will likely undergo several beta releases in the coming weeks — this is simply the first.
• While functional, beta components have not gone through thorough testing or benchmarking and they are not intended for production workloads.
• While we aim to avoid any major changes to the OpenTelemetry APIs between beta and GA release candidates, we cannot guarantee that there will not be any changes during this period.
• Some functionality is still missing from the first beta and will be added in subsequent releases; this is documented in each component’s GitHub repository.

In the coming weeks, you can expect additional beta releases from the first wave of OpenTelemetry components and others. In particular, we expect the API + SDK for .Net and the Java auto-instrumentation agent to be ready soon. Eventually, components will reach a level of maturity and testing where we’ll feel confident in naming them a release candidate (RC), after which we will not make any breaking changes to the APIs for that component.

This beta milestone is a huge accomplishment for the OpenTelemetry community, and every contributor should be proud of the fact that OpenTelemetry is now working and ready to integrate with. This is a great opportunity for the maintainers of client libraries to begin integrating with the OpenTelemetry APIs, for end-users to start integrating it into their services, and for anyone interested in contributing to join our rapidly growing community by joining our mailing lists, Gitter chats, and the monthly community meeting!

By Morgan McLean, Product Manager

Semantic Reactor: A tool for experimenting with NLU models

Companies are using natural language understanding (NLU) to create digital personal assistants, customer service bots, and semantic search engines for reviews, forums and the news.

However, the perception that using NLU and machine learning is costly and time consuming prevents a lot of potential users from exploring its benefits.

To dispel some of the intimidation of using NLU, and to demonstrate how it can be easily used with pre-trained, generic models, we have released a tool, the Semantic Reactor, and open-sourced example code, The Mystery of the Three Bots.

The Semantic Reactor

The Semantic Reactor is a Google Sheets Add-On that allows the user to sort lines of text in a sheet using a variety of machine-learning models. It is released as a whitelisted experiment, so if you would like to check it out, fill out this application at the Google Cloud AI Workshop. Once approved, you’ll be emailed instructions on how to install it.

The tool offers ranking methods that determine how the list will be sorted. With the semantic similarity method, the lines more similar in meaning to the input will be ranked higher.



With the input-response method, the lines that are the most appropriate conversational responses are ranked higher.

Why use the Semantic Reactor?

There are a lot of interesting things you can do with the Semantic Reactor, but let’s look at the following two:

• Writing dialogue for a bot that exists within a well-defined environment and has a clear purpose (like a customer service bot) using semantic similarity.
• Searching within large collections of text, like from a message board. For that, we will use input-response.

Writing Dialogue for a Bot Using Semantic Similarity

For the sake of an example, let’s say you are writing dialogue for a bot that answers questions about a product, in this case, cookies.

If you’ve been running a cookie hotline for a while, you probably can list the most common cookie questions. With that data, you can create your cookie bot. Start by opening a Google Sheet and writing the common questions and answers (questions in the A column, answers in the B).

Here is the start of what that Sheet might look like. Make a copy of the Sheet, which will allow you to use the Semantic Reactor Add-on. Use the tool to experiment with new QA pairs and how each model reacts to them.

Here are a few queries to try, using the semantic similarity rank method:

Query: What are cookie ingredients?
Returns: What are cookies made of?

Query: Are cookies biscuits?
Returns: Are cookies also called biscuits?

Query: What should I serve with cookies?
Returns: What drinks go well with cookies?



Of course, that small list of responses won’t cover many of the questions people will ask your cookie bot. What the Reactor allows you to do is quickly add new QA pairs as you learn about what your users want to ask.

For example, maybe people are asking a lot about cookie calories.

You’d write the new question in column A, and the new answer in column B, and then test a few different phrasings with the Reactor. You might need to tweak the target response a few times to make sure it matches a wide variety of phrasings. You should also experiment with the three different models to see which one performs the best.

For instance, let’s say the new target question you want the model to match to is: “How many calories does a typical cookie have?”

That question might be phrased by users as:

• Are cookies caloric?
• A lot of calories in a cookie?
• Will cookies wreck my diet?
• Are cookies fattening?

The more you test with live users, the more you’ll find that they phrase their questions in ways you don’t expect. As with all things based on machine learning, constantly refreshing data, testing and improvement is all part of the process.

Searching Through Text Using Input-Response

Sometimes you can’t anticipate what users are going to ask, and sometimes you might be dealing with a lot of potential responses, maybe thousands. In cases like that, you should use the input-response ranking method. That means the model will examine the list of potential responses and then rank each one according to what it thinks is the most likely response.

Here is a Sheet containing a list of simple conversational responses. Using the input-response ranking method, try a few generic conversational openers like “Hello” or “How’s it going?”

Note that in input-response mode, the model is predicting the most likely conversational response to an input and not the most semantically similar response.

Note that “Hello,” in input-response mode, returns “Nice to meet you.” In semantic similarity mode, “Hello” returns what the model thinks is semantically closest to “Hello,” which is “What’s up?”

Now try your own! Add potential responses. Switch between the models and ranking methods to see how it changes the results (be sure to hit the “reload” button every time you add new responses).

Example Code

One of the models available on TensorFlow Hub is the Universal Sentence Encoder Lite. It’s only 1.6MB and is suitable for use within websites and on-device applications.

An open sourced sample game that uses the USE Lite is Mystery of the Three Bots on Github. It’s a simple demonstration that shows how you can use a small semantic ML model to drive conversations with game characters. The corpora the game uses were created and tested using the Semantic Reactor.

You can play a running version of the game here. You can experiment with the corpora of two of the characters, the Maid and the Butler, contained within this Sheet. Be sure to make a copy of the Sheet so you can edit and add new QA pairs.

Where To Get The Models Used Within The Semantic Reactor

All of the models used in the Semantic Reactor are published and available online.

• Local – Minified TensorFlow.js version of the Universal Sentence Encoder.
• Basic Online – Basic version of the Universal Sentence Encoder.
• Multilingual Online – Universal Sentence Encoder trained on question/ answer pairs in 16 languages.

Final Thoughts

These language models are far from perfect. They use their training to give a best estimate on what to return based on the list of responses you gave it. Machine learning is about calculation, prediction, and training. Models can be improved over time with more data and tuning, and in turn, be made more accurate.

Also, because conversational models are trained on dialogue between people, and because people are biased, the models will display biases that exist in the data that they were trained on, sometimes in ways you can’t predict. For more on model bias, and more detail about how these models were trained, see the Semantic Experiences for Developers page.

By Ben Pietrzak, Steve Pucci, Aaron Cohen — Google AI  

A Season of Docs story

Lack of clear and reliable documentation is one of the main shortcomings of many open source projects. Last year, Google set out to help change that by announcing the first ever Season of Docs.

Season of Docs is an initiative that brings together technical writers and open source projects to collaborate for a few months, benefitting both the communities and writers.

This is the story of Audrey Tavares, one of the writers who signed up for Season of Docs.

Turning incipient curiosity into an opportunity

In 2019, Audrey was completing the Technical and Professional Communication program at Glendon College, exploring technical writing out of curiosity. One of Google’s technical writers, Nicola Yap, completed the same program and visited Audrey’s class in March to talk about her career. It was an enlightening experience, showing technical writing as an attractive alternative with plenty of opportunities, and introducing Audrey to Season of Docs.

For Audrey, this experience meant stepping into unknown territory—she knew nothing about open source software. Naturally, the first step was to familiarize herself with the communities and understand the software development paradigm. After spending time learning she submitted her Technical Writer application—which was accepted—and was assigned to Oppia, an online educational platform.

Main challenges

Audrey had two mentors to help her on her journey: one in India and the other in the United States. As you can imagine, this revealed the first challenge—time zones. While the first few days were stressful, as navigating schedules across time zones was a daunting task,with a little work, they soon came up with an arrangement that worked for everyone.

The second challenge was learning the tools. For most of us, writing a document involves opening a word processor and typing some text, however, Audrey was about to find out, things are a bit more intricate when it comes to documenting code.

When presented with the choice of a documentation tool set, Audrey decided on Read the Docs. It seemed like a very popular tool among open source communities. How hard can it be to use, right? Well, it’s not so much about how difficult it is, but how different it is for someone unfamiliar with a common software development workflow since it entails learning a few things:

• Version control – tools to track changes in files.
• Markup languages – a way to write docs in plain text.
• A static site generator – a tool that turns the markup files into a beautiful website.
• The command line – an interactive program to issue commands to a computer, like in the good old days.

Audrey was not dismayed. She pushed forward and gradually learned these new tools. Both mentors were always available, willing to help, and answered all of her questions. Their mentorship was key to her success.

Every end is a new beginning

After Season of Docs was over, Audrey decided to remain part of the Oppia community to actively contribute to make the platform even better.

The experience allowed Audrey to walk away from Season of Docs with a new set of technical skills, communication skills with software engineers, an extended professional network, and a new item in her résumé. She now works as a technical writer for a software company in Toronto.

Applications for Season of Docs 2020 start on April 13 for open source organizations and on May 11 for technical writers. Check the official announcement to learn how to participate.

By Geri Ochoa, Google Cloud

Announcing Season of Docs 2020

Google Open Source is delighted to announce Season of Docs 2020!

Season of Docs brings technical writers and open source projects together for a few months to work on open source documentation. 2019 was the first year of Season of Docs, bringing together open source organizations and technical writers to create 44 successful documentation projects!

Docs are key to open source success

Survey after survey show the importance of good documentation in how developers choose and use open source:

• 72% of surveyed developers say “Established policies and documentation” is a key decision factor when choosing open source
• 93% of surveyed developers say “Incomplete or outdated documentation is a pervasive problem” in open source
• “Lack of documentation” was the top reason developers gave for deciding against using an open source project

Open source communities know this, and still struggle to produce good documentation. Why? Because creating documentation is hard. But...

There are people who know how to do docs well. Technical writers know how to structure a documentation site so that people can find and understand the content. They know how to write docs that fit the needs of their audience. Technical writers can also help optimize a community’s processes for open source contribution and onboarding new contributors.

Season of Docs brings open source projects and technical writers together with the shared goal of creating great documentation. The writers bring their expertise to the projects, and the project mentors help the technical writers learn more about open source and new technologies. Communities gain new docs contributors and technical writers gain valuable open source skills.

Together the technical writers and mentors build a new doc set, improve the structure of the existing docs, develop a much-needed tutorial, or improve contribution processes and guides. See more ideas for technical writing projects.

By working together in Season of Docs we raise awareness of open source, docs, and technical writing.

How does it work?

April 13 – May 4	Open source organizations apply to take part in Season of Docs
May 11	Google publishes the list of accepted mentoring organizations, along with their ideas for documentation projects
May 11 – July 9	Technical writers choose the project they’d like to work on and submit their proposals to Season of Docs
August 10	Google announces the accepted technical writer projects
August 11 – September 11	Community bonding: Technical writers get to know mentors and the open source community, and refine their projects in collaboration with their mentors
September 11 – December 6	Technical writers work with open source mentors on the accepted projects, and submit their work at the end of the period
January 7, 2021	Google publishes the list of successfully-completed projects.

See the timeline for details, including the provision for projects that run longer than three months.

Join us

Explore the Season of Docs website at g.co/seasonofdocs to learn more about participating in the program. Use our logo and other promotional resources to spread the word. Check out the timeline and FAQ, and get ready to apply!

By Erin McKean, Google Open Source

Google and Binomial partner to open source high quality basis universal

Today, Google and Binomial are excited to announce the high quality update to the original Basis Universal release.

Basis Universal allows you to have state of the art web performance with your images, keeping images compressed even on the GPU. Older systems like JPEG and PNG may look small in storage size, but once they hit the GPU they are processed as uncompressed data! The original Basis Universal codec created images that were 6-8 times smaller than JPEG on the GPU while maintaining a similar storage size.

Today we release a high quality Basis Universal codec that utilizes the highest quality formats modern GPUs support, finally bringing the web up to modern GPU texture standards—with cross platform support. The textures are larger in storage size and GPU compressed size, but are still 3-4 times smaller than sending a JPEG or PNG file to be processed on the GPU, and can transcode to a lower quality format for older GPUs.

Original Image by Erol Ahmed from Unsplash.com

Visual comparison of Basis Universal High Quality

Best of all, we are actively working on standardizing Basis Universal with the Khronos Group.

Since our original release in Summer 2019 we’ve seen widespread adoption of Basis Universal in engines like three.js, Babylon.js, Godot, and more, changing what is possible for people to create on the web. Now that a high quality option is available, we expect to see even more adoption and groundbreaking applications created with it.

Please feel free to join our community on Github and check out the full demo there as well. You can also follow standardization efforts via Khronos Group events and forums.

By Stephanie Hurlburt, Binomial and Jamieson Brettle, Chrome Media

Pigweed: A collection of embedded libraries

We’re excited to announce Pigweed, an open source collection of embedded-targeted libraries, or as we like to call them, modules. Pigweed modules are built to enable faster and more reliable development on 32-bit microcontrollers.

Pigweed is in early development and is not suitable for production use at this time.

Getting Started with Pigweed

As of today, the source is available for everyone at pigweed.googlesource.com under an Apache 2.0 license. Instructions on getting up and running can be found in the README.

See Pigweed in action

Pigweed offers modules that address a wide range of embedded developer needs. These highlight how Pigweed can accelerate embedded development across the entire lifecycle:

• Setup: Get started faster with simplified setup via a virtual environment with all required tools
• Development: Accelerated edit-compile-flash-test cycles with watchers and distributed testing
• Code Submission: Pre-configured code formatting and integrated presubmit checks

Setup

A classic challenge in the embedded space is reducing the time from running git clone to having a binary executing on a device. Oftentimes, an entire suite of tools is needed for non-trivial production embedded projects. For example, your project likely needs:

• A C++ compiler for your target device, and also for your host
• A build system (or three); for example, GN, Ninja, CMake, Bazel
• A code formatting program like clang-format
• A debugger like OpenOCD to flash and debug your embedded device
• A known Python version with known modules installed for scripting
• … and so on

Below is a system with Python 2.7, clang-format, and no ARM compiler. The bootstrap script in Pigweed’s pw_env_setup module, sets up the current shell to have access to a standardized set of tools—Python 3.8, clang-format, and an ARM compiler among them. All of this is done in a virtual environment so the system’s default environment remains unmodified.

Development

In typical embedded development, adding even a small change involves the following additional manual steps:

• Re-building the image
• Flashing the image to a device
• Ensuring that the change works as expected
• Verifying that existing tests continue to pass

This is a huge disparity from web development workflows where file watchers are prevalent—you save a file and instantly see the results of the change.

Pigweed’s pw_watch module solves this inefficiency directly, providing a watcher that automatically invokes a build when a file is saved, and also runs the specific tests affected by the code changes. This drastically reduces the edit-compile-flash-test cycle for changes.

In the demo above, the pw_watch module (on the right) does the following:

• Detects source file changes
• Builds the affected libraries, tests, and binaries
• Flashes the tests to the device (in this case a STM32F429i Discovery board)
• Runs the specific unit tests

There’s no need to leave your code editor—all of this is done automatically. You can save additional time by using the pw_target_runner module to run tests in parallel across multiple devices.

Code submission

When developing code as a part of a team, consistent code is an important part of a healthy codebase. However, setting up linters, configuring code formatting, and adding automated presubmit checks is work that often gets delayed indefinitely.

Pigweed’s pw_presubmit module provides an off-the-shelf integrated suite of linters, based on tools that you’ve probably already used, that are pre-configured for immediate use for microcontroller developers. This means that your project can have linting and automatic formatting presubmit checks from its inception.

Other modules

There are many modules in addition to the ones highlighted above:

• pw_tokenizer – A module that converts strings to binary tokens at compile time. This enables logging with dramatically less overhead in flash, RAM, and CPU usage.
• pw_string – Provides the flexibility, ease-of-use, and safety of C++-style string manipulation, but with no dynamic memory allocation and a much smaller binary size impact. Using pw_string in place of the standard C functions eliminates issues related to buffer overflow or missing null terminators.
• pw_bloat – A module to generate memory reports for output binaries empowering developers with information regarding the memory impact of any change.
• pw_unit_test – Unit testing is important and Pigweed offers a portable library that’s broadly compatible with Google Test. Unlike Google Test, pw_unit_test is built on top of embedded friendly primitives; for example, it does not use dynamic memory allocation. Additionally, it is to port to new target platforms by implementing the test event handler interface.
• pw_kvs – A key-value-store implementation for flash-backed persistent storage with integrated wear levelling. This is a lightweight alternative to a file system for embedded devices.
• pw_cpu_exception_armv7m – Robust low level hardware fault handler for ARM Cortex-M; the handler even has unit tests written in assembly to verify nested-hardware-fault handling!
• pw_protobuf – An early preview of our wire-format-oriented protocol buffer implementation. This protobuf compiler makes a different set of implementation tradeoffs than the most popular protocol buffer library in this space, nanopb.

Why name the project Pigweed?

Pigweed, also known as amaranth, is a nutritious grain and leafy salad green that is also a rapidly growing weed. When developing the project that eventually became Pigweed, we wanted to find a name that was fun, playful, and reflective of how we saw Pigweed growing. Teams would start out using one module that catches their eye, and after that goes well, they’d quickly start using more.

So far, so good 😁

What’s next?

We’re continuing to evolve the collection and add new modules. It’s our hope that others in the embedded community find these modules helpful for their projects.

By Keir Mierle and Mohammed Habibulla, on behalf of the Pigweed team

Google Summer of Code 2020 now open for student applications!

If you’re a university student and want to sharpen your software development skills while doing good for the open source community, check out Google Summer of Code (GSoC) 2020! This will be our 16th year of GSoC!

We are now accepting student applications for our program that introduces university students from around the world to open source software communities, as well as our enthusiastic and generous community of mentors. For three months students code from the comfort of their homes (the program is entirely online!) and receive stipends based on the successful completion of their project milestones.

Past participants say the real-world experience that GSoC provides honed their technical skills, boosted their confidence, expanded their professional network, and enhanced their resume, all while making them better developers.

Interested students can submit proposals on the program site between now and Tuesday, March 31, 2020 at 18:00 UTC.

While many students began preparing in February when we announced the 200 participating open source organizations, it’s not too late for you to start! The first step is to browse the list of organizations and look for project ideas that appeal to you. Next, reach out to the organization to introduce yourself and determine if your skills and interests are a good fit. Since spots are limited, we recommend writing a strong proposal and submitting a draft early so you can communicate with the organization and get their feedback to increase your odds of being selected.

You can learn more about how to prepare by watching the video below and checking out the Student Guide and Advice for Students.



You can find more information on our website, including a full timeline of important dates. We also highly recommend reviewing the FAQ and Program Rules.

Remember to submit your proposals early as you only have until Tuesday, March 31 at 18:00 UTC. Good luck to all who apply!

By Stephanie Taylor, Google Open Source

WebAssembly brings extensibility to network proxies

With the Istio 1.5 release we are happy to introduce WebAssembly (Wasm) extensions in Envoy, built with our long running collaborators Lyft and IBM. With partners like Solo.io deepening their engagement as well, we are excited to see the community and ecosystem developing around this segment of the open source world.

The Envoy service proxy has taken the Cloud Native landscape by storm since it was open sourced by Lyft in 2016, quickly becoming a fixture in modern app deployment—both at the edge and as a sidecar. Since Google and IBM started the Istio project and selected Envoy as the proxy of choice for service mesh, we have been working with the Envoy community to improve performance and add functionality. In fact, Google now commits more code to Envoy than any other company.

Envoy has always had an extension mechanism, either with compiled-in C++ modules or Lua scripts—both with downsides. One of our design goals with Istio was to bring ease of extensibility to allow an ecosystem of policy, telemetry, and logging systems. We did this with a control plane component and out-of-process adapters that could be written in any language, but this approach introduced additional network hops and latency.

This is where Wasm comes in. Wasm is a binary instruction format, compilable from over 30 languages, with a runtime to execute it in a sandboxed environment. Already embedded in all major browsers and with a W3C working group defining the standards, we are now bringing it server-side via Envoy. It allows adding functionality to the Envoy proxy without recompiling it, without forking, and without difficult rollouts. Istio can distribute extensions to proxies and load them without even restarting. This really brings together the best of both worlds in terms of extensibility—choice of language and great performance.

“I am extremely excited to see Wasm support land in Envoy; this is the future of Envoy extensibility, full stop. Envoy’s Wasm support coupled with a community driven hub will unlock an incredible amount of innovation in the networking space across both service mesh and API gateway use cases. I can’t wait to see what the community builds moving forward.” – Matt Klein, Envoy creator

To make sure that developing Wasm extensions is a great experience, our partner Solo.io has been working hard on creating a great developer experience. Solo.io recently announced WebAssembly Hub, a service for building, sharing, discovering and deploying Wasm extensions. With the WebAssembly Hub, Wasm extensions are as easy to manage, install and run as containers.

“We are committed to creating the most user friendly developer experience for service mesh. Like Docker did for containers, our goal is to simplify the consumption of WebAssembly extensions, which is the ‘why' behind WebAssembly Hub. By working with Google and the Istio open source community, we are able to simplify the experience of creating, sharing and deploying WebAssembly extensions to Envoy proxy and Istio, to bring the power of WebAssembly to more languages, and to enable a broader set of developers to innovate on service mesh." said Idit Levine, CEO and Founder, Solo.io.

One major retailer is looking to use Wasm to integrate with their policy system as they standardize use of Envoy—at the edge, as a sidecar, and even in their stores. The ability to roll out a policy change that is enforced everywhere they serve traffic, all with a great developer experience, makes Wasm a very attractive option for them.

By Dan Ciruli, Istio

Measuring Compositional Generalization

People are capable of learning the meaning of a new word and then applying it to other language contexts. As Lake and Baroni put it, “Once a person learns the meaning of a new verb ‘dax’, he or she can immediately understand the meaning of ‘dax twice’ and ‘sing and dax’.” Similarly, one can learn a new object shape and then recognize it with different compositions of previously learned colors or materials (e.g., in the CLEVR dataset). This is because people exhibit the capacity to understand and produce a potentially infinite number of novel combinations of known components, or as Chomsky said, to make “infinite use of finite means.” In the context of a machine learning model learning from a set of training examples, this skill is called compositional generalization.

A common approach for measuring compositional generalization in machine learning (ML) systems is to split the training and testing data based on properties that intuitively correlate with compositional structure. For instance, one approach is to split the data based on sequence length—the training set consists of short examples, while the test set consists of longer examples. Another approach uses sequence patterns, meaning the split is based on randomly assigning clusters of examples sharing the same pattern to either train or test sets. For instance, the questions "Who directed Movie1" and "Who directed Movie2" both fall into the pattern "Who directed <MOVIE>" so they would be grouped together. Yet another method uses held out primitives—some linguistic primitives are shown very rarely during training (e.g., the verb “jump”), but are very prominent in testing. While each of these experiments are useful, it is not immediately clear which experiment is a "better" measure for compositionality. Is it possible to systematically design an “optimal” compositional generalization experiment?

In “Measuring Compositional Generalization: A Comprehensive Method on Realistic Data”, we attempt to address this question by introducing the largest and most comprehensive benchmark for compositional generalization using realistic natural language understanding tasks, specifically, semantic parsing and question answering. In this work, we propose a metric—compound divergence—that allows one to quantitatively assess how much a train-test split measures the compositional generalization ability of an ML system. We analyze the compositional generalization ability of three sequence to sequence ML architectures, and find that they fail to generalize compositionally. We also are releasing the Compositional Freebase Questions dataset used in the work as a resource for researchers wishing to improve upon these results.

Measuring Compositionality

In order to measure the compositional generalization ability of a system, we start with the assumption that we understand the underlying principles of how examples are generated. For instance, we begin with the grammar rules to which we must adhere when generating questions and answers. We then draw a distinction between atoms and compounds. Atoms are the building blocks that are used to generate examples and compounds are concrete (potentially partial) compositions of these atoms. For example, in the figure below, every box is an atom (e.g., Shane Steel, brother, <entity>'s <entity>, produce, etc.), which fits together to form compounds, such as produce and <verb>, Shane Steel’s brother, Did Shane Steel’s brother produce and direct Revenge of the Spy?, etc.

Building compositional sentences (compounds) from building blocks (atoms)

An ideal compositionality experiment then should have a similar atom distribution, i.e., the distribution of words and sub-phrases in the training set is as similar as possible to their distribution in the test set, but with a different compound distribution. To measure compositional generalization on a question answering task about a movie domain, one might, for instance, have the following questions in train and test:

Train set	Test set
Who directed Inception? Did Greta Gerwig direct Goldfinger? ...	Did Greta Gerwig produce Goldfinger? Who produced Inception? ...

While atoms such as “directed”, “Inception”, and “who <predicate> <entity>” appear in both the train and test sets, the compounds are different.

The Compositional Freebase Questions dataset

In order to conduct an accurate compositionality experiment, we created the Compositional Freebase Questions (CFQ) dataset, a simple, yet realistic, large dataset of natural language questions and answers generated from the public Freebase knowledge base. The CFQ can be used for text-in / text-out tasks, as well as semantic parsing. In our experiments, we focus on semantic parsing, where the input is a natural language question and the output is a query, which when executed against Freebase, produces the correct outcome. CFQ contains around 240k examples and almost 35k query patterns, making it significantly larger and more complex than comparable datasets — about 4 times that of WikiSQL with about 17x more query patterns than Complex Web Questions. Special care has been taken to ensure that the questions and answers are natural. We also quantify the complexity of the syntax in each example using the “complexity level” metric (L), which corresponds roughly to the depth of the parse tree, examples of which are shown below.

L	Question → Answer
10	What did Commerzbank acquire? → Eurohypo; Dresdner Bank
15	Did Dianna Rhodes’s spouse produce Soldier Blue? → No
20	Which costume designer of E.T. married Mannequin’s cinematographer? → Deborah Lynn Scott
40	Was Weekend Cowgirls produced, directed, and written by a film editor that The Evergreen State College and Fairway Pictures employed → No
50	Were It’s Not About the Shawerma, The Fifth Wall, Rick’s Canoe, White Stork Is Coming, and Blues for the Avatar executive produced, edited, directed, and written by a screenwriter’s parent? → Yes

Compositional Generalization Experiments on CFQ

For a given train-test split, if the compound distributions of the train and test sets are very similar, then their compound divergence would be close to 0, indicating that they are not difficult tests for compositional generalization. A compound divergence close to 1 means that the train-test sets have many different compounds, which makes it a good test for compositional generalization. Compound divergence thus captures the notion of "different compound distribution", as desired.

We algorithmically generate train-test splits using the CFQ dataset that have a compound divergence ranging from 0 to 0.7 (the maximum that we were able to achieve). We fix the atom divergence to be very small. Then, for each split we measure the performance of three standard ML architectures — LSTM+attention, Transformer, and Universal Transformer. The results are shown in the graph below.

Compound divergence vs accuracy for three ML architectures. There is a surprisingly strong negative correlation between compound divergence and accuracy.

We measure the performance of a model by comparing the correct answers with the output string given by the model. All models achieve an accuracy greater than 95% when the compound divergence is very low. The mean accuracy on the split with highest compound divergence is below 20% for all architectures, which means that even a large training set with a similar atom distribution between train and test is not sufficient for the architectures to generalize well. For all architectures, there is a strong negative correlation between the compound divergence and the accuracy. This seems to indicate that compound divergence successfully captures the core difficulty for these ML architectures to generalize compositionally.

Potentially promising directions for future work might be to apply unsupervised pre-training on input language or output queries, or to use more diverse or more targeted learning architectures, such as syntactic attention. It would also be interesting to apply this approach to other domains such as visual reasoning, e.g. based on CLEVR, or to extend our approach to broader subsets of language understanding, including the use of ambiguous constructs, negations, quantification, comparatives, additional languages, and other vertical domains. We hope that this work will inspire others to use this benchmark to advance the compositional generalization capabilities of learning systems.

By Marc van Zee, Software Engineer, Google Research – Brain Team

USB Keystroke Injection Protection

USB keystroke injection attacks have been an issue for a long time—problematic and affordable, due to the availability and price of keystroke injection tools. Those attacks send keystrokes immensely fast, in a human eyeblink, while being effectively invisible to the victim. Initially proposed to ease system administrator tasks, attackers learned how to use this technology for their purpose and compromise user systems. Here is an attack example, with a more or less benign payload:



To make the life of an attacker harder, we propose a tool that measures the timing of incoming keystrokes and determines if it is an attack based on predefined heuristics (without a user being involved in the decision). In contrast to the successful “attack” shown above, the following shows the same payload but with the tool installed on the system:

Choosing the RUN mode

The tool offers two different modes of operation: MONITOR and HARDENING. When running it in monitoring mode it won’t block a device that was classified as malicious, but will write a log line with information about the device to syslog. If it is run in hardening mode, it will immediately block a device that was classified as malicious/attacking. Out of the box, the tool is shipped in HARDENING mode.

Investigation

If the tool is running in monitoring mode, it logs information to the syslog. For one time inspection, this log can be read by simply using journalctl:

journalctl -u ukip.service

If it is rolled out to more machines in a network, it makes sense to collect each syslog at a centralized place for investigation.

Choosing the heuristics

A challenge when running the tool is the proper selection of the two main heuristic variables: KEYSTROKE_WINDOW and ABNORMAL_TYPING, which control the behaviour of the tool and its detection capabilities. The first one is the number of keystrokes it looks at, to determine whether it’s dealing with an attack or not. The lower the number, the higher the false positive rate; if the number is 2, the tool only looks at 1 interarrival time (the time between 2 keystrokes) to determine if it's an attack. Since users sometimes hit two keys almost at the same time it leads to the aforementioned false positive. Based on internal observations, 5 is an effective value, but should be adjusted based on the specific user’s experiences and typing behavior. The second variable specifies what interarrival time should be classified as malicious. More false-positives arise with a higher number (normal typing speed will be classified as malicious), versus with a lower number where more false-negatives arise (even very fast typing attacks will be classified as benign). That said, the preset 50000 after initial installation is a safe default but should be changed to a number reflecting the typing speed of the user using the tool. Finding the proper speed can be achieved in two ways: 1) By using one of the various online tools to measure the typing speed, and 2) using the Monitoring mode and letting it run for a few days (or even weeks) and gradually lower the false positive rate until it’s gone.

Getting it up and running

The README on Github contains a step-by-step guide to prepare the tool, set it up and run it as a systemd daemon, that is enabled on reboot. Over time it may be necessary to revise the variables for the tool by simply adjusting the values on top of /usr/sbin/ukip and restarting of the daemon:

sudo systemctl restart ukip.service

A note on silver bullets

The tool is not a silver bullet against USB-based attacks or keystroke injection attacks, since an attacker with access to a user’s machine (required for USB-based keystroke injection attacks) can do worse things if the machine is left unlocked. The tool is meant to provide another layer of protection and to defend a user sitting in front of their unlocked machine by them seeing the attack happening. They are able to see the attack either because the keystrokes are delayed enough to circumvent the tool’s logic or fast enough to be detected by it, i.e., blocking the device by unbinding its driver and logging information to syslog.

Keystroke injection attacks are difficult to detect and prevent since they’re delivered over USB (the most widely used computer peripheral connector) and require a Human Interface Device Driver (available on likely every operating system for mouse and keyboard input). The proposed tool raises the bar making it more difficult for the attacker while removing the user in the decision about whether a device is malicious or benign, apart from the refinement of the heuristic variables mentioned above. The tool can be complemented with other Linux tools, such as fine-grained udev rules or open source projects like USBGuard, to make successful attacks more challenging. The latter lets users define policies and allow/block specific USB devices or block USB devices while the screen is locked. That feature is specifically useful, since an attacker could plug in a device while the user is away from their keyboard and launch an attack once they are back. With USBGuard in place, the device would need to be replugged when the system is unlocked to work correctly.

By Sebastian Neuner, Google Information Security Engineering Team

Season of Docs announces the successful 2019 long-running projects

Season of Docs is happy to announce that all eight of the 2019 long-running documentation projects have finished successfully!

The successful long-running documentation projects are (in alphabetical order):

Apache Cassandra (Project Report, Project Description)

CERN-HSF (Project Report, Project Description)

Kolibri (Project Report, Project Description)

Mattermost (Project Report, Project Description)

MDAnalysis (Project Report, Project Description)

Open Food Facts (Project Report, Project Description)

Open Source Geospatial Foundation (Project Report, Project Description)

Tor Project (Project Report, Project Description)

Congratulations to the technical writers and organization mentors on these successful projects!

During the program, technical writers spent a few months working closely with an open source community. They brought their technical writing expertise to improve the project's documentation while the open source projects provided mentors to introduce the technical writers to open source tools, workflows, and the project's technology.

The technical writers and their mentors did a fantastic job with the inaugural year of Season of Docs! Participants in the 2019 program represented countries across all continents except for Antarctica!

You can view a list of the 44 successfully completed technical writing projects and read their project reports on the Season of Docs website.

What’s next?

Program participants should expect an email in the next few weeks about how to get their Season of Docs 2019 t-shirt (sure to be a collector’s item)!

If you’re interested in participating in a future Season of Docs, stay tuned for more information shortly—watch for posts on this blog and sign up for the announcements email list.

By Erin McKean and Kassandra Dhillon, Google Open Source

FuzzBench: Fuzzer Benchmarking as a Service

We are excited to launch FuzzBench, a fully automated, open source, free service for evaluating fuzzers. The goal of FuzzBench is to make it painless to rigorously evaluate fuzzing research and make fuzzing research easier for the community to adopt.

Fuzzing is an important bug finding technique. At Google, we’ve found tens of thousands of bugs (1, 2) with fuzzers like libFuzzer and AFL. There are numerous research papers that either improve upon these tools (e.g. MOpt-AFL, AFLFast, etc) or introduce new techniques (e.g. Driller, QSYM, etc) for bug finding. However, it is hard to know how well these new tools and techniques generalize on a large set of real world programs. Though research normally includes evaluations, these often have shortcomings—they don't use a large and diverse set of real world benchmarks, use few trials, use short trials, or lack statistical tests to illustrate if findings are significant. This is understandable since full scale experiments can be prohibitively expensive for researchers. For example, a 24-hour, 10-trial, 10 fuzzer, 20 benchmark experiment would require 2,000 CPUs to complete in a day.

To help solve these issues the OSS-Fuzz team is launching FuzzBench, a fully automated, open source, free service. FuzzBench provides a framework for painlessly evaluating fuzzers in a reproducible way. To use FuzzBench, researchers can simply integrate a fuzzer and FuzzBench will run an experiment for 24 hours with many trials and real world benchmarks. Based on data from this experiment, FuzzBench will produce a report comparing the performance of the fuzzer to others and give insights into the strengths and weaknesses of each fuzzer. This should allow researchers to focus more of their time on perfecting techniques and less time setting up evaluations and dealing with existing fuzzers.

Integrating a fuzzer with FuzzBench is simple as most integrations are less than 50 lines of code (example). Once a fuzzer is integrated, it can fuzz almost all 250+ OSS-Fuzz projects out of the box. We have already integrated ten fuzzers, including AFL, LibFuzzer, Honggfuzz, and several academic projects such as QSYM and Eclipser.

Reports include statistical tests to give an idea how likely it is that performance differences between fuzzers are simply due to chance, as well as the raw data so researchers can do their own analysis. Performance is determined by the amount of covered program edges, though we plan on adding crashes as a performance metric. You can view a sample report here.

How to Participate

Our goal is to develop FuzzBench with community contributions and input so that it becomes the gold standard for fuzzer evaluation. We invite members of the fuzzing research community to contribute their fuzzers and techniques, even while they are in development. Better evaluations will lead to more adoption and greater impact for fuzzing research.

We also encourage contributions of better ideas and techniques for evaluating fuzzers. Though we have made some progress on this problem, we have not solved it and we need the community’s help in developing these best practices.

Please join us by contributing to the FuzzBench repo on GitHub.

By: Jonathan Metzman, Abhishek Arya, Google OSS-Fuzz Team and László Szekeres‎, Google Software Analysis Team