opensource.google.com

Menu

Posts from 2020

Celebrating 10 years of WebM and WebRTC

Wednesday, May 27, 2020

Originally posted on the Chromium Blog

Ten years ago, Google planted the seeds for two foundational web media technologies, hoping they would provide the roots for a more vibrant internet. Two acquisitions, On2 Technologies and Global IP Solutions, led to a pair of open source projects: the WebM Project, a family of cutting edge video compression technologies (codecs) offered by Google royalty-free, and the WebRTC Project building APIs for real-time voice and video communication on the web.

These initiatives were major technical endeavors, essential infrastructure for enabling the promise of HTML5 with support for video conferencing and streaming. But this was also a philosophical evolution for media as Product Manager Mike Jazayeri noted in his blog post hailing the launch of the WebM Project:
“A key factor in the web’s success is that its core technologies such as HTML, HTTP, TCP/IP, etc. are open and freely implementable.”
As emerging first-class participants in the web experience, media and communication components also had to be free and open.

A decade later, these principles have ensured compression and communication technologies capable of keeping pace with a web ecosystem characterized by exponential growth of media consumption, devices, and demand. Starting from VP8 in 2010, the WebM Project has delivered up to 50% video bitrate savings with VP9 in 2013 and an additional 30% with AV1 in 2018—with adoption by YouTube, Facebook, Netflix, Twitch, and more. Equally importantly, the WebM team co-founded the Alliance for Open Media which has brought the IP of over 40 major tech companies in support of open and free codecs. With Chrome, Edge, Firefox and Safari supporting WebRTC, more than 85% of all installed browsers globally have become a client for real-time communications on the Internet. WebRTC has become a stable standard and it is now the default solution for video calling on the Web. These technologies have succeeded together, as today over 90% of encoded WebRTC video in Chrome uses VP8 or VP9.

The need for these technologies has been highlighted by COVID-19, as people across the globe have found new ways to work, educate, and connect with loved ones via video chat. The compression of open codecs has been essential to keeping services running on limited bandwidth, with over a billion hours of VP9 and AV1 content viewed every day. WebRTC has allowed for an ecosystem of interoperable communications apps to flourish: since the beginning of March 2020, we have seen in Chrome a 13X increase in received video streams via WebRTC.

These successes would not have been possible without all the supporters that make an open source community. Thank you to all the code contributors, testers, bug filers, and corporate partners who helped make this ecosystem a reality. A decade in, Google remains as committed as ever to open media on the web. We look forward to continuing that work with all of you in the next decade and beyond.

By Matt Frost, Product Director Chrome Media and Niklas Blum, Senior Product Manager WebRTC

Google Cardboard XR Plugin for Unity

Tuesday, May 26, 2020

Late in 2019, we decided to open source Google Cardboard. Since then, our developer community has had access to create a plethora of experiences on both iOS and Android platforms, while reaching millions of users around the world. While this release has been considered a success by our developer community, we also promised that we would release a plugin for Unity. Our users have long preferred developing Cardboard experiences in Unity, so we made it a priority to develop a Unity SDK. Today, we have fulfilled that promise, and the Google Cardboard open source plugin for Unity is now available via the Unity Asset Store

What's Included in the Cardboard Unity SDK

Today, we’re releasing the Cardboard Unity SDK to our users so that they can continue creating smartphone XR experiences using Unity. Unity is one of the most popular 3D and XR development platforms in the world, and our release of this SDK will give our content creators a smoother workflow with Unity when developing for Cardboard.

In addition to the Unity SDK, we are also providing a sample application for iOS/Android, which will be a great aid for developers trying to debug their own creations. This release not only fulfills a promise we made to our Cardboard community, but also shows our support, as we move away from smartphone VR and leave it in the more-than-capable hands of our development community.



If you’re interested in learning how to develop with the Cardboard open source project, please see our developer documentation or visit the Google VR GitHub repo to access source code, build the project, and download the latest release.

By Jonathan Goodlow, Product Manager, AR & VR

Knative elects new Technical Oversight Committee members

Monday, May 18, 2020

Towards the end of 2019, Knative project initiated a series of changes to its governance to ensure sustainability in the long term. Over the last week, the project reached a new milestone by successfully wrapping up its first Technical Oversight Committee (TOC) elections, bringing more vendor diversity to the technical stewardship of Knative.

Google has grown thousands of open source projects throughout the years, and it is this collective knowledge that informed the changes proposed to Knative governance. Over the last six months, we worked together with the other members of the Knative Steering Committee, and with the project’s contributors to create a clear set of rules for technical leadership and governance, describing the many ways in which contributors could engage with the project. This process was key to developing trust with Knative’s community, the project’s most valued stakeholder. In the exercise of this vote, the community was able to test the new election process, which proved to be solid: it will be repeated annually for this project, and can serve as a model for other projects as well.

The TOC election, which had a turnout of 70% of active contributors to the project, yielded a new technical stewardship for Knative, with members representing RedHat, VMWare and Google, as follows:

Nghia Tran (Google) - new member
Markus Thömmes (Red Hat) - new member
Grant Rodgers (Google) - new member
Matt Moore (VMWare) - existing member
Evan Anderson (VMWare) - existing member

Members of Knative TOC not only have the technical stewardship of the project in their hands for the next two years, they also model the community’s values: they have strong technical skills, they contribute to the project, and they are collegial, mentoring other contributors and helping the project to grow in a sustainable and healthy way.

We celebrated this important milestone for Knative at the last community meetup. Watch the video to meet the new TOC members, and check out the contribution guidelines to join the project.

By María Cruz, Google Open Source

SoD: experiences and tips for participation

Thursday, May 14, 2020

Have you ever heard about a program that lets technical writers contribute to open source documentation? Towards this effort, Google launched its pilot program for Season of Docs (SoD) around April 2019. SoD immediately caught my attention because it is the first-ever program that seamlessly brings tech writing to the open source world. By carefully selecting open source organizations and their projects, SoD allows seasoned or aspiring technical writers around the world to work closely with an open source organization, and understand their open source product, processes, and tools.

Luckily, I discovered SoD 2019, submitted my application—and after being selected to work for Tor—started my journey into the world of open source. I successfully completed my project in March 2020, and my project report was called out on the Results page for SoD 2019 that lists all successful contributions.

Now in its second year, SoD has gained momentum and is the talk of most technical writing communities and student circles. For the benefit of SoD aspirants, I am recounting my participation experiences along with useful tips in this blog post. I have included answers to questions that I am frequently asked, along with some simple tips to help you successfully complete your project.
  • Firstly, all aspiring participants for SoD 2020 must go through the program and its timeline hosted on the Season of Docs website. Make sure you read the Technical writer guide to understand the different phases of the program.
  • It’s a common myth that only seasoned technical writers who have coding knowledge are at fit for this program as SoD 2019 saw a lot of students participating in the program. Experienced technical writers or even students, who can show their interest in tech documentation are free to participate. From the highly to moderately technical projects available, you can opt for a project depending on your expertise and knowledge.
  • During the application phase, try to shortlist organizations closest to your domain knowledge and work area, and submit project proposals for two or three organizations. This increases your chance of being accepted by an organization. I shortlisted Tor and Hydra because of my interest in anonymous communications and my experience of working with APIs.
  • Before submitting your proposal, it’s a good idea to establish interaction with the org mentors to understand if you’re going in the right direction with your project proposal and most importantly to break the ice for future communications. I found some interesting project ideas and reached out to the mentors to understand their expectations of the project before submitting my application. In hindsight, this proved to be extremely valuable, as I was able to fine-tune my proposal in collaboration with my mentor, and I had already built a rapport with them.
  • Build your project proposal based on whether you’re opting for a three- or a five-month program. Break down details by months and weeks, which will testify your commitment to the project. I had chosen a standard three-month long project and detailed my work by months. When scoping your project, always allow time for roadblocks and unforeseen situations.
  • After your proposal is selected by an organization, it’s community-bonding time. Take this phase seriously and use this time to get to know your peers in the organization, build a rapport with your mentors, set up a communication channel with them, and prepare your work environment. My project proposal for Tor was accepted, and my mentors reached out to me to quickly get me up-to-speed with their communication tools, meeting timings, and smoothly inducted me into the networking team that I was going to work with. Even before I started working on the project, I was attending their weekly meetings and learning more about the work they do.
  • During the doc development phase, try to accomplish everything you promised in your proposal. At the same time, don’t feel bogged down by any changes that arise due to the complexity of your proposal. I faced hiccups during this phase because some of my ideas were not possible to implement and I had to rescope my proposal. Thanks to the invaluable support from my mentors and my peers at Tor, I was able to overcome all the obstacles and move forward with my project. The key to overcoming hurdles during your project is to keep your mentors updated about your work with frequent communication.
  • Once the work’s done complete your project report which will serve as the final assessment. Ensure that this report clearly shows all the work you’ve done; nothing is too big or too small to highlight in this report. A well-written report is more important than your project proposal as this decides whether your project has been successful. Based on this report, your mentor gives you a pass or fail mark for your project.
  • If your project is successful, you receive a stipend at the end of the program if you opted for one. Choosing to opt out of the stipend does not increase your chances of being selected to the program. This depends solely on your project proposal and your efforts at bonding with the chosen communities before applying.
For any more questions or concerns that you may have at any point in the program, I’d suggest looking up the FAQ for technical writers. You can also give a shout out to the extremely helpful program admins at season-of-docs-support@googlegroups.com.

I hope I have inspired SoD 2020 applicants to make their participation successful. I wish each one the very best.

By Swati Thacker, guest writer from Oracle

Insights from mixing writers with open source

Wednesday, May 13, 2020

The OSGeo Foundation participated in Google’s first Season of Docs, where Google sponsored technical writers to contribute to open source projects. The Open Geospatial Foundation (OSGeo), is an umbrella organization for around 50 geospatial open source projects. I’ve contributed to a number of these projects over the years and co-mentored the two Season of Docs technical writers allocated to us.
Screenshot from the OSGeoLive distribution we've been documenting, available under a CC-By license.
During our involvement we discovered that, like many open source projects, we knew little about:
  • The state of our docs.
  • What we were aiming for.
  • What our priorities were.
  • The details of the challenges we faced.
  • How to improve.
We learned:
  • How hard it is to keep tech docs current.
  • Skillsets from multiple roles are needed to create good docs.
  • Open source’s docs and writing processes are immature when compared to software development.
It is an exciting problem space with high-value challenges ready to be tackled. It reminds me of the early days of open source before it became trendy with business.

What should tech writers work on?

Open source communities welcomed the chance to have tech writers improve our docs, and expressed a pressing need for it, but found it challenging to articulate what exactly needed fixing.
  • People explained that their project docs often hadn’t been updated between doc releases.
  • Some projects had noticed new features that had not been documented.
  • Other projects had issue lists—collating observed deficiencies—but had no systematic review.
  • Most observed that docs were created by developers with no formal tech writing training.
  • Many noted that docs written by non-native language speakers and would benefit from grammatical review.
But where should we start? We needed to decide on what we wanted, what we should work on first.

What’s the definition of good docs?

And then we realized that we didn’t have a good definition of “good documentation.” For our software projects, we have a comprehensive incubation process to assess the maturity of software and the project’s community, but we couldn’t find a similar set of metrics to define “good documentation.” So we started TheGoodDocsProject, to collate “best-practice templates and writing instructions for documenting open source software.” This helped us define what we were aiming for, and prioritize what we can achieve with our available resources.

Documentation audit

Once we knew what good docs looked like, we were then able to audit the status of project’s docs:
  • What documentation do we have?
  • Does it cover all the functionality?
  • Does it cover end-user needs?
  • Is the documentation any good?
We discovered that the quality, currency, and completeness of our OSGeo docs were immature when compared to the quality software they described.

It takes a village to raise good docs

In researching open source projects’ documentation needs, it’s become clear that crafting good docs requires multiple skillsets. Ideally, a doc team would have access to:
  • A developer with a deep understanding of the software being described.
  • A software user who’s able to explain the application within the context of the application’s domain.
  • An educator who understands the principles of learning.
  • An information architect who understands how to structure material.
  • A writer who writes clearly and concisely with good grammar.
  • A translator who can translate docs into multiple languages.
  • A DevOps person who can set up doc build pipelines.
  • A community builder, facilitator, and coordinator, who can inspire collective action, capture offers of help, and help all these different personas collaborate together.
Technical writers usually have a high-level understanding of most of these domains and their skills are often under-appreciated and under-utilized, especially if directed with a vague “just clean up the grammar and stuff”. The best docs typically have been influenced by multiple stakeholders, which can be partly achieved using templates to collaborate between domains, timeframes, projects and organizations.

Tools for documenting open source projects are painful

We experienced significant difficulties trying to convert between writing and software toolsets. We love the versioning of git, are frustrated by clunky Markdown interfaces, and want access to editing and review workflows of Word and Google docs, along with grammar and syntax plugin tools such as Grammarly. Translation tools such as Transifex are pretty cool, too.

If a project were to address this use case, it would be an awesome gift to the open source community. Having someone write an application which addresses this use case would be helpful. Maybe there is an idea in here for a future Google Summer of Code?

Achievements during OSGeo’s Season of Docs

We’re quite proud of our achievements during OSGeo’s participation in the Season of Docs. Our allocated tech writers have amplified the effectiveness of our existing documentation communities, and our documentation communities have amplified the effectiveness of these tech writers.
  • Felicity Brand worked with around 50 of OSGeo’s open source projects to update their Quickstarts as part of our OSGeoLive distribution of software.
  • Swapnil Ogale worked directly with GeoNetwork’s documentation team, auditing the breadth of docs, and their quality, setting up templates for future docs to work towards, and updating a number of the docs.
Further:
  • We kicked off TheGoodDocsProject—“Best practice templates and writing instructions for documenting open source software.”
  • In conjunction with the OGC and ISO spatial standards communities, we kicked off an OSGeo Lexicon project, to coordinate official definitions for terminology used within the OSGeo context. This will apply best practice definitions to prior haphazard glossaries.
  • We did a deep-dive analysis of the documentation challenges faced by QGIS, one of OSGeo’s most successful projects. Surprisingly, their biggest problem isn’t a lack of tech writers or complicated tools (although they are factors). The key problems center around:
    • Poorly capturing community goodwill and offers of assistance.
    • A lack of direction.
    • Struggling to keep up with a rapidly evolving software baseline.
    • Insufficient writing expertise.
    • A high technical barrier to entry.
    • Documentation and training being generated outside of the core project.
    • Awkward documentation tools and processes.

Season of Docs 2020

Does tech writing interest you? If so, check the Season of Docs projects for 2020 and consider taking part.

By Cameron Shorter, Google technical writer and geospatial open source developer

Apache Beam presents its mascot to the world

Tuesday, May 12, 2020

Four years after it graduated from The Apache Software Foundation’s incubator, Apache Beam welcomes a new addition to the family: its mascot, the Beam firefly! Apache Beam is an open source data streaming programming model that runs on the back end of some of your favorite apps. It is the technology behind many popular apps that need to process big data in real time. And the reason it has come this far is the community of developers that contribute to this open source project every week. In the last few months, we worked with Apache Beam contributors to collaboratively design a mascot for the project—a creative asset that can represent the values of the project and attract new users and contributors—to keep growing the project and expanding its reach.

In these four years, the Apache Beam project has been a busy… firefly. According to the Apache Software Foundation’s 2019 Annual Report, Beam ranks fourth in the top five most active projects by commits, and it ranks first in the top five most active dev@ mailing list, showing a strong and transparent communication exchange within the community of developers. On top of that, 53 Meetup groups across the globe are directly or indirectly connected to sharing knowledge about Apache Beam use cases, applications and functionalities.

With this much momentum and enthusiasm for this project, it was a good opportunity to cement some of Apache Beam’s most valued characteristics, to help raise awareness of this rapidly growing project, and convert more users to contributors. “Projects with a mascot are more relatable. They signal that there is more to the project than its technical vision. It signals that there is more to the project than its code,” said project contributor Maximilian Michel. In November of last year, the community discussed adopting a mascot and as a result, 11 animal ideas emerged for a possible mascot: beaver, hedgehog, lemur, owl, salmon, trout, robot dinosaur, firefly, cuttlefish, dumbo octopus, and angler fish. After 48 contributors expressed their vote, the collective decision was a firefly. In January of this year, artist Julián Bruno set out to bring the mascot to life. There were four rounds of feedback on different iterations of the mascot, plus a final vote, where 18 people participated; Engagement increased with every new round. In the end, this process produced an original mascot, a model sheet (so that anyone may reproduce it), and two adaptations of the Beam firefly: one where it is learning, and one where it is doing what it does best… stream data! You can read more about this process on the Apache Software Foundation’s blog.

In a year that has presented a lot of challenges to bring people together, working on the mascot project with the Apache Beam community was refreshing, and felt like a medium for contributors to connect beyond code and technical questions. It is our wish that Apache Beam continues to grow as a project, and we hope to continue to support its community to: support newcomers, share what works, and collaborate with others to build great solutions.

By María Cruz, Google Open Source

Season of Docs announces participating organizations for 2020

Monday, May 11, 2020

Season of Docs has announced the 50 participating open source organizations! You can view the list of participating organizations on the website.

During the technical writer exploration phase, which runs from now until June 8, 2020, interested applicants explore the list of participating organizations and their project ideas. They should reach out to the organizations to gain a better understanding of the organizations and discuss project ideas before applying to Season of Docs. Technical writer applications open on June 9, 2020 at 18:00 UTC. 

For more information about the technical writer exploration phase, visit the technical writer guide on the website.

What is Season of Docs?

Documentation is essential for the adoption of open source projects as well as to the success of their communities. Season of Docs brings together technical writers and open source projects to foster collaboration and improve documentation in the open source space. You can find out more about the program on the introduction page of the website.

During the program, technical writers spend a few months working closely with an open source community. They bring their technical writing expertise to the project's documentation and, at the same time, learn about the open source project and new technologies.

The open source projects work with the technical writers to improve the project's documentation and processes. Together, they may choose to build a new documentation set, redesign the existing docs, or improve and document the project's contribution procedures and onboarding experience.

How do I take part in Season of Docs as a technical writer?

First, take a look at the technical writer guide on the website. The guide includes information on eligibility and the application process.

Then participate in the technical writer exploration phase, create a technical writing application and prepare your application materials. On June 9, 2020 at 18:00 UTC, Season of Docs will begin accepting technical writer applications and publish a link to the application form on the website. The deadline for technical writer applications is July 9, 2020 at 18:00 UTC.

Is there a stipend for participating technical writers?

Yes. There is an optional stipend that technical writers can request as part of their application. The stipend amount is calculated based on the technical writer's home location. See the technical writer stipends page for more information.

If you have any questions about the program, please email us at season-of-docs-support@googlegroups.com.

General timeline

May 11 – June 8Technical writers explore the list of participating organizations and project ideas
June 9 – July 9Technical writers submit their proposals to Season of Docs
August 16Google announces the accepted technical writer projects
August 17 – September 13Community bonding: Technical writers get to know mentors and the open source community, and refine their projects in collaboration with their mentors
September 14 – December 5Technical writers work with open source mentors on the accepted projects, and submit their work at the end of the period
January 6, 2021Google publishes the list of successfully-completed projects
See the full timeline for details, including the provision for projects that run longer than three months.

Care to join us?

Explore the Season of Docs website at g.co/seasonofdocs to learn more about participating in the program. Use our logo and other promotional resources to spread the word. Examine the timeline, check out the FAQ, and apply now!

By Kassandra Dhillon and Erin McKean, Google Open Source

The Tekton Pipelines Beta release

Friday, May 8, 2020

Tekton is a powerful and flexible open-source framework for creating CI/CD systems, allowing developers to build, test, and deploy across cloud providers and on-premise systems. The project recently released its Beta, which creates higher levels of stability by bringing the best features into the Pipelines Beta and creates more trust between the users and the features.


Tekton is used for infrastructure development on top of Kubernetes; it provides an open source framework for creating CI/CD systems, easily allowing developers to build, test, and deploy applications across applications.

With the new Beta functionality, users can rest assured that Beta features will not be removed, and that there will be a 9-month window dedicated to finding solutions for incompatible API changes. Since many in the Tekton community are using Tekton Pipelines to run APIs, this new release helps guarantee that any new developments on top of Tekton are reliable and optimized for best performance, with a budget of several months to make any necessary adjustments.

As platform builders require a stable API and feature set, the Beta launch includes Tasks, ClusterTasks and TaskRuns, Pipelines and PipelineRuns, to provide a foundation that users can rely on. Google created working groups in conjunction with other contributors from various companies to drive the Beta release. The team continues to churn out new Pipeline features towards a GA launch at the end of the year, while also focussing on bringing other components like metadata storage, Triggers, and the Catalog to Beta.


While initially starting as part of the Knative project from Google, in collaboration with developers from other organizations, Tekton was donated to the Continuous Delivery Foundation (CDF) in early 2019. Tekton’s initial design for the interface was even inspired by the Cloud Build API—and to this day—Google remains heavily involved in the commitment to develop Tekton, by participating in the governing board, and staffing a dedicated team invested in the success of this project. These characteristics make Tekton a prime example of a collaboration in open source.

Since its launch in February 2019, Tekton has had 3712 pull requests from 262 contributors across 39 companies spanning 16 countries. Many widely used projects across the open source industry are built on Tekton:
  • Puppet Project Nebula
  • Jenkins X
  • Red Hat OpenShift Pipelines
  • IBM Cloud Continuous Delivery
  • Kabanero – open source project led by IBM
  • Rio – open source project led by Rancher
  • Kf – open source project led by Google
Interested in trying out Tekton yourself? To install Tekton in your own kubernetes cluster (1.15 or newer), use kubectl to install the latest Tekton release:

kubectl apply -f
https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

You can jump right in by saving this Task to a file called task.yaml:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: hello-world
spec:
  steps:
  - image: ubuntu
    script: |
      echo "hello world"


Tasks are one of the most important building blocks of Tekton! Head over to tektoncd/catalog for more examples of reusable Tasks.

To run the hello-world Task, first apply it to your cluster with kubectl:

kubectl apply -f task.yaml

The easiest way to start running our Task is to use the Tekton command line tool, tkn. Install tkn using the right method for your OS, and you can run your Task with:

tkn task start hello-world --showlog

That’s just a taste of Tekton! At tekton.dev/try the community is hard at work adding interactive tutorials that let you try Tekton in a virtual environment. And you can dump straight into the docs at tekton.dev/docs and join the Tekton community at github.com/tektoncd/community.

Congratulations to all the contributors who made this Beta release possible!

By Radha Jhatakia and Christie Wilson, Google Open Source

GSoC 2020 student announcements!

Monday, May 4, 2020

The 16th Google Summer of Code (GSoC) application period is complete and the results are in: 199 mentoring organizations reviewed 8,902 applications—submitted by 6,626 students from 121 countries—selecting 1,199 students from 66 countries they will now be working with over the next few months. This was a groundbreaking year with applications! Congratulations to our 2020 GSoC students and a sincere thank you to everyone who applied.

The next step for participating students is the Community Bonding period, which runs from May 4th through May 31st. During this time, students will have the opportunity to get up to speed on the culture and code base of their new community. They’ll also get acquainted with their mentor(s) and learn more about the languages, or tools, that they will need to complete their projects. Coding will begin on June 1st continuing throughout the summer until August 24th.

If you were not selected for this year’s Summer of Code, don’t be discouraged! Many students apply more than once to GSoC before being accepted—persistence is key! You can improve your chances for next time by contributing to the open source project of your choice directly; organizations are always eager for new contributors! Look around for a project that interests you and get started. We hope that whether through GSoC or on your own, you pursue your interest in open source.

Happy coding, everyone!

By Stephanie Taylor, GSoC Program Lead

Cloud Spanner Emulator

Thursday, April 30, 2020

After Cloud Spanner’s launch in 2017, there has been huge customer adoption across several different industries and verticals. With this growth, we have built a large community of application developers using Cloud Spanner. To make the service even more accessible and open to the broader developer community, we are introducing an offline emulator for the Cloud Spanner service. The Cloud Spanner emulator is intended to reduce application development cost and improve developer productivity for the customers.

The Cloud Spanner emulator provides application developers with the full set of APIs, including the breadth of SQL and DDL features that could be run locally for prototyping, development and testing. This open source emulator will provide application developers with the transparency and agility to customize the tool for their application use.

This blog introduces the Cloud Spanner emulator and will guide you through installation and use of the emulator with the existing Cloud Spanner CLI and client libraries.

What is Cloud Spanner Emulator?

The emulator provides a local, in-memory, high-fidelity emulator of the Cloud Spanner service. You can use the emulator to prototype, develop and hermetically test your application locally and in your integration test environments.

Because the emulator stores data in-memory, it will not persist data across runs. The emulator is intended to help you use Cloud Spanner for local development and testing (not for production deployments); However, once your application is working with the emulator, you can proceed to end-to-end testing of your application by simply changing the Cloud Spanner endpoint configuration.

Supported Features

The Cloud Spanner emulator exposes the complete set of Cloud Spanner APIs including instances, databases, SQL, DML, DDL, sessions, and transaction semantics. Support for querying schema metadata for a database is available via Information Schema. Both gRPC and REST APIs are supported and can be used with the existing client libraries, OSS JDBC driver as well as the Cloud SDK. The emulator is supported natively on Linux, and requires Docker on MacOS and Windows platforms. To ease the development and testing of an application, IDEs like IntelliJ and Eclipse can be configured to directly communicate with the Cloud Spanner emulator endpoint.

The emulator is not built for production scale and performance, and therefore should not be used for load testing or production traffic. Application developers can use the emulator for iterative development, and to implement and run unit and integration tests.

A detailed list of features and limitations is provided on Cloud Spanner emulator README. The emulator is currently (as of April 2020) in beta release and will be continuously enhanced for feature and API parity with Cloud Spanner service.

Using the Cloud Spanner Emulator

This section describes using the existing Cloud Spanner CLI and client libraries to interact with the emulator.

Before You Start

Starting the emulator locally

The emulator can be started using Docker or using the Cloud SDK CLI on Linux, MacOS and Windows. In either case, MacOS and Windows would require an installation of docker.

Docker

$ docker pull gcr.io/cloud-spanner-emulator/emulator
$ docker run -p 9010:9010 -p 9020:9020 gcr.io/cloud-spanner-emulator/emulator

Note: The first port is the gRPC port and the second port is the REST port.

Cloud SDK CLI

$ gcloud components update beta
$ gcloud beta emulators spanner start
Other alternatives to start the emulator, including pre-built linux binaries, are listed here.

Setup Cloud Spanner Project & Instance

Configure Cloud Spanner endpoint, project and disable authentication:
$ gcloud config configurations create emulator
$ gcloud config set auth/disable_credentials true
$ gcloud config set project test-project
$ gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
Note:
To switch back to the default config:
`$ gcloud config configurations activate default`
To switch back to the emulator config:
`$ gcloud config configurations activate emulator`

Verify gCloud is working with the Cloud Spanner Emulator.
$ gcloud spanner instance-configs list

NAME               DISPLAY_NAME
emulator-config    Emulator Instance Config
Create a Cloud Spanner Instance
$ gcloud spanner instances create test-instance --config=emulator-config --description="Test Instance" --nodes=1

Using Cloud Spanner Client Libraries

With the beta lunch, the latest versions of Java, Go and C++ Cloud Spanner client libraries are supported to interact with the emulator. Use the Getting Started guides to try the emulator.

Prerequisite: Setup Cloud Spanner Project and Instance from step above.

This is an example of running the Java client library with the emulator:
# Configure emulator endpoint
$ export SPANNER_EMULATOR_HOST="localhost:9010"

# Cloning java sample of client library.
$ git clone https://github.com/GoogleCloudPlatform/java-docs-samples && cd java-docs-samples/spanner/cloud-client

$ mvn package

# Create database
$ java -jar target/spanner-google-cloud-samples-jar-with-dependencies.jar \
    createdatabase test-instance example-db


# Write
$ java -jar target/spanner-google-cloud-samples-jar-with-dependencies.jar \
    write test-instance example-db


# Query
$ java -jar target/spanner-google-cloud-samples-jar-with-dependencies.jar \
    query test-instance example-db
Follow the rest of the sample for Java client library using the Getting Started Guide.

Using the Cloud SDK CLI

Prerequisite: Setup Cloud Spanner Project and Instance from step above.

Configure emulator endpoint

$ gcloud config configurations activate emulator
Create a database

$ gcloud spanner databases create test-database --instance test-instance --ddl "CREATE TABLE TestTable (Key INT64, Value STRING(MAX)) PRIMARY KEY (Key)"
Write into database
$ gcloud spanner rows insert --table=TestTable --database=test-database --instance=test-instance --data=Key=1,Value=TestValue1
Read from database
$ gcloud spanner databases execute-sql test-database --instance test-instance --sql "select * from TestTable"

Using the open source command-line tool spanner-cli

Prerequisite: Setup Cloud Spanner Project, Instance and Database from step above.

Follow examples for an interactive prompt to Cloud Spanner database with spanner-cli.
# Configure emulator endpoint
$ export SPANNER_EMULATOR_HOST="localhost:9010"

$ go get github.com/cloudspannerecosystem/spanner-cli
$ go run github.com/cloudspannerecosystem/spanner-cli -p test-project -i test-instance -d test-database

spanner> INSERT INTO TestTable (Key, Value) VALUES (2, "TestValue2"), (3, "TestValue3");
Query OK, 2 rows affected

spanner> SELECT * FROM TestTable ORDER BY Key ASC;

+-----+----------------+
| Key | Value |
+-----+----------------+
| 2 | TestValue2 |
| 3 | TestValue3 |
+-----+----------------+
2 rows in set

spanner> exit;

Conclusion

Cloud Spanner emulator reduces application development cost and improves developer productivity for Cloud Spanner customers. We plan to continue building and supporting customer requested features and you can follow Cloud Spanner emulator on GitHub for more updates.

By Sneha Shah, Google Open Source

Announcing the 2020 first quarter Google Open Source Peer Bonus winners

Friday, April 17, 2020

We are very pleased to announce the latest Google Open Source Peer Bonus winners and their projects.

The Google Open Source Peer Bonus rewards external open source contributors nominated by Googlers for their exceptional contributions to open source. Historically, the program was primarily focused on rewarding developers. Over the years the program has evolved—rewarding not just software engineers but all types of contributors—including technical writers, user experience and graphic designers, community managers and marketers, mentors and educators, ops and security experts. 

In support of diversity, equity and inclusion initiatives worldwide, we had decided to devote this cycle to amazing women in open source, especially since it coincided with celebrating International Women’s Day on March 8. We are very excited and pleased to share the following statistics with you.

We have 56 winners this cycle representing 17 countries all over the world: Australia, Belgium, Canada, Estonia, France, Germany, India, Italy, Japan, Republic of Korea, Netherlands, Russia, Sweden, Switzerland, Ukraine, United Kingdom, and the United States.

Even though the cycle was open to ALL contributors, the number of female nominees went up from 8% to 25% in comparison to the previous cycle. That’s an amazing number celebrating amazing women!

Also, we are very pleased to see the number of docs contributors increase from 7% to 15%. Documentation is the #1 factor for project adoption, so this shift is very important and encouraging. To strengthen this trend and emphasize the importance of documentation in open source, the next cycle will be devoted (but not limited!) to docs contributors.

Below is the list of current winners who gave us permission to thank them publicly:
WinnerProject
Matt Mower
AMP HTML
Sergey Zakharov
Android Open Source Project
Pawel Kozlowski
Angular
Jakob Homan
Apache Airflow, Apache Kafka, Apache Hadoop
Chad Dombrova
Apache Beam
Myrle Krantz
Apache Software Foundation - Diversity and Inclusion committee + board
Katia Rojas
Apache Software Foundation Outreachy Program
Greg Hesp
assistant-relay
Beka Westberg
Blockly
Siebrand Mazeland
Blockly Games
Dave Mielke
BRLTTY
Vijay Hiremath
Chromium; platform/ec
Daniel Stenberg
curl / libcurl
Simon Binder
Dart build system
Aloďs Deniel
device_preview
Fatima Sarah Khalid
Drupal
Gregory Popovitch
Filament
Amr Yousef
Flutter
Remi Rousselet
Flutter
Pooja Bhaumik
Flutter
Elijah Newren
Git
Roger Peppe
Go
Oleksandr Porunov
JanusGraph
Tim Bannister
Kubernetes
June Yi
Kubernetes
Karen Bradshaw
Kubernetes
James Le Cuirot
leptonica
Stefan Weil
leptonica
Egor Pugin
leptonica
Bert Frees
LibLouis
Christian Egli
LibLouis
Richard Hughes
Linux Vendor Firmware Service (LVFS)
James (purpleidea)
mgmt
Mike Ryan
NgRx
Stefano Bonicatti
osquery
Alyssa Rosenzweig
panfrost
Carol Willing
Project Jupyter
Mariatta Wijaya
Python programming language
Alexander Neumann
restic
Nicholas Jamieson
rxjs (core member), rxjs-tslint-rules, rxjs-etc, ts-action
Kate Temkin
Several, mostly educational (see in Reasons)
Alyssa Ross
SpectrumOS / Nix
Rosalind Benoit
Spinnaker
Brian Le
Spinnaker
Vincent Demeester
Tekton
Chmouel Boudjnah
Tekton
Andrea Frittoli
Tekton
Simon Kaegi
Tekton
Cameron Shorter
The Good Docs Project
Ando Saabas
TreeInterpreter
Daz Wilkin
Trillian, Prometheus Exporter for GCP, KeyTransparency , OpenCensus
Gerrit Birkeland
typedoc
Wilson Snyder
Verilator
Thomas Oster
VisiCut
Koen Kanters
zigbee2mqtt
Jia Li
Zone.js
Congratulations to our winners! We look forward to your continued support and contributions to open source!

By Maria Tabak, Google Open Source
.