opensource.google.com

Menu

Posts from January 2018

Announcing the Winners of Google Code-in 2017

Wednesday, January 31, 2018

Google Code-in (GCI) 2017 was epic in every regard. It was a very, very busy 7 weeks for everyone - we had 3,555 students from 78 countries completing 16,468 tasks with a record 25 open source organizations!

Today we are excited to announce the Grand Prize Winners and Finalists with each organization. The 50 Grand Prize Winners completed an impressive 1,739 tasks between them while also helping other students.

Each of the Grand Prize Winners will be awarded a four day trip to Google’s campus in northern California to meet with Google engineers, meet with one of the mentors they worked with during the contest, and enjoy some fun in the California sun with the other winners. We look forward to meeting these winners in a few months!

Grand Prize Winners

The Grand Prize Winners hail from 12 countries, listed by first name alphabetically below:
Name Organization Country
Aadi Bajpai CCExtractor India
Aarnav Bos OpenWISP India
Abishek V Ashok FOSSASIA India
Aditya Giri OpenWISP India
Akshit Dewan XWiki United States
Albert Wolszon Wikimedia Poland
Andrew Dassonville coala United States
Arav Singhal MovingBlocks India
Arun Pattni XWiki United Kingdom
Aryaman Agrawal Systers Community India
Bartłomiej Rasztabiga OpenMRS Poland
Carol Chen Sugar Labs Canada
Chandra Catrobat Indonesia
Chirag Gupta The Mifos Initiative India
Cynthia Lin Zulip United States
Erika Tan Systers Community United States
Eshan Singh MetaBrainz India
Euan Ong Sugar Labs United Kingdom
Fawwaz Yusran OpenMRS Indonesia
Grzegorz Stark Apertium Poland
Hiếu Lê Haiku Vietnam
Jake Du LibreHealth United States
Jatin Luthra JBoss Community India
Jeff Sieu BRL-CAD Singapore
Jerry Huang OSGeo United States
Jonathan Pan Apertium United States
Jude Birch Catrobat United Kingdom
Konrad Krawiec Ubuntu Poland
Mahdi Dolatabadi BRL-CAD Canada
Marcin Mikołajczak Ubuntu Poland
Marco Burstein Zulip United States
Mateusz Grzonka LibreHealth Poland
Matthew Katz The Mifos Initiative Canada
Mehant Kammakomati SCoRe India
Nalin Bhardwaj coala India
Naveen Rajan FOSSASIA Sri Lanka
Nikita Volobuiev Wikimedia Ukraine
Omshi Samal Liquid Galaxy Project India
Owen Pan Haiku United States
Padam Chopra SCoRe India
Palash Taneja CloudCV India
Pavan Agrawal CloudCV United States
Sheik Meeran Ashmith Kifah Drupal Mauritius
Shiyuan Yu CCExtractor China
Sunveer Singh OSGeo India
Tanvish Jha Drupal India
Tarun Ravi Liquid Galaxy Project United States
Thomas O'Keeffe MovingBlocks United States
Vriyas Hartama Adesaputra MetaBrainz Indonesia
Zhao Wei Liew JBoss Community Singapore

Finalists

And a big congratulations to our 75 Finalists from 20 countries who will receive a special hoodie to commemorate their achievements in the contest. They are listed alphabetically by organization below:
Name Organization Name Organization
Alexander Mamaev Apertium Shamroy Pellew MetaBrainz
Robin Richtsfeld Apertium Aleksander Wójtowicz MovingBlocks
Ryan Chi Apertium Jindřich Dítě MovingBlocks
Caleb Parks BRL-CAD Nicholas Bates MovingBlocks
Lucas Prieels BRL-CAD Jyothsna Ashok OpenMRS
Mitesh Gulecha BRL-CAD Matthew Whitaker OpenMRS
Aditya Rathore Catrobat Tomasz Domagała OpenMRS
Andreas Lukita Catrobat Alan Zhu OpenWISP
Martina Hanusova Catrobat Hizkia Felix Winata OpenWISP
John Chew CCExtractor Vidya Haikal OpenWISP
Matej Plavevski CCExtractor Ethan Zhao OSGeo
William CCExtractor Neev Mistry OSGeo
Adam Štafa CloudCV Shailesh Kadam OSGeo
Adarsh Kumar CloudCV Emily Ong Hui Qi Sugar Labs
Naman Sood CloudCV Koh Pi Rong Sugar Labs
Anu Dookna coala Sanatan Chaudhary Sugar Labs
Marcos Gómez Bracamonte coala Adhyan Dhull SCoRe
Wonsang Chung coala Gaurav Pandey SCoRe
Kartik Goel Drupal Moses Paul SCoRe
Sagar Khatri Drupal Fidella Widjojo Systers Community
Tanish Kapur Drupal Valentin Sergeev Systers Community
Aditya Dutt FOSSASIA Yuyuan Luo Systers Community
Saarthak Chaturvedi FOSSASIA Janice Kim The Mifos Initiative
Yash Kumar Verma FOSSASIA Muhammad Rafly Andrianza The Mifos Initiative
Bach Nguyen Haiku Shivam Kumar Singh The Mifos Initiative
Đắc Tùng Dương Haiku Daniel Lim Ubuntu
Xiang Fan Haiku Qazi Omair Ahmed Ubuntu
Anhai Wang JBoss Community Simran Ubuntu
Divyansh Kulshreshtha JBoss Community David Siedtmann Wikimedia
Sachin Rammoorthy JBoss Community Rafid Aslam Wikimedia
Adrien Zier LibreHealth Yifei He Wikimedia
Miguel Dinis LibreHealth Akash Chandrasekaran XWiki
Vishwas Adiga LibreHealth Siddh Raman Pant XWiki
Shruti Singh Liquid Galaxy Project Srijan Jha XWiki
Kshitijaa Jaglan Liquid Galaxy Project Freddie Miller Zulip
Surya Tanwar Liquid Galaxy Project Priyank Patel Zulip
Enjeck Mbeh Cleopatra MetaBrainz Steven Hans Zulip
Kartik Ohri MetaBrainz

GCI is a contest that the Google Open Source team is honored to run every year. We saw immense growth this year, the seventh year of the contest, both in the number of students participating and the number of countries represented by these students. 

Our 730+ mentors, the heart and soul of GCI, are the reason the contest thrives. Mentors volunteer their time to help these bright students become open source contributors. Mentors spend hundreds of hours during their holiday breaks answering questions, reviewing submitted tasks, and welcoming the students to their communities. GCI would not be possible without their patience and tireless efforts.

We will post more statistics and fun stories that came from GCI 2017 here on the Google Open Source Blog over the next few months, so please stay tuned!

Congratulations to our Grand Prize Winners, Finalists, and all of the students who spent the last couple of months learning about and contributing to open source.

By Stephanie Taylor, Google Open Source

Googlers on the road: FOSDEM 2018

Monday, January 22, 2018

The Google Open Source team is currently enjoying summer weather in Sydney at Linux.conf.au, but soon we return to winter weather and head to Brussels for FOSDEM 2018. FOSDEM is a special event, famous for being non-commercial, volunteer-organized, and free to attend. It’s also huge, attracting more than 5,000 attendees.

FOSDEM logo licensed CC BY 2.0 SE.
This year FOSDEM is particularly special as it falls on top of the 20th anniversary of the open source movement and its steward, the Open Source Initiative. (In case you’re wondering, this September will mark the 35th anniversary of the free software movement.) We’re looking forward to celebrating the occasion!

You’ll find us in the hallways, at satellite events, and at our table in the stands area. You’ll also find some Googlers in the conference schedule, as well as folks sharing their experience of the most recent Google Summer of Code and Google Code-in.

If you’d like to say hello or chat, swing by our table in Building K. The highlight of our trip is meeting hundreds of the thousands of students and mentors who have participated in our programs!

Below are the Googlers who will be giving presentations:

Saturday, February 3rd
12:30pm  Google’s approach to distributed systems observability for Go by JBD (also at 2:30pm)
3:05pm    Testing and Validating distributed systems by Holden Karau

Sunday, February 4th
10:20am  Regular Expression Derivatives in Python by Michael Paddon
11:30am  Advocating For FOSS Inside Companies a panel including Max Sills
3:00pm    Your Build in a Datacenter by Jakob Buchgraber
4:00pm    Accelerating Big Data Outside of the JVM by Holden Karau

Hope to see you there!

By Josh Simmons, Google Open Source

Wrapping up Google Code-in 2017

Thursday, January 18, 2018

Today marks the conclusion of the 8th annual Google Code-in (GCI), our contest that teaches teenage students through contributions to open source projects. As with most years, the contest evolved a bit and grew. And it grew. And it doubled. And then it grew some more...
These numbers may increase as mentors finish reviewing the final work submitted by students.
Mentors from each of the 25 open source organizations are now busy reviewing the last of the work submitted by student participants. We’re looking forward to sharing the stats.

Each organization will pick two Grand Prize Winners who will be flown to Northern California to visit Google’s headquarters, enjoy a day of adventure in San Francisco, and meet their mentors and Google engineers.

We’d like to congratulate all of the student participants for challenging themselves and making a contribution to open source in the process! We’d also like to congratulate the mentors for surviving the unusually busy contest.

Further, we’d like to thank the mentors and the organization administrators. They are the heart of this program, volunteering countless hours creating tasks, reviewing student work, and helping students into the world of open source. Mentors teach young students about the many facets of open source development, from community standards and communicating across time zones to version control and testing. We couldn’t run this program without you!

Stay tuned, we’ll be announcing the Grand Prize Winners and Finalists on January 31st.

By Josh Simmons, Google Open Source

OpenCensus: A Stats Collection and Distributed Tracing Framework

Wednesday, January 17, 2018

Today we’re pleased to announce the release of OpenCensus, a vendor-neutral open source library for metric collection and tracing. OpenCensus is built to add minimal overhead and be deployed fleet wide, especially for microservice-based architectures.

The Need for Instrumentation & Observability 

As a startup, often the focus is to get an initial version of the product out the door, rapidly prototype and iterate with customers. Most startups start out with monolithic applications as a simple model-view-controller (MVC) web application. As the customer base, code, and number of engineers increase, they migrate from monolithic architecture to a microservices architecture. A microservices architecture has its advantages, but often makes debugging more challenging as traditional debugging and monitoring tools don’t always work in these environments or are designed for monolithic use cases. When operating multiple microservices with strict service level objectives (SLOs), you need insights into the root cause of reliability and performance problems.

Not having proper instrumentation and observability can result in lost engineering hours, violated SLOs and frustrated customers. Instead, diagnostic data should be collected from across the stack. This data can be used for incident management to identify and debug potential bottlenecks or for system tuning and performance improvement.

OpenCensus

At Google scale, an instrumentation layer with minimal overhead is a requirement. As Google grew, we realized the importance of having a highly efficient tracing and stats instrumentation library that could be deployed fleet wide.

OpenCensus is the open source version of Google’s Census library, written based on years of optimization experience. It aims to make the collection and submission of app metrics and traces easier for developers. It is a vendor neutral, single distribution of libraries that automatically collects traces and metrics from your app, displays them locally, and sends them to analysis tools. OpenCensus currently supports Prometheus, SignalFx, Stackdriver and Zipkin.

Developers can use this powerful, out-of-the box library to instrument microservices and send data to any supported backend. For an Application Performance Management (APM) vendor, OpenCensus provides free instrumentation coverage with minimal work, and affords customers a simple setup experience.

Below are Stackdriver Trace and Monitor screenshots showing traces generated from a demo app, which calls Google’s Cloud Bigtable API and uses OpenCensus.



We’d love to hear your feedback on OpenCensus. Try using it in your app, tell us about your success story, and help by contributing to our existing language-specific libraries, or by creating one for an not-yet-supported language. You can also help us integrate OpenCensus with new APM tools!

We hope you find this as useful as we have. Visit opencensus.io for more information.

By Pritam Shah, Census team

Googlers on the road: Linux.conf.au 2018

Monday, January 15, 2018

It’s summer in Sydney and Linux.conf.au (LCA) 2018 is just a week away. LCA, an annual event that attracts people from all over the globe, including Googlers, runs January 22nd to 26th.

LCA is a cornerstone of the free and open source software (FOSS) community. It’s volunteer-run, administered by Linux Australia, and has been running since 1999. Despite its name, the conference program covers all things FOSS. The event is five days long and includes two days of miniconfs that make the program even more interesting.

The Google Open Source team is escaping “wintery” Northern California and will be hosting a Birds of a Feather (BoF) session and co-hosting an event with GDG Sydney, both focused on our student programs.

A few Googlers ended up with sessions in the program and one is running a miniconf:

Tuesday, January 23rd
All day     Create hardware with FPGAs, Linux and Python Miniconf hosted by Tim Ansell (sold out)
11:40am  Learn by Contributing to Open Source by Josh Simmons
5:15pm    Assembling a balsa-wood Raspberry Pi case by Josh Deprez
5:45pm    Streaming Machine Learning with Apache Spark Meetup by Holden Karau

Wednesday, January 24th
1:40pm    Dealing with Contributor Overload by Holden Karau
3:50pm    Securing the Linux boot process by Matthew Garrett

Thursday, January 25th
12:25pm  Google Summer of Code and Google Code-in Birds of a Feather session
6:00pm    Google Summer of Code and Google Code-in Meetup with GDG Sydney

Friday, January 26th
11:40am  The State of Kernel Self-Protection by Kees Cook
1:40pm    QUIC: Replacing TCP for the Web by Jana Iyengar
2:35pm    The Web Is Dead! Long Live The Web! by Sam Thorogood

Not able to make the conference? They’ll be posting session recordings to YouTube afterwards, thanks in part to students who have worked on TimVideos, a suite of open source software and hardware for recording video, as part of Google Summer of Code.

Naturally, you will also find the Google Open Source team at other upcoming events including FOSDEM. We look forward to seeing you in 2018!

By Josh Simmons, Google Open Source

Container Structure Tests: Unit Tests for Docker Images

Tuesday, January 9, 2018

Usage of containers in software applications is on the rise, and with their increasing usage in production comes a need for robust testing and validation. Containers provide great testing environments, but actually validating the structure of the containers themselves can be tricky. The Docker toolchain provides us with easy ways to interact with the container images themselves, but no real way of verifying their contents. What if we want to ensure a set of commands runs successfully inside of our container, or check that certain files are in the correct place with the correct contents, before shipping?

The Container Tools team at Google is happy to announce the release of the Container Structure Test framework. This framework provides a convenient and powerful way to verify the contents and structure of your containers. We’ve been using this framework at Google to test all of our team’s released containers for over a year now, and we’re excited to finally share it with the public.

The framework supports four types of tests:
  • Command Tests - to run a command inside your container image and verify the output or error it produces
  • File Existence Tests - to check the existence of a file in a specific location in the image’s filesystem
  • File Content Tests - to check the contents and metadata of a file in the filesystem
  • A unique Metadata Test - to verify configuration and metadata of the container itself
Users can specify test configurations through YAML or JSON. This provides a way to abstract away the test configuration from the implementation of the tests, eliminating the need for hacky bash scripting or other solutions to test container images.

Command Tests

The Command Tests give the user a way to specify a set of commands to run inside of a container, and verify that the output, error, and exit code were as expected. An example configuration looks like this:
globalEnvVars:
  - key: "VIRTUAL_ENV"
    value: "/env"
  - key: "PATH"
    value: "/env/bin:$PATH"

commandTests:

# check that the python binary is in the correct location
  - name: "python installation"
    command: "which"
    args: ["python"]
    expectedOutput: ["/usr/bin/python\n"]

# setup a virtualenv, and verify the correct python binary is run
  - name: "python in virtualenv"
    setup: [["virtualenv", "/env"]]
    command: "which"
    args: ["python"]
    expectedOutput: ["/env/bin/python\n"]

# setup a virtualenv, install gunicorn, and verify the installation
  - name: "gunicorn flask"
    setup: [["virtualenv", "/env"],
            ["pip", "install", "gunicorn", "flask"]]
    command: "which"
    args: ["gunicorn"]
    expectedOutput: ["/env/bin/gunicorn"]
Regexes are used to match the expected output, and error, of each command (or excluded output/error if you want to make sure something didn’t happen). Additionally, setup and teardown commands can be run with each individual test, and environment variables can be specified to be set for each individual test, or globally for the entire test run (shown in the example).

File Tests

File Tests allow users to verify the contents of an image’s filesystem. We can check for existence of files, as well as examine the contents of individual files or directories. This can be particularly useful for ensuring that scripts, config files, or other runtime artifacts are in the correct places before shipping and running a container.
fileExistenceTests:

# check that the apt-packages text file exists and has the correct permissions
  - name: 'apt-packages'
    path: '/resources/apt-packages.txt'
    shouldExist: true
    permissions: '-rw-rw-r--'
Expected permissions and file mode can be specified for each file path in the form of a standard Unix permission string. As with the Command Tests’ “Excluded Output/Error,” a boolean can be provided to these tests to tell the framework to be sure a file is not present in a filesystem.

Additionally, the File Content Tests verify the contents of files and directories in the filesystem. This can be useful for checking package or repository versions, or config file contents among other things. Following the pattern of the previous tests, regexes are used to specify the expected or excluded contents.
fileContentTests:

# check that the default apt repository is set correctly
  - name: 'apt sources'
    path: '/etc/apt/sources.list'
    expectedContents: ['.*httpredir\.debian\.org/debian jessie main.*']

# check that the retry policy is correctly specified
  - name: 'retry policy'
    path: '/etc/apt/apt.conf.d/apt-retry'
    expectedContents: ['Acquire::Retries 3;']

Metadata Test

Unlike the previous tests which all allow any number to be specified, the Metadata test is a singleton test which verifies a container’s configuration. This is useful for making sure things specified in the Dockerfile (e.g. entrypoint, exposed ports, mounted volumes, etc.) are manifested correctly in a built container.
metadataTest:
  env:
    - key: "VIRTUAL_ENV"
      value: "/env"
  exposedPorts: ["8080", "2345"]
  volumes: ["/test"]
  entrypoint: []
  cmd: ["/bin/bash"]
  workdir: ["/app"]

Tiny Images

One interesting case that we’ve put focus on supporting is “tiny images.” We think keeping container sizes small is important, and sometimes the bare minimum in a container image might even exclude a shell. Users might be used to running something like:
`docker run -d "cat /etc/apt/sources.list && grep -rn 'httpredir.debian.org' image"`
… but this breaks without a working shell in a container. With the structure test framework, we convert images to in-memory filesystem representations, so no shell is needed to examine the contents of an image!

Dockerless Test Runs

At their core, Docker images are just bundles of tarballs. One of the major use cases for these tests is running in CI systems, and often we can't guarantee that we'll have access to a working Docker daemon in these environments. To address this, we created a tar-based test driver, which can handle the execution of all file-related tests through simple tar manipulation. Command tests are currently not supported in this mode, since running commands in a container requires a container runtime.

This means that using the tar driver, we can retrieve images from a remote registry, convert them into filesystems on disk, and verify file contents and metadata all without a working Docker daemon on the host! Our container-diff library is leveraged here to do all the image processing; see our previous blog post for more information.
structure-test -test.v -driver tar -image gcr.io/google-appengine/python:latest structure-test-examples/python/python_file_tests.yaml

Running in Bazel

Structure tests can also be run natively through Bazel, using the “container_test” rule. Bazel provides convenient build rules for building Docker images, so the structure tests can be run as part of a build to ensure any new built images are up to snuff before being released. Check out this example repo for a quick demonstration of how to incorporate these tests into a Bazel build.

We think this framework can be useful for anyone building and deploying their own containers in the wild, and hope that it can promote their usage everywhere through increasing the robustness of containers. For more detailed information on the test specifications, check out the documentation in our GitHub repository.

By Nick Kubala, Container Tools team

Talk shop with Google Open Source

Monday, January 8, 2018

Hello world! The Google Open Source team is ringing in the new year by launching accounts on Twitter, Facebook, and Google+ to engage more with the community and keep folks up to date.
Free and open source software (FOSS) is fundamental to computing, the internet, and Google. Since 2004, Google Open Source has helped Googlers get code in and out of Google and supported FOSS through student programs and financial support. One thing is clear after 14 years: FOSS is all about community.

We’re part of that community, seeing people at events, on mailing lists, and in the trenches of code repositories. And few things are more enjoyable and productive than talking with people in the community…

… so we thought we’d start doing more of that. We want to:
We hope you’ll come along and let us know. You’ll find us at @GoogleOSS and +GoogleOpenSource, as well as on Facebook and YouTube.

By Josh Simmons, Google Open Source

Seeking open source projects for Google Summer of Code 2018

Thursday, January 4, 2018

Do you lead or represent a free or open source software organization? Are you seeking new contributors? (Who isn’t?) Do you enjoy the challenge and reward of mentoring new developers? Apply to be a mentor organization for Google Summer of Code 2018!

We are seeking open source projects and organizations to participate in the 14th annual Google Summer of Code (GSoC). GSoC is a global program that gets student developers contributing to open source. Each student spends three months working on a project, with the support of volunteer mentors, for participating open source organizations.

Last year 1,318 students worked with 198 open source organizations. Organizations include individual projects and umbrella organizations that serve as fiscal sponsors, such as Apache Software Foundation or the Python Software Foundation.

You can apply starting today. The deadline to apply is January 23 at 16:00 UTC. Organizations chosen for GSoC 2018 will be posted on February 12.

Please visit the program site for more information on how to apply, a detailed timeline of important deadlines and general program information. We also encourage you to check out the Mentor Guide and join the discussion group.

Best of luck to all of the applicants!

By Josh Simmons, Google Open Source

.