opensource.google.com

Menu

Posts from July 2019

The Apache Beam Community in 2019

Monday, July 22, 2019

2019 has already been a busy time for the Apache Beam. The ASF blog featured our way of community building and we've had more Beam meetups around the world. Apache Beam also received the Technology of the Year Award from InfoWorld.
As these events happened, we were building up to the 20th anniversary of the Apache Software Foundation. The contributions of the Beam community were a part of Maximilian Michels blog post on the success of the ASF's open source development model:

As the founder of the first Beam meetup in London back in 2017, seeing the community flourish on a larger and worldwide scale is something that makes me happy. And we have come quite a long way since 2017, both in terms of geographical spread:



As well as in numbers:



All of this culminates in two Beam Summits this year—one we already had a few weeks ago in Berlin, and the other which will take place in a few weeks in Las Vegas, where we worked together with Apache and the ApacheCon team.

In that spirit, let's have a more detailed overview of the things that have happened, what the next few months look like, and how we can foster even more community growth.

Meetups

We've had a flurry of activity, with several meetups in the planning process and more popping up globally over time. As diversity of contributors is a core ASF value, this geographic spread is exciting for the community. Here's a picture from the latest Apache Beam meetup organized at Lyft in San Francisco:



We have more Bay Area meetups coming soon, and the community is looking into kicking off a meetup in Toronto and New York! In Europe, London had its first meetup of 2019 at the start of April, as did Stockholm at the start of May:
Meetup groups are becoming active in Berlin and New York also, so stay tuned for events there and more meetups internationally! If you are interested in starting your own meetup, feel free to reach out! Good places to start include our Slack channel, the dev and user mailing lists, or the Apache Beam Twitter. Even if you can’t travel to these meetups, you can stay informed on the happenings of the community. The talks and sessions from previous conferences and meetups are archived on the Apache Beam YouTube channel. If you want your session added to the channel, don’t hesitate to get in touch!

Summits

The first summit of the year was held in Berlin this past June. You can read about the inaugural edition of the Beam Summit Europe here. At these summits, you have the opportunity to meet with other Apache Beam creators and users, get expert advice, learn from the speaker sessions, and participate in workshops. We are proud to say that the Summit doubled in size this year with attendees from 24 countries across 4 continents.

You can find resources from this year’s Summit here:
  • 📹 the recordings can be found on our YouTube channel.
  • 📊 presentations of the Summit are made available via the website and in this folder.
  • We strongly encourage you to get involved again this year! You can still sign up for the upcoming summit in North America.
  • 🎫 If you want to secure your ticket to attend the Beam Summit North America 2019, check our the ApacheCon website.
  • 🔈 In case you want to get involved in speaking at events, do not hesitate to contact us via email or Twitter.

Why community engagement matters

Why we need a strong Apache Beam community:
  • We’re gaining lots of code contributions and need committers to review them
  • We want people to feel a sense of ownership to the project. By fostering this level of engagement, the work becomes even more exciting.
  • A healthy community has a further reach and leads to more growth. More hours can be contributed to the project as we can spread the work and ownership.
Why are we organizing these summits:
  • We’d like to give folks a place to meet and share ideas.
  • We know that offline interactions often changes the nature of the online ones in a positive manner.
  • Building an active and diverse community is part of the Apache Way. These summits provide an opportunity for us to engage people from different locations, companies, and backgrounds.
By Matthias Baetens, Google Developer Expert for Cloud, Apache Beam committer and community organiser

Announcing Docsy: A Website Theme for Technical Documentation

Wednesday, July 10, 2019

Have you ever struggled with the process of creating documentation for an open source project? Do you have an open source project that's outgrown its README? Open source projects need great docs to succeed, but great open source doc sites aren't always easy to produce and share.

Google supports over 2000 open source projects, and there has been growing demand from these projects for tooling and guidance to help them write and publish their documentation. To meet this need we created Docsy: a documentation website with templates and guidance for documentation, which we’re open sourcing to the public to use and help improve the tool.

Docsy builds on existing open source tools, like Hugo, and our experience with open source docs, providing a fast and easy way to stand up an OSS documentation website with features specifically designed to support technical documentation. Special features include everything from site navigation to multi-language support – with easy site deployment options provided by Hugo. We also created guidance on how to add additional pages, structure your documentation, and accept community contributions, all with the goal of letting you focus on creating great content.

Who’s using it?

The Kubeflow, Knative, and Agones websites were built using the Docsy theme, with more projects in the pipeline. We’ve also created an example site that uses lots of Docsy features for you to explore and copy.

Ready to get started?

Visit the Docsy site to find out how to create your first site with Docsy! You can either use Docsy like a regular Hugo theme, or clone our example site. Docsy is an open source project—of course—and we welcome your issues, suggestions, and contributions!

Come Meet the Google Open Source Team at OSCON!

Tuesday, July 9, 2019

Google Cloud is proud to be a Diamond Sponsor at OSCON, and we’re excited for another year of connecting, learning, and sharing with the open source community! Google is deeply grateful to all of your amazing open source efforts, so to celebrate, our booth will have an Open Gratitude wall where we will acknowledge your contributions, and where we encourage you to express your gratitude for those who have helped you in open source!

Once you’ve recognized your open source heroes on the Open Gratitude wall, stick around at the Google Open Source booth to learn about topics such as open source governance, documentation, open source in ML and gaming, encouraging non-code contributions, and about Google’s open source outreach programs in general. At our booth sessions you can also explore open source projects such as Kubernetes, Istio, Go, and Beam (as well as other Apache projects). Booth office hours run from 10:15am to 7pm Wednesday, July 17, and from 10:15am to 4:10pm on Thursday, July 18. The full schedule will be posted at the booth—please come by and check it out!

In addition to the events at the booth, the Google open source team has two workshops on Tuesday, July 16:
This half-day workshop kicks off with an overview of research-backed documentation best practices. Andrew Chen, Erin McKean, and Aizhamal Nurmamat kyzy lead you through a hands-on exercise in which you'll create the skeleton of a ready-to-deploy documentation website for your open source project.
Paris Pittman takes you through the ins and outs of the Kubernetes contributor community so you can land your first PR. You'll learn about SIGs, the GitHub workflow, its automation and continuous integration (CI), setting up your dev environment, and much more. Stick around until the end, and you'll have time to work on your first PR with the help of current contributors.
We also hope you attend the main conference sessions presented by Googlers, especially the keynotes on Wednesday (Built to last: What Google and Microsoft have learned growing open source communities) and Thursday (Be a Docs Star), and the sessions on Wednesday:
And Thursday:
As part of our commitment to creating a diverse and inclusive community, we’ve redirected our conference swag budget into diversity scholarships. (We believe you’d prefer to have more interesting conversations with a wider range of people over another pair of socks!) But if you are looking for a souvenir of your time in Portland there will be a special Portland-themed sticker featuring Pancakes, the (extremely adorable) gRPC mascot, and we encourage projects to take and leave stickers in our sticker-swap space!

OSCON is one of the highlights of the year for those of us who love open source—we’re thrilled to be able to share what we’ve learned with you, and to learn what you’re interested in and excited about (and also what you think could improve). See you in Portland!

Truth 1.0: Fluent Assertions for Java and Android Tests

Monday, July 8, 2019

Software testing is important—and sometimes frustrating. The frustration can come from working on innately hard domains, like concurrency, but too often it comes from a thousand small cuts:
assertEquals("Message has been sent", getString(notification, EXTRA_BIG_TEXT));
assertTrue(
    getString(notification, EXTRA_TEXT)
        .contains("Kurt Kluever <kak@google.com>"));
The two assertions above test almost the same thing, but they are structured differently. The difference in structure makes it hard to identify the difference in what's being tested.
A better way to structure these assertions is to use a fluent API:
assertThat(getString(notification, EXTRA_BIG_TEXT))
    .isEqualTo("Message has been sent");
assertThat(getString(notification, EXTRA_TEXT))
    .contains("Kurt Kluever <kak@google.com>");
A fluent API naturally leads to other advantages:
  • IDE autocompletion can suggest assertions that fit the value under test, including rich operations like containsExactly(permission.SEND_SMS, permission.READ_SMS).
  • Failure messages can include the value under test and the expected result. Contrast this with the assertTrue call above, which lacks a failure message entirely.
Google's fluent assertion library for Java and Android is Truth. We're happy to announce that we've released Truth 1.0, which stabilizes our API after years of fine-tuning.



Truth started in 2011 as a Googler's personal open source project. Later, it was donated back to Google and cultivated by the Java Core Libraries team, the people who bring you Guava.
You might already be familiar with assertion libraries like Hamcrest and AssertJ, which provide similar features. We've designed Truth to have a simpler API and more readable failure messages. For example, here's a failure message from AssertJ:
java.lang.AssertionError:
Expecting:
  <[year: 2019
month: 7
day: 15
]>
to contain exactly in any order:
  <[year: 2019
month: 6
day: 30
]>
elements not found:
  <[year: 2019
month: 6
day: 30
]>
and elements not expected:
  <[year: 2019
month: 7
day: 15
]>
And here's the equivalent message from Truth:
value of:
    iterable.onlyElement()
expected:
    year: 2019
    month: 6
    day: 30

but was:
    year: 2019
    month: 7
    day: 15
For more details, read our comparison of the libraries, and try Truth for yourself.

Also, if you're developing for Android, try AndroidX Test. It includes Truth extensions that make assertions even easier to write and failure messages even clearer:
assertThat(notification).extras().string(EXTRA_BIG_TEXT)
    .isEqualTo("Message has been sent");
assertThat(notification).extras().string(EXTRA_TEXT)
    .contains("Kurt Kluever <kak@google.com>");
Coming soon: Kotlin users of Truth can look forward to Kotlin-specific enhancements.
By Chris Povirk, Java Core Libraries

Google's robots.txt Parser is Now Open Source

Monday, July 1, 2019

Originally posted on the Google Webmaster Central Blog

For 25 years, the Robots Exclusion Protocol (REP) was only a de-facto standard. This had frustrating implications sometimes. On one hand, for webmasters, it meant uncertainty in corner cases, like when their text editor included BOM characters in their robots.txt files. On the other hand, for crawler and tool developers, it also brought uncertainty; for example, how should they deal with robots.txt files that are hundreds of megabytes large?

Today, we announced that we're spearheading the effort to make the REP an internet standard. While this is an important step, it means extra work for developers who parse robots.txt files.

We're here to help: we open sourced the C++ library that our production systems use for parsing and matching rules in robots.txt files. This library has been around for 20 years and it contains pieces of code that were written in the 90's. Since then, the library evolved; we learned a lot about how webmasters write robots.txt files and corner cases that we had to cover for, and added what we learned over the years also to the internet draft when it made sense.

We also included a testing tool in the open source package to help you test a few rules. Once built, the usage is very straightforward:

robots_main <robots.txt content> <user_agent> <url>

If you want to check out the library, head over to our GitHub repository for the robots.txt parser. We'd love to see what you can build using it! If you built something using the library, drop us a comment on Twitter, and if you have comments or questions about the library, find us on GitHub.

Posted by Edu Pereda, Lode Vandevenne, and Gary, Search Open Sourcing team
.