ZuriHac 2014: Haskell hackathon in Zurich

The Google Open Source Programs Office recently co-sponsored a three-day hackathon for Haskell, an open source functional programming language. Johan Tibell from Google’s Zurich office talks more about the event below.

June 6th saw the third installment of ZuriHac, a three-day Haskell hackathon in Zurich, Switzerland. With roughly 100 attendees from all over the world, this was the biggest ZuriHac to date.

In addition to hacking, hallway discussions, a demo session, and impromptu talks on topic ranging from GHC hacking to category theory, there were two one-hour talks. Simon Marlow talked about haxl, a library for automatically parallelization and batching of data requests. Edward Kmett followed with a talk about new and exciting persistent data structures based on cache-oblivious algorithms.

The hacking was all over the map. An entire room was dedicated to teaching newcomers how to hack on GHC, the Haskell compiler. A smaller group of people worked on various improvements to Cabal (the Haskell build system) by improving the dependency solver and making sure that parallel package builds work well. More lighthearted projects included a remake of 1990s cult video game Stunts.

Thanks to sponsorships by Google and Better, a local Haskell startup, we were well fed and had a roof over our heads.

By Johan Tibell, Software Engineer, and ZuriHac organizer

Google Summer of Code new organizations - Part two

We have two great new Google Summer of Code organizations to spotlight this week, OpenKeychain and Linaro.  It’s hard to believe that we are almost halfway through the coding period for GSoC 2014. Midterm evaluations for both mentors and students are due today, Friday, June 27.

OpenKeychain allows you to manage cryptographic keys and encrypt messages as well as files for your contacts on Android. Our project is focused on providing an easy user interface based on Android design principles. It implements the OpenPGP standard, often referred to as GPG or PGP, and enables a secure end-to-end communication in times of mass surveillance.

As a new organization to Google Summer of Code, we secured two student slots. One of our students will work on a better abstraction between cryptography backend and user interface code, resulting in some unit tests, while also working on a better integration of the OpenPGP web-of-trust. The other student will work on a better integration of OpenKeychain into the Android OS, including better support for file encryption and decryption using Android 4.4 features, integration with Android's contact application, and work on potential improvements for the API.

By Dominik Schürmann, OpenKeychain Organization Administrator

Linaro is the place where engineers from the world's leading technology companies define the future of Linux on ARM. Linaro is a not-for-profit engineering organization with over 200 engineers consolidating and optimizing open source software for the ARM architecture in many areas. We have teams working on the GCC toolchain, the Linux kernel, power management, graphics and multimedia interfaces, networking and more.

This is our first year participating in the Google Summer of Code, and we are welcoming three students to our team. Gaurav Minocha's project is Linux Flattened Device Tree Self-checking, mentored by Grant Likely. Ricardo de Freitas Gesuatto will be working on a project entitled "Lightweight IP Stack on top of OpenDataPlane", mentored by Maxim Uvarov. Finally, Varad Gautam will be Porting UEFI to a Low-Cost Embedded Platform (BeagleBoneBlack), mentored by Leif Lindholm. We're looking forward to seeing how our newest engineers succeed this summer!

By Steve McIntyre, Linaro Organization Administrator

Cayley: graphs in Go

Four years ago this July, Google acquired Metaweb, bringing Freebase and linked open data to Google. It’s been astounding to watch the growth of the Knowledge Graph and how it has improved Google search to delight users every day.

When I moved to New York last year, I saw just how far the concepts of Freebase and its data had spread through Google’s worldwide offices. I began to wonder how the concepts would advance if developers everywhere could work with similar tools. However, there wasn’t a graph available that was fast, free, and easy to get started working with.

With the Freebase data already public and universally accessible, it was time to make it useful, and that meant writing some code as a side project.

So today we are excited to release Cayley, an open source graph database.

Cayley is a spiritual successor to graphd; it shares a similar query strategy for speed. While not an exact replica of it’s predecessor, it brings it’s own features to the table:
• RESTful API
• Multiple (modular) backend stores, such as LevelDB and MongoDB
• Multiple (modular) query languages
• Easy to get started
• Simple to build on top of as a library
and of course
• Open Source

Cayley is written in Go, which was a natural choice. As a backend service that depends upon speed and concurrent access, Go seemed like a good fit. Go did not disappoint; with a fantastic standard library and easy access to open source libraries from the community, the necessary building blocks were already there. Combined with Go’s effective concurrency patterns compared to C, creating a performance-competitive successor to graphd became a reality.

To get a sense of Cayley, check out the I/O Bytes video we created where we “Build A Small Knowledge Graph”. The video includes a quick introduction to graph stores as well as an example of processing Freebase and Schema.org linked data.

You can also check out the demo dataset in a live instance running on Google App Engine. It’s running with the sample dataset in the repository — 30,000 movies and their actors, roles, and directors using Freebase film schema. For a more-than-trivial query, try running the following code, both as a query and as a visualization; what you’ll see is the neighborhood of the given actor and how the actors who co-star with that actor interact with each other:

costar = 

g.M().In("/film/performance/actor").In("/film/film/starring")

function getCostars(x) {

 return g.V(x).As("source").In("name")

         .Follow(costar).FollowR(costar)

         .Out("name").As("target")

}

function getActorNeighborhood(primary_actor) {

 actors = getCostars(primary_actor).TagArray()

 seen = {}

 for (a in actors) {

   g.Emit(actors[a])

   seen[actors[a].target] = true

 }

 seen[primary_actor] = false

 actor_list = []

 for (actor in seen) {

   if (seen[actor]) {

     actor_list.push(actor)

   }

 }

 getCostars(actor_list).Intersect(g.V(actor_list)).ForEach(function(d)
{

   if (d.source < d.target) {

     g.Emit(d)

   }

 })

}

getActorNeighborhood("Humphrey Bogart")

To get involved, check out the project on GitHub and join the mailing list. But most importantly, have fun building your own graphs!

By Barak Michener, Software Engineer, Knowledge NYC

Google Summer of Code 2014 midterms are here!

This week marks the halfway point of Google Summer of Code 2014. Both students and mentors will be submitting their midterm evaluations of one another through Friday, June 27 as indicated in our timeline. If you would like to read more about these midterm evaluations, please check out the "How Do Evaluations Work?" link on our FAQ.

The next milestone for the program will be the “pencils down” date of August 11 after which students can take a week to scrub their code, write tests, improve calculations and generally polish their work before the firm end of coding on August 18.

There has been fantastic progress made so far, and we encourage all the students, mentors, and org admins to keep up the great work!

By Carol Smith, Open Source Team

Google Summer of Code 2014 by the numbers: Part two

Our first “by the numbers” post was about what countries this year’s accepted Google Summer of Code students are from - all 73 countries - which made for a big list. This time we’re serving up a mix of interesting stats in smaller charts.

"How old are the students?"

Google Summer of Code is for students ages 18 and older - but note the lack of an upper limit. While most of the students are relatively young, we also welcome non-traditional students to participate. This year our oldest student is 57.

“Am I the only undergraduate in the program?”
Year after year the majority of GSoC students, more than 50%, are undergraduates, but Master and PhD programs are well represented also. 2014 is no exception.

"How many women are participating in GSoC 2014?"

We are very pleased to report that just over 10% of this year’s accepted students are women. The percentage of female students has been increasing year over year since 2006, but this is the first time we’ve broken the 10% barrier.  We are obviously still a very long way from gender parity, but we’re glad this number continues to trend upward.

We will be doing additional posts about the statistics for GSoC 2014 in the next few weeks. If you have questions, please drop us a comment and we’ll do what we can to answer in an upcoming post.

By Cat Allman, Open Source Programs

FlatBuffers: a memory efficient serialization library

Today, we are releasing FlatBuffers, a C++ serialization library that allows you to read data without unpacking or allocating additional memory, as an open source project.

FlatBuffers stores serialized data in buffers in a cross-platform way, supporting format evolution that is fully forwards and backwards compatible through a schema. These buffers can be stored in files or sent across the network as-is, and accessed in-place without parsing overhead.

The FlatBuffers schema compiler and runtime is written in platform independent C++ with no library dependencies outside the STL, which makes it possible to use on any platform that has a C++ compiler. We have provided methods to build the FlatBuffers library, example applications, and unit tests for Android, Linux, OSX and Windows.

The schema compiler can generate code to read and write FlatBuffers binary files for C++ and Java. It can additionally parse JSON-formatted data into type-safe binaries.

Game developers can use this library to store game data with less overhead than alternative solutions (e.g. Protocol Buffers or JSON).  We’re excited about the possibilities, and want to hear from you about how we can make this even better!

Download the latest release from our github page and join our discussion list!

By Wouter van Oortmerssen, Fun Propulsion Labs at Google*

*Fun Propulsion Labs is a team within Google that's dedicated to advancing gaming on Android and other platforms.

My Google Code-in grand prize trip

Today’s post comes from Mateusz Maćkowski, one of the 20 grand prize winners of Google Code-in, an open source programming contest for 13-17 year old students. Mateusz came all the way from Poland to California for the trip and details the four days of technical presentations and fun activities the winners took part in.

The Beginning
I first found out that I was a grand prize winner for Google Code-in 2013 (GCI) for the Wikimedia Foundation in the middle of January, about a week after the contest ended. I then had three months for my excitement to build before my trip in April to the United States to meet the other 19 Grand Prize Winners and a mentor from each of the 10 participating open source organizations.

Day 1
The opening meet and greet dinner started the festivities and as we entered the room, we were greeted by Stephanie Taylor, Cat Allman and Mary Radomile, three of the four members of the Google Open Source Programs team responsible for organizing and preparing the contest and trip.

After spending about an hour eating and chatting with other Grand Prize Winners, their family members, and our mentors we received backpacks full of goodies (t-shirts, stickers, notebooks, a jacket, etc.) followed by a short icebreaker game. Each student received a list of personality traits or talents (such as “Can paint”, “Has a dog”, “Can speak fluently three or more languages”, etc.).  We each had to find another person who matched the particular description. It was a great way to interact with each of the other students. The winners were the two people who were able to match the largest number of people. After the game, we received more swag, and – a huge surprise to most in the room – Samsung Chromebooks!

Day 2
The next day all 50 of us piled onto a large bus in San Francisco heading to the Googleplex in Mountain View, California. When we arrived in Mountain View we cruised around the various buildings of the Google Campus before settling into our large event room for the day.

We started with a brief presentation from Stephanie on various information and statistics about GCI. After that we had our awards ceremony where Chris DiBona, Director of Social Impact and Open Source at Google, gave us each of our awards. Our mentors then presented each of us with a plaque for our achievements. We took tons of individual photos and group shots (just a few of the many to come) and then headed to lunch.

Google employees from all parts of the company and from each of the countries represented by the Grand Prize Winners joined us for lunch.  It was great to be able to talk one-on-one with a Polish Googler about their experiences in Silicon Valley. After lunch another Googler spoke about the famous Google self-driving car project.

Next up was a tour of the Google campus. The tour included some of the most recognizable places at the Googleplex, including the Android statues representing each of the Android releases. As you can see, it was a perfect spot for group and individual photos.

After the tour concluded several more Googlers gave talks about their products and services — Google Giving, Google Maps, Chrome and the open source project Samba. The last Googler talking that day was a contributor to Melange, the open source software that Google Summer of Code and Google Code-in is run on. He is a past Google Summer of Code student and has been an active contributor to Melange for several years.  

Day 3 – “San Francisco Fun Day”
We spent our third day touring San Francisco. We had the choice between two tours: a Segway tour, or a visit to Alcatraz. I chose the Segway and couldn’t have been more excited. For me, it was one of the best parts of the whole Grand Prize trip.

After the Segway tour it was time to visit the California Academy of Sciences, which is one of the largest natural history museums in the United States.

The last event of the day was a surprise — all we knew was that we’d go on an “adventure”. What an adventure and nice surprise it was! We took a yacht tour in San Francisco Bay under the Golden Gate Bridge and around Angel Island. We spent the evening talking with other students, mentors and several Google employees. Day 3 was just as cool as the previous one.

Day 4
The last day of the 2013 GCI Grand Prize trip took place at the Google office in San Francisco. It was a nice and easy walk from our hotel to the office along San Francisco’s Embarcadero which is a large walkway along the waterfront. There was a breakfast buffet waiting for us, and because it was Google, the choices were, to say the least, significant. During and after the breakfast we listened to Google speakers who talked about Google Summer of Code and the Go programming language.

We then had a short tour of the San Francisco office where we could see beautiful views of the San Francisco–Oakland Bay Bridge.

After a few additional speakers, it was finally time for what I was anticipating most — the mentors from each of the 10 GCI open source organizations gave short lightning talks (3-5 minutes) about their projects and the work the GCI students accomplished during the 2013 contest

Finally it was time to return home. Below is an image of human misery — flying away from San Francisco at night seen from the airplane window…

When people ask me about the trip my response is usually “It was fantastic until I had to return!” My final words? Participate in Google Code-in! A friend told me that I shouldn’t really care about winning, because the number of people participating is so high that I wouldn’t stand a chance. When I later told him that I was chosen as a winner, his face was “priceless”. Even if you don’t end up on the Grand Prize Trip, it is definitely still worth the time and effort. It was a great experience for me to be able to create software that is actually used by MediaWiki users from around the world as a teenager.

By Mateusz Mackowski, GCI Grand Prize Winner for Wikimedia

An update on container support on Google Cloud Platform

Cross posted from the Google Cloud Platform Blog

Everything at Google, from Search to Gmail, is packaged and run in a Linux container. Each week we launch more than 2 billion container instances across our global data centers, and the power of containers has enabled both more reliable services and higher, more-efficient scalability. Now we’re taking another step toward making those capabilities available to developers everywhere.

Support for Docker images in Google App Engine
Last month we released improved Docker image support in Compute Engine. Today, we’re building on that work and adding a set of extensions that allow App Engine developers to build and deploy Docker images in Managed VMs. Developers can use these extensions to easily access the large and growing library of Docker images, and the Docker community can easily deploy containers into a completely managed environment with access to services such as Cloud Datastore. If you want to try it, sign up via this form.

Kubernetes—an open source container manager
Based on our experience running Linux containers within Google, we know how important it is to be able to efficiently schedule containers at Internet scale. We use Omega within Google, but many developers have more modest needs. To that end, we’re announcing Kubernetes, a lean yet powerful open-source container manager that deploys containers into a fleet of machines, provides health management and replication capabilities, and makes it easy for containers to connect to one another and the outside world. (For the curious, Kubernetes (koo-ber-nay'-tace) is Greek for “helmsman” of a ship.) Kubernetes was developed from the outset to be an extensible, community-supported project. Take a look at the source and documentation on GitHub and let us know what you think via our mailing list. We’ll continue to build out the feature set, while collaborating with the Docker community to incorporate the best ideas from Kubernetes into Docker.

Container stack improvements
We’ve released an open-source tool called cAdvisor that enables fine-grain statistics on resource usage for containers. It tracks both instantaneous and historical stats for a wide variety of resources, handles nested containers, and supports both LMCTFY and Docker’s libcontainer. It’s written in Go with the hope that we can move some of these tools into libcontainer directly if people find them useful (as we have).

A commitment to open container standards
Finally, I'm happy that I've been nominated to Docker's Governance Committee to continue working with the Docker community toward better open container standards. Containers have been a great building block for Google and by working together we can make them the key building block for “cloud native” applications.

-Posted by Eric Brewer, VP of Infrastructure

Google Summer of Code 2014 New Organizations - Part One

Every year, we spend time highlighting each of the “rookie” organizations who have joined Google Summer of Code (GSoC). With over 40 new organizations to the program in 2014, we’ll dedicate Fridays this summer to spotlight their mission and goals of participating in GSoC. This week, the Organization Administrators from Amahi and Code Mirror tell us more about their organizations.

Amahi is an open source home server solution based on linux distribution, developed with the goal of making networking simple. It provides all the functionality you would want in a home server (e.g. DHCP, DNS, File Sharing), while being as easy to use as a web browser. Designed as a modular architecture, Amahi is easily expandable through one click application installs to provide additional features such as Media Streaming, VPN, Disk Pooling and more.

This is Amahi’s first year as a mentoring organization in Google Summer of Code and we were
fortunate to have three student participants. Kasun Thennakoon will work on a Disk Wizard plugin which is one of the most requested features on the platform side. This will provide an intuitive interface for adding new storage to your home server. Arpit Goyal will work on “what is next”, by upgrading the platform to the latest technology (i.e. Rails 4) and facilitate the application installation by improving the Amahi plugin system. Last but not least, we have Artur Dryomov who will work on the Amahi Anywhere android app, that will give users access to their home server data from any location without VPN or port forwarding.

By Carlos Puchol and Bogdan Mitrea, Amahi Organization Administrators

CodeMirror is a versatile text editor implemented in JavaScript for your browser. It is specialized for editing code and comes with a number of language modes and add-ons that implement more advanced editing functionality. A rich programming API and a CSS theming system are available for customizing CodeMirror to fit your application. We've had a pretty narrow developer base thus far—participating in Google Summer of Code is a great way for us to get some talent on board for the summer, and hopefully longer.

This summer we have two Google Summer of Code students. One who is working on improving bidirectional text support and the other student will work on improving the vim bindings (specifically the visual mode and undo tree).

By Marijn Haverbeke, Code Mirror Organization Administrator

A chip off the ol’ Google Summer of Code block

We here in the Google Open Source Programs Office are always excited to hear about programs that are similar (or even inspired by) our flagship student program, Google Summer of Code (GSoC). Contributing to open source software in any capacity makes us happy, and learning about students exchanged in open source development makes us just plain giddy. We’d like to use today’s post to highlight some of the great open source student programs from past and present.

• Outreach Program For Women - a program that encourages women to get involved in open source software
• Open Summer of Code - an open source program for students in Belgium
• Rails Girls Summer of Code - three-month full time scholarships to pairs of Rails Girls students, supported by coaches
• Ruby Summer of Code - a student internship program designed to help fund student development of Ruby coding projects
• European Space Agency Summer of Code - offers student developers stipends to write code for various space-related open source software projects
• Oman Summer of Code - a open source coding program based in Oman
• Summer of Learning at CMU - students learn how to program robots using free software

And don’t forget that Melange, the software used to run GSoC (as well as our contest for younger students, Google Code-in), is also open sourced. You are welcome to use it to start your own program. The source code can be found here.

Please use the comments section on the blog to tell us about your favorite Google Summer of Code-esqe program. We’d love to hear more!

By Mary Radomile, Open Source Programs