CernVM’s fruitful summer

This was the first year CERN participated in Google Summer of Code, and it turned out to be an amazing experience for us! We were given four students to mentor, all of whom proved to be very skilled developers. The students quickly familiarized themselves with our code base and managed to make valuable contributions within the three month time frame of Google Summer of Code. Our students were very open and willing to learn and spent a considerable amount of their time researching tools, libraries, and the latest technological developments. As a result, all four students were able to solve their problems and come up with interesting ideas for future development. The code and the documentation they produced is available here. The specific problems (projects) that we suggested to our students spanned several domains, ranging from consistent replication of terabytes of data across several remote sites to automated testing of virtual machine releases.
                                           

Josip Lisec was working on the development of the monitoring system for the CernVM Co-Pilot framework, which is mainly used as a distributed computing platform within the LHC@home 2.0 volunteer computing project. The LHC@home 2.0 project currently has more than 9,000 registered users who contribute their spare CPU cycles for the simulation of the particle collision events in CERN's Large Hadron Collider (LHC). After some research, Josip decided to integrate existing tools with the Co-Pilot as opposed to trying to reinvent the wheel by rewriting everything from scratch. This resulted in a nicely engineered monitoring framework, parts of which were put into production while the Google Summer of Code was still going on (Josip's developments have now been fully integrated after completion of the program). Since this was Josip's first encounter with Perl, he has been seen adding support for 'my' keyword to every other major programming language since the Google Summer of Code concluded.

The goal of Yin Qiu's project was to devise a mechanism for a consistent replication of changes made to the central repository of CernVM File System (CernVM-FS) to a globally distributed set of mirror servers. CernVM-FS is used to host and distribute the application software of CERN LHC experiments to hundreds of Grid sites, as well as the laptops and workstations of users worldwide. As such, it is currently one of the central components of the distributed computing infrastructures on which CERN ATLAS and LHCb experiments rely. Yin's approach was to organize CernVM-FS mirrors into a Paxos-managed replication network and to enforce state machine version transitions on them. Following the suggestion of Jakob, his mentor, Yin implemented a messaging framework which is used to orchestrate the replication process and facilitates the implementation of new features. He also managed to implement a couple of Python plugins which ensure the consistency of data across replicas. The project is currently in the state of a working prototype.

Jesse Williamson took up the challenge of designing a new library for CernVM-FS to consolidate support for various cryptographic hashing algorithms. The first task was to survey the implementation of CernVM-FS and establish a list of requirements. Next, quite a bit of effort was spent on designing the library specifically so that it would be easy to use, comparatively simple to extend, and robust enough to support extensions like a streaming interface and compression. Since CernVM-FS is heavily used in production, it has been very important to make sure that the new developments do not break anything. Jesse has developed a set of unit tests which ensure that all the existing features and properties were maintained.

The design of new C++ libraries was certainly an improvement, but it also became clear late in the cycle that a further abstraction to fully separate digests and hash functions will be necessary to avoid memory fragmentation issues and ensure stronger const-correctness

Jonathan Gillet worked on implementing a solution for automating the testing of CernVM virtual machine images on multiple hypervisors and operating systems. The solution, which is a ready to use testing infrastructure for CernVM, was developed in collaboration with other open source projects such as AMD Tapper (used for the reports and web interface), libvirt (interaction with hypervisors), and Homebrew (OS X support).  The main goals of the project were accomplished with support for all major hypervisors running on Linux and OS X platforms. The framework automates the task of downloading and configuring the CernVM images on the fly, and executing a series of thorough tests which check various features of CernVM images before release. Documentation was also an important goal of the project; in total there are now over two hundred pages of documentation which cover everything from setting up the testing infrastructure and virtual machines to a complete API reference.

We certainly enjoyed Google Summer of Code 2011, and we sincerely congratulate all of our students and mentors for successfully completing the program!

By Artem Harutyunyan, Senior Fellow, CernVM Project (CERN) and Google Summer of Code Mentor

How the world was open sourced

Once in awhile at Google our illustrators get excited about lasers, Morse code, H. G. Wells’s The War of the Worlds – and then come up with beautiful Google doodles that find their way onto our homepage. Sometimes our programmers also get excited and team up with the illustrators, and that’s how we found ourselves with Google doodles celebrating Les Paul’s guitar, Pac-Man, Jules Verne’s bathyscaphe, and even your own customized turkey that you could then share on Google+.

I’m one of those people who is more comfortable with 80 monospaced characters endlessly repeated than with a paintbrush. Earlier this year I worked with Sophia Foster-Dimino from the Google doodle team on a doodle celebrating Stanisław Lem, my favorite sci-fi writer and philosopher.

 

Just like picking the right paintbrush and palette is important for all our doodles, so is figuring out the right technologies and proper user interface for those we want to make interactive. That’s something I’m personally really excited about and that’s why today I wanted to share that excitement and the entire source code of the Stanisław Lem doodle with you – accompanied with an article explaining HTML5 technologies that we used… or didn’t use:

• read the article about Stanisław Lem doodle on HTML5 Rocks
• view the complete source code 

Please note: We are sharing the code of the doodle under the Apache 2.0 License, but the images and animations accompanying the doodle under the Creative Commons BY-NC-SA 3.0 License. The big difference between those two is that the first one allows commercial re-use, whereas the second one forbids it.

So take it for a spin, play with it, and if you do something interesting, find a flaw, or have a comment – let us know at stanislaw-lem-google-doodle@googlegroups.com. Thanks!

By Marcin Wichary, Senior user experience designer, Chrome

Teaching with Mobile

Ever wish you could get your students to stop texting and start using their phones productively in the classroom? Do you ever wish for an easy and quick way to measure what your students have learned? If so, you’ll be excited to learn that we recently open-sourced our internal Quiz & Poll App for Android. Developed by our internal learning systems team, “Quiz & Poll” enables educators to engage and challenge their students inside the classroom (using polls) and outside the classroom (using quizzes).

How it works
With Quiz & Poll, teachers & educators can create quizzes and polls easily using Google Spreadsheets. Quizzes and Polls are distributed to students via the Android app or an embeddable webplayer. Statistics data is written back to the spreadsheet so the instructor can track student responses and understanding.

Key Features

Quizzes and polls are easily created and administered on Google Spreadsheets. 

Students can download the Android App to take the quiz or live classroom poll. 

For students without Android phones, quizzes and polls can be accessed via a web interface (works on laptops, iPhones, iPads etc.).

Administrators can view quiz and poll statistics on the same Spreadsheet.

The quizzing app and leaderboard

How it’s built

The Quiz & Poll application showcases how you can combine several Google technologies to make an easy to use app. The code is useful for developers curious about following technologies and how they interact:

• Android app communicating with backend on AppEngine (Python+Django)
• AppEngine backend is communicating with GData Spreadsheets API
• Google Apps Script (used for interface to administer polls)
• Closure Javascript framework (used for web player)

Try it out

To see the app in action watch our how-to demo videos:

Polling   or    Quiz Games

If you’d like to explore the project code and documentation, check it out on Google Code, where you can also find a sample implementation that you can try out. 

Happy quizzing!

By Bridget Campbell, The Learning Technology Team

Only Halfway to the Finish Line for Google Code-in

Students add to SymPy

SymPy is a computer algebra system (CAS) written in pure Python. The core allows basic manipulation of expressions (like differentiation or expansion) and it contains many modules for common tasks (limits, integrals, differential equations, series, matrices, quantum physics, geometry, plotting, and code generation).

SymPy has participated in the Google Summer of Code program in previous years under the umbrellas of Python Software Foundation, Portland State University, and the Space Telescope Science Institute, where we were very successful. In fact, several of our core developers, including four of the mentors from this year, started working with SymPy as Google Summer of Code students. This was our first year participating as a standalone organization, and we would like to share our experience.

As part of the application process we required each student to submit a patch (as a GitHub pull request) that had to be reviewed and accepted. This allowed us to see that each applicant knew how to use git as well as communicate effectively during the review process.This also encouraged only serious applicants to apply. We had over 10 mentors available and we ended up with 9 students, all of whom were successful at final evaluations.

Tom Bachmann - Definite Integration using Meijer G-functions, mentored by Aaron Meurer

Tom implemented an algorithm for computing symbolic definite integrals that uses so-called Meijer G-functions. This is the state-of-the-art algorithm for computing definite integrals, and indeed the results of his project are very impressive. This project has pushed SymPy forward a long way to becoming the strongest open source computer algebra system with respect to symbolic definite integration.

Vladimir Peric - Porting to Python 3, mentored by Ronan Lamy

Vladimir ported SymPy to work on Python 3 and ported all testing infrastructure so that SymPy gets regularly tested in Python 2.x, 3.2 and PyPy. Thanks to Vladimir’s work, the next version of SymPy, 0.7.2, which will hopefully be released later this year, will work in both Python 2 and Python 3, and it may support PyPy as well.

Gilbert Gede - PyDy, mentored by Luke Peterson

Gilbert implemented a physics module to assist in generating symbolic equations of motion for complex multibody systems using Kane's Method. He expanded on the code written by his mentor, Luke, in 2009, and the module can now generate equations of motion for a bicycle. Gilbert also wrote very thorough documentation both for the Kane’s Method and the module in SymPy.

Tomo Lazovich - Position and Momentum Bases for Quantum Mechanics, mentored by Brian Granger

Tomo has greatly improved the quantum mechanics module by implementing position/momentum representations for operators and eigenstates in various coordinate systems (including cartesian, cylindrical, and spherical) that allows you to easily represent many of the "textbook" quantum mechanics systems, including particle in a box, simple harmonic oscillator, hydrogen atom, etc.

Saptarshi Mandal - Combinatorics package for Sympy, mentored by Christian Muise

Saptarshi’s project was to mimic the various capabilities of Combinatorica, a Mathematica package for combinatorics. Most of the functionality involving elementary combinatorial objects such as Permutations, Partitions, Subsets, Gray codes and Prufer codes are complete.

Sherjil Ozair - Symbolic Linear Algebra, mentored by Vinzent Steinberg

Sherjil improved the speed of the linear algebra module by using efficient coefficient types for values of entries of matrices. Previously, SymPy used generic expressions in this place, which slowed down computations considerably and caused trouble with solving of the zero equivalence problem. He also implemented sparse matrix representation and unified the API with dense matrices. In addition, Sherjil also added a few linear algebra related algorithms (e.g. Cholesky decomposition).

Matthew Rocklin - SymPy Stats: Random Variables, mentored by Andy Terrel

Matthew improved the statistics module to use symbolics and introduced a Random Variable type, with support for finite, continuous, and multivariable normal random variables. With these you can symbolically compute things like probabilities of a given condition, conditional spaces, and expectation values. As a side consequence of this project, he also improved some of our Sets classes and implemented a MatrixExpr class, which allows you to compute with matrices symbolically, including computing with block matrices.

Sean Vig - Symbolic Clebsch-Gordon coefficients/Wigner symbols and Implementing Addition of Spin Angular Momenta, mentored by Ondřej Čertík

Sean was working on the quantum mechanics module and has implemented symbolic Clebsch-Gordan coefficients, Wigner D function, and related mathematical concepts that are used very often in quantum physics when dealing with angular momentum and then the necessary classes to support coupled spin algebra.

Jeremias Yehdegho - Implementing F5, mentored by Mateusz Paprocki

Jeremias worked on implementing algorithms related to Groebner bases. Groebner bases are a useful tool in many areas of computer algebra. He implemented the F5B algorithm, which is an improved version of the classical Buchberger’s algorithm that was previously implemented in SymPy, and an algorithm for converting Groebner bases between different orders of monomials and worked on applications of Groebner bases. This allowed for handling problems of much larger size in SymPy.

The full report can be found here, where each student wrote a wiki page about their experience during the summer and you can also find their blogs and links to applications. Each student was required to blog about their progress each week and all blogs were synchronized at planet.sympy.org.

In previous years, there was usually one student from each summer who became a regular contributor and also a mentor for the next year. It has been a rewarding experience for the whole SymPy community.

By Ondřej Čertík, Aaron Meurer and Mateusz Paprocki, SymPy Google Summer of Code Mentors

GitTogether 2011

It has become an autumn tradition to host GitTogether, two days of unconference where Git developers, enthusiasts, and users gather at Google’s headquarters in Mountain View, California to share ideas on future directions of Git and to discuss the tools that support the larger ecosystem. This year was no exception. More than 55 people showed up for GitTogether 2011, and one thing that is notable is that at many of the attendees’ $DAY_JOB, the combination of Git on the desktop with Gerrit Code Review on the server side has become the standard set-up to run development work.

Photo by Brandon Casey

Google’s Dave Borowitz gave a talk about the recent addition of Git support to Google Code Project Hosting. Google also recently moved the Git server that hosts the Android open source community from kernel.org to an in-house Git server, which was the topic of Shawn Pearce’s talk. Junio C Hamano briefly described additional workflow elements being designed to help the kernel developers ensure the authenticity of the patch-flow after the recent break-in incident at kernel.org.

Since last year’s GitTogether, submodule support has been heavily enhanced. Jens Lehmann and Heiko Voigt, two Git developers who have been primarily working on the submodule support, gave an overview of where we are and what the future entails. The general direction is to make it easier to run various operations from the top-level superproject and have it go into submodules recursively when the project and the user choose to. There has been interest in using the native submodule support to update the implementation of “repo” wrapper used in Android and other projects. It seems that we are getting closer.

Support for large blobs that would not fit in the memory has been always lacking in Git. There recently has been a lot of work in the native support (e.g. storing them straight to the object store without having to read and hold the whole thing in core, checking out from the object store to the working tree without having to hold the whole thing in core, etc.). There are a few third-party tools and approaches with their own pros-and-cons, but it was generally agreed that adding a split-object encoding like Avery Pennarun’s “bup” tools uses would be the right way to help support object transfer between repositories to advance the native support of large objects in Git further.

The animated diff in Roberto Tyley’s Agit (git running on Android) demo was cool and received warm applause.

In addition to the technical discussions, we had a chance to get to know each other socially, enjoying lunches in the conference room (unlike the previous years, the group has grown too large to visit Google cafes without disturbing other Googlers) and dinners at some local restaurants. True to the tradition, there was a “patch of questionable value” to add different hash algorithm to Git in case SHA-1 hash collisions are found.

By Junio C Hamano, Open Source Programs

Students write code and save lives with OpenMRS

Earlier this year OpenMRS participated in Google Summer of Code, a worldwide program organized by Google’s Open Source Programs Office to expose university students to the world of free and open source software, and encourage those students to become long-term contributors to projects that interest them. OpenMRS is a web-based medical record system originally designed for use in the developing world, and is now used on every continent on the globe. OpenMRS is used in all kinds of health care environments, from research laboratories to hospitals to small clinics in remote villages, and even via mobile devices that collect data door-to-door.

OpenMRS has been participating in Google Summer of Code every year since 2007, and our 5th year was arguably our most successful yet. This year, 15 motivated students successfully completed projects to focus or extend the OpenMRS health care IT platform in ways that will have significant impact for our global community of users. Throughout the summer our students became full contributors in good standing in the OpenMRS community. They presented their projects’ work in progress to other developers and users and often contributed their code to our software releases to support health care professionals saving lives around the world. Unlike many other summer internships that students may have during the summer, our students were responsible for planning and delivery of “real-life projects” that came from needs and requests from people installing and using OpenMRS.

Some projects were dedicated to improving the core OpenMRS platform, and some built add-on modules to support specific types of clinical activities. There were projects focused on making the installation of OpenMRS easier, and others focused on helping improve collaboration for our volunteer community. And if the presentations our students made this semester were any indication, all of the projects were exciting ways to write code and save lives. There’s not space here to describe each project in detail, but we encourage you to check out our students and their projects on the OpenMRS Wiki and learn more about them:

• Balachandiran Ajanthan created an add-on module to deploy reusable “SMART” health care apps inside OpenMRS.
• Christopher Zakian reimagined a “universal” search within OpenMRS that allows users to search for any system data from anywhere within the system
• Gaurav Paliwal created an add-on module to allow OpenMRS users to provide application feedback to their system administrators and the larger open source community.
• Gauthami Pingili improved both the UI of the OpenMRS Patient Matching module and improved its accuracy of finding duplicate patients.
• Goutham Vasireddi helped make it faster and easier for developers to write add-on modules for OpenMRS by creating a “wizard” for Maven.
• Jelena Skorucak reworked the attributes a person has within OpenMRS, giving clinics the flexibility to record more information about the persons.
• João Portela made significant improvements to our HTML Form Entry editor, allowing non-programmers to create more detailed, useful data collection forms for health care.
• Piotr Bryk enhanced our Metadata Sharing module to make it easier to manage the export and import of OpenMRS system configurations.
• Rahul Akula’s work helped make it possible for OpenMRS to interoperate with external laboratory information systems.
• Sai Manohar Nethi worked to create a framework for a comprehensive Human Resource add-on module for OpenMRS, allowing the system to help manage clinic personnel.
• Sreya Janaswamy created a way for OpenMRS users to translate phrases used by the application into other languages, inside the application itself.
• Sriskandarajah Suhothayan created a way for the OpenMRS Notifiable Condition Detector module to watch for certain large-scale patterns and send notifications to clinicians via SMS or e-mail.
• Suranga Kasthurirathne created a new way for OpenMRS to store clinical observations that reference other people or locations.
• Taras Chorny built a system to allow OpenMRS to be installed and upgraded using a variety of languages.
• Victor Chircu built an “Atlas” add-on module that allows OpenMRS users to opt-in to report their location, type of clinic, and number of patients on a shared map to represent the active OpenMRS community.

Since we started participating in Google Summer of Code, we’re very proud that so many of our students have stayed active in the OpenMRS community and continued to contribute their talents after the program ended. In fact, three of our students have gone on to become full-time OpenMRS developers paid by various organizations involved in our community.

We continue to be more and more impressed with the students who are interested in our work, and are proud to welcome them into the OpenMRS family! In fact, this year, 2011 Google Summer of Code student Suranga Kasthurirathne was able to join us in October for our annual OpenMRS implementers meeting in Kigali, Rwanda. Suranga provided some excellent feedback about his involvement in Google Summer of Code this year, and about his experience meeting the OpenMRS community face to face. Read his blog post for more of his thoughts.

Once again, this year we were blown away by our amazing students during Google Summer of Code. Thanks to both our students who participated, and our community mentors who gave much of their time and talents to help spread the good work of the OpenMRS community!

By Michael Downey, Google Summer of Code Program Administrator for OpenMRS