Posts from October 2010

Integrating R with C++: Rcpp, RInside, and RProtobuf

Thursday, October 28, 2010

Last week the R “intergrouplet” at Google invited Dirk Eddelbuettel and Romain Francois to come and give a Tech Talk about their joint work on Rcpp, RInside, and RProtoBuf. Dirk and Romain agreed and also let us share the video of their talk:

We were also happy to have John Chambers, creator of the S programming language and member of the R Core team, join us for the talk and discussions afterward.

At Google we use a lot of C++ and an increasing amount of R for data analysis, so the Rcpp package is very interesting to us as a modern type-safe way to write R packages that interface with other Google technologies written in C++. Likewise, the RProtoBuf package makes it easy for analysts and engineers to use R with Protocol Buffers read from Bigtable or other data stores.

By Murray Stokely, Quantitative Engineering Team

Facebook Likes Google Summer of Code

Wednesday, October 27, 2010

This summer Facebook participated in Google Summer of Code for the first time and we want to share an update on the progress our students made. Unlike most organizations participating in the program, we have a number of open source projects rather than just a single project. This meant that we accepted a few students to work across open source projects which we actively contribute to in addition to projects that we've released.

We received many applications from students all over the world who were excited to spend their summer working on adding new features to HBase, HipHop for PHP, Scribe, Three20 and XHP. While some projects are still working on merging their changes, by the end of the summer:
Chongxin Li designed and coded some initial support for snapshotting in Apache HBase that will allow for easier recovery from data loss.

Hui Chen ported HipHop for PHP to 32-bit operating systems, implemented some missing extensions, and helped improve the stability of the project. All of these changes are already in the core and some are running on the very web servers you're using right now.

Souvik Roy added regular expression support to Scribe for category names and implemented some unit tests into the existing system which makes future development easier.

Chih-Wei Lee spent his time working on adding iPad support to Three20, the UI library we released which is used by many iPhone applications. Some of this support has been merged into trunk and the rest is being reviewed.

Avgoustinos Kadis added support for new HTML5 elements to XHP. This allows PHP developers to write their HTML once and have XHP handle gracefully degradation from native HTML5 elements to JavaScript implementations in older browsers. (We wrote about how we’re using HTML5 two weeks ago.)
As part of Google Summer of Code, both the students and mentoring projects receive a stipend, and we asked that what would be given to Facebook is donated to the Apache Software Foundation instead. We really enjoyed having the opportunity to take part in Google Summer of Code and want to thank these five students for their awesome work!

By Scott MacVicar, Facebook mentor for Google Summer of Code

This post is cross posted from Facebook’s Engineering Notes

Perl and Parrot Spread Open Source Love

Saturday, October 23, 2010

The Perl Foundation and the Parrot Foundation took part in Google Summer of Code this year, and as the organization administrator, I am very proud of and humbled by all the students and mentors that I worked with. I am constantly reminded that there are very intelligent developers who are very young, and the Perl and Parrot Foundations are very lucky to attract them and have them in our communities. I firmly believe that the passing Google Summer of Code 2010 projects have had a large positive impact on our codebases and many people will benefit from them for years to come.

We were lucky to get proposals from very bright and capable students. We started the summer with 10 students and had 8 students pass their final evaluations. The passing projects include:
Ryan Jendoubi -- Ctypes for Perl
Mentor: Reini Urban (Repo)
This project is exciting many Perl developers, because it minimizes the need to use XS, which makes many more pure-Perl modules possible. This improves portability, because XS-based modules are notorious for being fragile across operating systems and compiler versions. This adds up to a whole lot of WIN.

Nat Tuck -- Hybrid Threads for Parrot
Mentor: Andrew Whitworth (Repo)
Threads allow a single program to use more than one CPU, which is becoming increasingly important these days. Even mobile phones are multicore! This work aimed to add threading support to Parrot Virtual Machine. Much was accomplished, but this effort is still on-going. So-called "green threads" were implemented, which is a necessary step to get Hybrid threads working.

Tyler Curtis -- A PAST Optimization Framework for Parrot
Mentor: chromatic (Repo)
This project is about providing a framework for optimizing PASTs (Parrot Abstract Syntax Trees). This will be used by language implementers when optimizing their HLLs (High Level Languages). This framework allows all languages on Parrot to benefit from optimizations that are written once, instead of each language implementer writing their own optimizations.

Daniel Arbelo Arrocha -- NFG and single-representation strings for Parrot
Mentor: Allison Randal (Repo)
NFG stands for Normal Form Grapheme, and basically means having a standard internal representation of Unicode strings, so that very expensive conversions do not have to repeatedly take place. This makes string-heavy computations much faster and unifies a lot of code.

Carl Masak -- Add support for binary data in Rakudo
Mentor: Jonathan Worthington
Rakudo Perl 6 now supports various binary data formats that were implemented as part of this project. Many relevant tests were also added to the Perl 6 Spec Test Suite as well as improvements and clarifications to the Perl 6 Specification.

Muhd Khairul Syamil Hashim -- Instrumentation Tool for Parrot
Mentor: Christoph Otto (Repo)
This instrumentation tool for Parrot allows developers to dynamically peek into the execution of Parrot op-codes. This allows for profiling tools that can answer questions like "who calls functions X" and "how many Objects of type X were created."

John Harrison -- Improvements to NCI/LLVM Stack Frame Builder for Parrot
Mentor: Peter Lobsinger (Repo)
This project is a prerequisite for a JIT (Just In Time compilation) framework, which is an important goal for the Parrot community, since Parrot decided that our old JIT subsystem was broken beyond repair and removed it. Parrot has decided to use the very popular LLVM project in our rewrite of our JIT, and this project brings us a step closer on our journey.

Pawel Murias -- Mildew and SMOP on CPAN
Mentor: Daniel Ruoso (Repo)
This project involved working on Mildew and SMOP. Mildew is a Perl 6 implementation, and SMOP is the library Mildew uses for meta-object programming. You can think of Mildew as a sister to Rakudo Perl 6. Having many implementations of Perl 6 helps to better define the Perl 6 specification. Updated versions of SMOP and Mildew are now available on CPAN.
The two projects that did not pass the final evaluations were:
Justin Hunter -- Rework Catalyst framework instance initialization code
Mentor: Florian Ragwitz

Mirko Westermeier -- Bulletproofing the Mojolicious test suite
Mentor: Marcus Ramberg
Both of these projects passed their midterms, but due to circumstances outside of the program, these students were not able to complete their goals for their final evaluation. Sometimes Real Life throws you a curve ball, like starting a new job, moving to a new city, having a baby or similar things. We wish these students the best of luck, and hope that they complete their projects outside the structure of Google Summer of Code.

To all that participated in Google Summer of Code - Rock on and keep spreading the open source love!

By Jonathan “Duke” Leto, Google Summer of Code Administrator for Parrot and Perl

Cross posted from, my personal blog about Perl 5, Perl 6, and Parrot Virtual Machine.

Greetings from Santa Kurara, Kariforunia

Tuesday, October 19, 2010

Hello from the Unicode Conference in Santa Clara, California, where the Maps Transliteration team is giving a talk about ICU-based transliteration. Transliterating this originally Spanish city name to Japanese, we get サンタ・クララ, which (when morphed back to the Latin writing system) becomes “Santa Kurara.”

Machine Transliteration is an active area of research (slides), which means it can be rather challenging in general. Typically, transliteration emulates the pronunciation, but sometimes it also preserves some aspects of the original written spelling. We created transliteration modules with the open-source ICU library for languages that have highly regular spelling; if you’re using Google Maps in Japanese, Russian or Chinese, you can see how we use it to display labels in both the local language and your own:

Today, we’re announcing the contribution of our ICU transliteration rules for Czech, Italian, Japanese, Korean, Mandarin, Polish, Romanian, Russian, Slovak and Spanish to the Unicode Common Locale Data Repository. (For languages with very irregular spelling, like English, we supplement ICU with some more advanced techniques.) If you would like to try writing rules for your own language, have a look at the instructions in the ICU user guide.

アスタ・ラ・ビスタ — “Asuta ra bisuta,” from sunny “Kariforunia!”

By Sascha Brawer, Martin Jansche, Hiroshi Takenaka, and Yui Terashima, Maps Transliteration Team

Geek Time with Rusty Russell

Monday, October 18, 2010

Free software advocate and Linux developer Rusty Russell sat down for an interview with Google’s Jeremy Allison when they were both in Japan for LinuxCon. They discuss Rusty’s role maintaining the Linux kernel (0:15), why Australia has produced so many top notch open source developers (3:13), and suggestions for people starting out in their careers and looking to get into open source (9:13). Enjoy!

Thanks to Simon Horman for operating the camera.

By Ellen Ko, Open Source Team

Another Successful Summer for Haiku

Wednesday, October 13, 2010

For the past four years, the Haiku Project has had both the honor and privilege of being a mentoring organization in Google Summer of Code. This has been a wonderful opportunity for the project, as it exposes us to many young and energetic minds that are interested in developing open source software. Even more exciting, it provides a unique opportunity for the project to generate income while growing a handful of carefully selected students into knowledgeable and potentially long-term contributors.

Seven students were allotted for the Haiku Project in 2010 and of those, five students successfully passed the program! More important than the short-term contributions of code is the fact that the students are now more experienced open source developers -- specifically for Haiku. During the program, one student (Alex Wilson) was granted commit access to the Haiku Project, and several other students have expressed plans to continue contributing in the future. As an open source project that develops an operating system, our pool of active committers is relatively small and being able to embrace new contributors is a wonderful thing.

Student Summaries
Atis Elsts, Implementing IPv6 (Code)
At the moment, IPv6 is not included in default images. However, basic IP address assignment, ICMP, and ping6 are working. Some code, which includes NetServer and ifconfig has yet to be merged and is accessible at Atis’s github account: haikuipv6.
Atis plans to continue working on the existing problems in the network stack (e.g., #6502) and to eventually bring IPv6 to full working status.

Janito Vaqueiro Ferreira Filho, Implementing ext3 support (Code)
Ext3 Indexed Directory Lookup was committed in r37295 and was needed for implementing read support. For write support, the basics of journaling is implemented, but needs more thorough testing. The majority of his code was committed earlier as r38573. In addition to working on ext support, Janito fixed some issues relating to BFS and the block cache (r37899, r38103).

Christopher Humphries, Improving and Extending Media Player (Code)
Shortly after the start of the coding period, Christopher received news that his university had retracted their word on accepting Google Summer of Code as a valid internship. This necessitated changes to his project proposal -- specifically reducing the scope of the project to DVD support. Originally, he started out making a media plug-in, but finally opted for an add-on. The latter provided better interfacing and control for applications -- mostly because the library needs path information. Other problems cropped up, such as the absence of a demuxing node and faulty mpeg decoding. This led to the realization that the kit needs more work before it can do the job. Christopher has even presented some thoughts to the community: Pitching Media Kit ideas, Alternatives to plug-in sniffing, and A true streaming equivalent of BDataIO. His code is hosted on his github account, Haiku-DVD-addon.

Christophe Huriaux, Creating Services Kit core elements (Code)
As mentioned in his recent blog post, the Services Kit is now capable of handling HTTP requests (file uploading, POST requests, authentication, cookie support, etc). It supports asynchronous as well as synchronous requests. Christophe has also started updating WebPositive to utilize Services Kit. This provides a real world test-bed scenario and will improve WebPositive by replacing the cURL backend with native API.

Alex Wilson, Taking the Haiku Layout API public, (Code)
This involved archiving of BLayout, BLayoutItem and subclasses (#5525), deriving BLayout from BLayoutItem (#6407), and updating numerous applications to use the new Layout API. In addition to his original proposal, numerous archiving features were implemented as part of #6256 (BArchiver, BUnarchiver, BArchivable). Alex was also granted commit access halfway through the coding period! In the future, he plans to working towards making the Layout API suitable for public consumption.

In closing, many thanks to Google for sponsoring this program and allowing the Haiku Project to participate, to Carol Smith, Google Summer of Code Program Administrator, and to our mentors who helped make this endeavor successful. All in all, this was another successful year for Haiku. Congratulations to all who have participated!

By Matt Madia, Haiku Administrator for Google Summer of Code

This post is cross posted from the Haiku blog, where there are more details about our participation in this year’s Google Summer of Code.

Geek Time with James Bottomley

Monday, October 11, 2010

When people first meet Linux Kernel SCSI subsystem maintainer James Bottomley, one question that often comes up is, “Why the bow tie?” Jeremy Allison, Samba co-founder and Google Open Source Programs Office team member, spent time with James at LinuxCon last month and uncovered the answer. After that, Jeremy gets the backstory on the inspiration behind James’ LinuxCon talk about how to convince management to embrace open source (4:58). James also talked about how he became the SCSI subsystem maintainer (1:54) and how he started working on open source (6:47).

Thanks to Sergio Victorino for operating the camera.

By Ellen Ko, Open Source Team

Google Code-in: School’s Out, Code’s In!

Thursday, October 7, 2010

We are very pleased to announce Google Code-in, an open source development and outreach contest targeted at 13-18 year old students around the world.

Some of you may remember the pilot program from 2007-2008 we ran called the Google Highly Open Participation Contest. This contest gave 400 students around the world an opportunity to help out open source projects on the following kinds of tasks:

1. Code: Tasks related to writing or refactoring code
2. Documentation: Tasks related to creating/editing documents
3. Outreach: Tasks related to community management and outreach/marketing
4. Quality Assurance: Tasks related to testing and ensuring code is of high quality
5. Research: Tasks related to studying a problem and recommending solutions
6. Training: Tasks related to helping others learn more
7. Translation: Tasks related to localization
8. User Interface: Tasks related to user experience research or user interface design and interaction

It was a huge success, and we’re looking forward to another great year ahead of us with the contest with its new name, Google Code-in.

Be sure to check out our Frequently Asked Questions about the contest for answers to your questions about participating.We’re hoping to get pre-university students from all over the world involved. So please help us spread the word.

Stay tuned to this blog and to our mailing list for more updates on the contest. We will announce the mentoring organizations that are participating on November 5. The contest starts on November 22, 2010!

Google Summer of Code report: WorldForge

Wednesday, October 6, 2010

For the third time in a row, Worldforge participated in Google Summer of Code, with three students completing the program this year. Worldforge is the original open-source Massively Multiplayer Online Role-Playing Game (MMORPG) project, so it’s great at getting students who are interested in games into open source.

This post showcases some of the work done by one of our students, Tiberiu Paunescu, to implement a series of improvements to the Ember UI. These improvements were all end-user focused and meant to provide a better and more streamlined user experience.

The first feature implemented was an improvement to the inventory which makes it possible to drag items from the inventory widget into the world, and place them there. When an item is dragged off the inventory and into the world a preview model is created, showing where the item will end up.

The second feature Tiberiu worked on was a new quick-help system. This is designed to provide context-sensitive help messages to the user, triggered by various actions performed. The overall goal is to make it easier for new users to quickly get immersed in the world, and to get proper help messages when they are needed.

The last feature implemented is a new action bar framework. The action bar allows the user quick access through either the keyboard or the mouse to frequently-used items and actions. Action bars can be created and destroyed on the fly, and items can be dragged from the inventory onto an action bar to create a new binding.

Action bars are also property persisted between server sessions.

As you can see from the videos, these are all solid and very useful features which greatly improve the user experience. Tiberiu is a great example of the amazing contributions that can be made in just one summer, we’re all very proud of what Tiberiu has achieved.

By Erik Hjortsberg, WorldForge Mentor

This post is cross posted from the WorldForge Developers Journal.

Geek Time with Ric Wheeler

Monday, October 4, 2010

Ric Wheeler is the File System Group Manager at Red Hat, and Jeremy Allison caught up with him at LinuxCon in Sao Paulo, Brazil earlier this month. Ric tells Jeremy how he got into file system development as a grad student, then how he progressed into building storage arrays, eventually becoming a Linux advocate. From there, Jeremy and Ric talk about the direction that Linux is headed and the future of desktop computing. At the very end of the video, you can even hear about Ric’s brush with Hollywood!

Thanks to Sergio Victorino for operating the camera.

By Ellen Ko, Open Source Team