Google Summer of Code Veteran Orgs: Monkey Project

Monkey Project is an open source organization that focuses on the development of web backend technologies with products such as Monkey HTTP Server: a fast and lightweight HTTP Server for Linux and Embedded Linux, and Duda I/O: an event-driven and scalable web services framework in C.

This is our second year participating as a mentoring organization for the Google Summer of Code program and this year we were given four student slots. During the evaluation period we chose the students not just by their technical skills, but also their motivation, previous contribution with the organization, capacity to learn and ability to adapt to change.

After months of hard work, all students accomplished their main goals and they worked very hard to learn networking programming and different areas of the architecture of our HTTP server. Below are descriptions of the four projects our students worked on this summer.

 - Monkey Architecture changes (Future Monkey proof for new protocols): Monkey HTTP Server is an event driven web server with a strong focus in Linux and its current architecture allows to extend the core features through plugins. But with the incoming improved protocols such as SPDY and HTTP/2.0, it requires a major refactoring to support multiple protocol layers in the proper way to avoid sacrifice performance. On this project, Sonny Karlsson is busy working on refactoring the plugin API and different scheduler mechanisms, the work completed is very promising and will be merged with mainline code shortly.

 - Monkey Caching plugin: the Monkey server is very fast but it can still perform better by reducing some I/O, so this summer we received a good proposal to add caching support to the server through the plugin interface. Hamza Zia did research and tested different caching mechanisms and has already implemented the first caching prototype which is showing good performance improvements.

- Monkey Proxy Reverse plugin: a HTTP server not only needs to serve static content, it is mostly a proxy for different resources. This project aims to create a Proxy Reverse plugin with different balancing algorithms to add a security layer in front of backend servers and redirect requests per different configuration schemas. Nikola Nikov has implemented all required components to accomplish this goal, the plugin is in a good state and already functional for testing purposes.

Nikola Nikov's workstation

 - Duda I/O - MariaDB/MySQL and PostgreSQL support: Duda I/O is a web services framework built on top of Monkey, it is an event driven extensible stack in C for high performance web applications. This goal of this project is the creation of two packages to add support for asynchronous connectivity to MariaDB/MySQL and PostgreSQL databases. Zeying Xie has done a very good job adding support for the relational databases mentioned. Now we support NoSQL and common DBs.

It is very important to us to keep our organization and community informed about "how" and "what" is being done by our students through the Google Summer of Code program. If you are interested in more technical details about each project, we invite you to follow the blogs (linked to the student’s names above) that are updated on a weekly basis.

By Eduardo Silva, Monkey Project - Google Summer of Code Organization Administrator

GanetiCon 2013

This year Ganeti, the cluster virtualization manager that Google released in August 2007, celebrated its sixth birthday. Over the years, there have been many improvements and changes to Ganeti as the user community continues to grow.

During the first week of September, Google, GRNET and Skroutz organized GanetiCon 2013, the first Ganeti user and developer conference, in Athens, Greece. Over the course of the three-day conference, 50 people met to discuss Ganeti and the future: how they use it, and what they'd like to see come out of it, with workshops on features and discussions between community members.

Among the topics we identified as key for Ganeti were:

- Better fleet management for environments with multiple clusters
- Improvement of OS installation and image management
- Better disk support
- Security improvements and reduction in the use of the root account
- Increased support for different types of "partial shutdown" situations

The community also presented software based around Ganeti. Over the years, two web managers were developed, Ganeti Web Manager and ganetimgr, as well as a full openstack compatible cloud based on Ganeti, Synnefo. Organizations gave presentations on their usage of Ganeti including the Debian Project, the Oregon State Open Source Lab (OSUOSL), Google, GRNET and Skroutz.

We are very excited about the feedback and community appreciation we received for organizing the event and we look forward to working on the most important issues, and to organize a new event in 2014!

By Guido Trotter, Ganeti

Google Summer of Code Veteran Orgs: 52 North

We recently wrapped up our summer blog series highlighting new Google Summer of Code organizations. Now it’s time to feature some of the 130+ veteran organizations that participated in Google Summer of Code 2013. Each Friday for the next few months, we will spotlight a veteran organization and the projects their students worked on over the summer. To kick it off, the organization administrator from 52°North talks about their project and their second year as a mentoring organization.

The open source software initiative 52°North is an open international network of partners from research, industry and public administration. Its main purpose is to foster innovation in the field of Geoinformatics through a collaborative R&D process. The R&D process takes place within 52°North R&D communities, which develop new concepts and technologies (examples include: for managing near real-time sensor data, integrating geoprocessing technologies into SDIs, and making use of GRID and Cloud technologies). These communities evaluate new macro trends, such as the Internet of Things, the Semantic Web or Linked Open Data, and find ways to unfold their use in practice. 

As a returning Google Summer of Code mentoring organization, 52°North was particularly thrilled to have four students work on different projects in the fields of Sensor Web, Geostatistics and Geoprocessing this summer. 52°North’s overall goals for the projects were to improve the usability of the products and extend the user base to new domains. 

Khalid Alqinyah created a new admin web application for the 52° North Web Processing Service (WPS). This new application will improve the WPS back end code with an updated implementation and provide the front end with a new and ergonomic user interface. 

Patrick Noble added support for seismic data within the 52° North Sensor Observation Service. This seismic data will then be visualized by the Sensor Web Client. The result of his work will be a comprehensive toolkit for publishing and visualizing seismic data.
Jinlong Yang developed classes and methods that enable trajectory analysis in the R language. We expect this enhancement to become a valuable tool for all researchers dealing with trajectory data as it will open up R for a whole range of new application scenarios. 

Mohammad Ahmed Hamed Yakoub implemented the Open Sensor Search (OSS) idea in an agile software development process. OSS is a platform for the discovery of in-situ sensor data. The resulting discovery solution will become a cornerstone within the 52°North Sensor Web framework, allowing the development of more sophisticated client applications. 

By Jan Schulte, 52°North Organization Administrator

Currently, students are busy putting the finishing touches on their projects and starting next week they will submit the code they have been working on these last 3 months. Good luck to all the students in these final days of Google Summer of Code 2013.

By Stephanie Taylor, Open Source Programs

Coder: a simple way to make web stuff on Raspberry Pi

Cross posted from the Google Developers Blog

I wrote my first two lines of BASIC as a five year old, and I’ve loved programming ever since. These days I’m teaching my own kids to code, and I’ve been thinking a lot about how they could have that same simple experience while learning to program for the web.

A quick sprint with my design friend +Jeff Baxter led to a little experiment called Coder. It’s an open source tool that turns Raspberry Pi into a simple, tiny, personal web server and web-based development environment – just what you need for crafting HTML, CSS, and JavaScript while you’re learning to code. Here’s a quick video that helps explain how it all works (and a huge shout out to our friends +Claire Stapleton and +Minji Hong who helped us with it):

We thought about all the stuff we could do to make Coder a more complete package, but we have a hunch that the sooner this gets into the open source and maker communities, the more we’ll learn about how it might be used. Hopefully, a few more folks will pitch in and help us make this even more accessible and helpful for new coders.

To learn more, visit goo.gl/coder, get a Pi, download Coder, and make something awesome.

By Jason Striegel, Creative Technologist, Google Creative Lab

Hesokuri - a personal backup solution for your source code

If you are a developer, source code is some of the most important data you have. It needs to be backed up regularly, it must be readily accessible from all of your machines, and it may even be confidential. Most of the code is probably already stored under version control.

With this in mind I developed Hesokuri, a background process that keeps Git repositories on multiple machines in sync. Changes are pushed aggressively to peers as they are committed. When a peer is offline, Hesokuri will retry a push regularly until the peer responds. In some cases, a peer will merge pushed commits into the current branch so they are immediately visible in any open text editor. This means that if Hesokuri is running on two or more networked machines, the Git repositories on them are duplicated, backed up, and widely accessible.

Once you have set up Hesokuri and written a simple configuration file, you can just use Git as you always have. Hesokuri also has a web interface so you can check what revisions of each repository have been pushed to each peer.

We are still working on the user interface and the list of features that we want to include, so thanks to the community in advance for your bug reports, pull requests, and other contributions!

For the source code and how to get started, check out the Hesokuri project page. You can also get help using Hesokuri and discuss improvements in the Google discussion group.

By Matthew DeVore, Google+ Team

Open Source Security

Free and open source software is a crucial part of the security infrastructure of the internet. As a Googler working on security issues, especially those involving open source software, I want to describe some of the open source security projects Google is involved with. This list is far from comprehensive, but comprises some of the projects I have personally had a hand in.

OpenSSL

Google employs two members of the OpenSSL team, myself and Bodo Möller. As well as helping to address security issues as they arise (sometimes discovered by other Googlers) we’ve also been helping with work to improve OpenSSL’s capabilities, for example implementing application-defined TLS extensions, adding support for ALPN (http://tools.ietf.org/html/draft-ietf-tls-applayerprotoneg-01) and adding a framework for Authenticated Encryption with Additional Data (AEAD) (work in progress).

Capsicum

Over the last decade or so, it has become increasingly (and painfully) apparent that ACLs and MAC, which were originally designed to protect expensive mainframes from their users, and the users from each other, are failing to secure modern cheap machines with single users who need protecting from the software they run.

Instead, we need fine-grained access control and strong sandboxing.

My Cambridge colleague, Dr. Robert Watson says:

In August 2010, the University of Cambridge and Google published the Capsicum security model at the USENIX Security Symposium. Initially funded by a Google Faculty Research Award, Capsicum combines security ideas from historic research capability systems with contemporary operating-system design to allow robust, fine-grained, flexible, and application-centred sandboxing. Since 2012, Google Open Source Programs Office and the FreeBSD Foundation have been jointly funding continuing open source development to transition Capsicum from research to practice; Google also has summer students working on Capsicum through Google Summer of Code under the FreeBSD organization. 

The project has been led by FreeBSD developer Pawel Dawidek, who has refined the Capsicum model as our experience has grown, updating existing privilege-separated applications to use Capsicum (e.g., OpenSSH and dhclient). Pawel has also adapted new applications to use Capsicum compartmentalisation "out-of-the-box", including system tools services (e.g., tcpdump, kdump,the high-availability storage daemon [hastd], the security audit-trail distribution daemon [auditdistd]), and UNIX pipeline components (e.g., grep, uniq). The goal is to move gradually towards ubiquitous sandboxing: since it is cheap (and often easy), explore what happens when you try to do it everywhere, mitigating as-yet unknown future vulnerabilities. 

A key contribution of this work is the Casper application framework. Casper manages the creation of sandboxes, launching them on demand with only the rights they require to operate (Saltzer and Schroeder's 1975 Principle of Least Privilege). Existing applications can be linked against Casper to make use of a growing suite of Casper components, including sandboxed DNS resolution, delegation of file-system subsets, cryptographic random number generation, network services (such as sockets), system monitoring interfaces (via sysctl), and system databases such as the user and password files. 

Pawel's current work is due to wrap up later in 2013. FreeBSD 10.0 will ship with Capsicum enabled by default and a suite of sandboxed applications.

Certificate Transparency

A key concern has been the repeated failures of Certificate Authorities (CAs) to properly defend their operations and correctly issue certificates, most recently exemplified by the DigiNotar incident, which not only compromised the security of hundreds of thousands of internet users, but also led to the company’s bankruptcy.

We’ve been working hard on a new idea in that area, Certificate Transparency - essentially a verifiably correct log of all issued certificates that can be used to quickly detect any missteps on the part of CAs. And, of course, we are hard at work on an open source implementation, with clients in C++, Java and Python, and a C++ server.

Tor

Tor (originally short for The Onion Router), is a free software for enabling online anonymity. Widely recognized as a key component in online privacy, Foreign Policy Magazine named its primary authors among its 2012 Top 100 Global Thinkers “for making the web safe for whistleblowers.” Originally released in 2002 by the US Naval Research Laboratory, Tor development is now supported by a broad range of governments, organizations and individuals. Google is proud to have had the Tor Foundation as a Google Summer of Code mentoring organization since 2007, providing the project with both funding and potential new contributors.

Libpurple

If you’ve ever “chatted” online chances are very good that you’ve used libpurple. Libpurple is the core of various instant messaging applications, notably Pidgin and Adium. Through a donation to Instant Messaging Freedom, Inc. we have sponsored a developer working on improving its security.

By Ben Laurie, Security Engineering

We are joining the Open edX platform

A year ago, we released Course Builder, an experimental platform for online education at scale. Since then, individuals have created courses on everything from game theory to philanthropy, offered to curious people around the world. Universities and non-profit organizations have used the platform to experiment with MOOCs, while maintaining direct relationships with their participants. Google has published a number of courses including Introduction to Web Accessibility which opens for registration today.  This platform is helping to deliver on our goal of making education more accessible through technology, and enabling educators to easily teach at scale on top of cloud platform services.

Today, Google will begin working with edX as a contributor to the open source platform, Open edX. We are taking our learnings from Course Builder and applying them to Open edX to further innovate on an open source MOOC platform.  We look forward to contributing to edX’s new site, MOOC.org, a new service for online learning which will allow any academic institution, business and individual to create and host online courses.

Google and edX have a shared mission to broaden access to education, and by working together, we can advance towards our goals much faster. In addition, Google, with its breadth of applicable infrastructure and research capabilities, will continue to make contributions to the online education space, the findings of which will be shared directly to the online education community and the Open edX platform.

We support the development of a diverse education ecosystem, as learning expands in the online world. Part of that means that educational institutions should easily be able to bring their content online and manage their relationships with their students. Our industry is in the early stages of MOOCs, and lots of experimentation is still needed to find the best way to meet the educational needs of the world. An open ecosystem with multiple players encourages rapid experimentation and innovation, and we applaud the work going on in this space today.

We appreciate the community that has grown around the Course Builder open source project. We will continue to maintain Course Builder, but are focusing our development efforts on Open edX, and look forward to seeing edX’s MOOC.org platform develop.  In the future, we will provide an upgrade path to Open edX and MOOC.org from Course Builder. We hope that our continued contributions to open source education projects will enable anyone who builds online education products to benefit from our technology, services and scale. For learners, we believe that a more open online education ecosystem will make it easier for anyone to pick up new skills and concepts at any time, anywhere.

By Dan Clancy, Director of Research

Who is New in Google Summer of Code - Final post

For our final post in the Google Summer of Code blog series highlighting new open source organizations participating in this year’s program, we have organization administrators from TimVideos.us and lmonade discussing their student’s projects.

TimVideos.us is a group of projects which together create a system for doing both recording and live event streaming for conferences, meetings, user groups and other presentations. The project combines both software (gst-switch, streaming website and tools, speaker tracking, etc.) and hardware projects (HDMI2USB - A HDMI/DVI Capturing Solution). Parts of the system have been been used at large open source conferences such as Linux.conf.au, PyCon US and others.

Google Summer of Code 2013 has generously funded three students to work on the following TimVideos.us projects:

• EDID Database Website - Code  Extended display identification data (EDID) is a data structure provided by a digital display to describe its capabilities to a video source. Many devices ship with bad or misleading EDID information. The goal of this project is to develop a website to list EDID information, allowing users to browse, search and update them. 

• Developing Python API for gst-switch - Code  gst-switch aims to do interactive live mixing of incoming video streams, designed to meet the needs of conference recording. Designed as a flexible replacement for DVSwitch, and based around GStreamer, this project aims to develop a Python API for controlling and testing gst-switch. 

• Porting Flumotion to the Gstreamer-1.0 API - Code  Flumotion is a streaming media server which uses Gstreamer at its core. GStreamer recently released it's 1.0 API which is not compatible with the old 0.10 Flumotion. 

We hope that through our projects, the costs and expertise currently required to produce live streaming events will be reduced to near zero. We wish to develop a system where everyone has the ability to record presentations and host live remote participants across the globe.  

By Tim Ansell, TimVideos.us Google Summer of Code Organization Administrator and Mentor

-----

lmonade is a scientific software distribution that can be installed without administrative rights on Unix based systems. Building on the Gentoo packaging system, we hope to solve the dependency nightmare experienced by all sufficiently complex scientific software packages which have release schedules that do not fit the restrictions of packagers for large GNU/Linux distributions.

We are thrilled to be a part of Google Summer of Code as a new organization this year. As an umbrella organization, lmonade promotes ideas to improve various open source/free mathematical software, especially computational algebra projects. After the midterm evaluations, we are continuing the Google Summer of Code with 4 projects:

• Tom Bachmann - C++ wrapper for FLINT - FLINT is a highly optimized library for performing computations in number theory, written in C. This project aims to create C++ wrappers using expression templates, which compile down to code which achieves performance as close to native C as possible. 

• Remus Barbatei - Continuous integration platform for lmonade - This project is about improving the existing CI infrastructure used by lmonade to take advantage of the build instructions and dependency information stored in its package repository and easily set up nightly testing facilities for more scientific software packages. 

• Verónica Suaste - New decoding algorithms for error correcting codes in Sage - The main goal of this work is to implement decoding algorithms based on Gröbner bases methods in Sage. While improving the coding theory module of Sage, this will provide the opportunity to compare performance of this new algorithm with implementations in computer algebra systems. 

• Ioana Tamas - Binary decision diagrams for Boolean polynomial rings - Zero-suppressed binary decision diagrams are used by Polybori for efficiently representing Boolean polynomials. At the moment, they are manipulated via CUDD, which is not specialized on these types of diagrams and only uses C in the implementation. The goal of this project is to implement an independent library in C++ that is specialized on zero-suppressed binary decision diagrams. 

We wish all our students continuing success in their projects. 

By Burcin Erocal, lmonade Organization Administrator 

It’s hard to believe the student coding for Google Summer of Code 2013 will conclude in less than three weeks! To view a complete list of the projects students are working on this summer, visit the Google Summer of Code program site.

By Stephanie Taylor, Open Source Programs

Patchfield for Android

Some of the best apps are those that specialize in a single task. With Patchfield, a new open-source audio library for Android, you can create highly focused audio apps (such as synthesizers and effects) and then combine their capabilities by connecting them to one another in a virtual patch bay. Inspired by JACK, the JACK Audio Connection Kit, Patchfield provides a simple, callback-driven API for implementing audio modules, a flexible API for managing the signal processing graph, and support for inter-app audio routing. Patchfield is the 20% project of Peter Brinkmann, an engineer on the speech infrastructure team.

Running as a remote service, Patchfield allows audio developers to create interoperable apps that focus on a single task each, such as synthesis or effects or recording. The output of one app can be connected to the input of another, for a combinatorial explosion of possibilities. Running locally within a single app, Patchfield provides a powerful way of organizing the audio components of an app. The implementation resides entirely in user space and works on many stock consumer devices, such as Nexus 7 and 10.

Patchfield is available at GitHub, under the the Apache 2.0 License. The repository includes the core library as well as a number of sample projects that illustrate how to use Patchfield. Happy hacking!

DevBytes video: http://www.youtube.com/watch?v=LTisevoxm64

By Peter Brinkmann, Speech Infrastructure team