ns-3 networks with students over the summer

ns-3 is a discrete-event network simulator, with a particular emphasis on network research and education. ns-3 aims to allow researchers to move effortlessly between simulations, testbeds, and live experiments. It is designed to offer a high degree of realism in its models, through frameworks for running real application and kernel code, and also for integrating with virtual machine environments and testbeds.

At the end of the summer, we asked mentors of each of our Google Summer of Code projects to provide summaries of their respective students’ contributions over the summer, each of which is listed below.

Dizhi Zhou developed a set of MAC schedulers to be used with the ns-3 LTE module. The ns-3 LTE module already supported a real-world API (the FemtoForum LTE MAC Scheduler Interface Specification) for the implementation of LTE MAC schedulers. When the summer started only a couple of simple scheduler implementations were available (Round Robin and Proportional Fair). To address this problem, Dizhi implemented eight schedulers (FDMT, TDMT, TTA, FDBET, TDBET, FDTBFQ, TDTBFQ, PSS). In addition to the implementations, Dizhi put a significant effort into writing documentation and validating the performance of the new schedulers using some known reference scenarios. The result is a set of well-documented and well-tested scheduler models that are expected to be merged with the main ns-3 code soon, and will be very useful for many ns-3 users.
By Nicola Baldo, Mentor

Sindhuja Venkatesh developed IPv4 Netfilter and NAT code for ns-3 during her 2012 Google Summer of Code project. This project started by picking up netfilter and connection tracking code that had been partially completed during a previous Google Summer of Code project by Qasim Javed. The first portion of the program worked on updating and refining the Netfilter portion of the code, which provides an extensible mechanism for inserting packet filtering and mangling operations into well-defined locations in the IPv4 stack.  As proof-of-concept, a basic NAT was written on top of this framework.  After the midterm exam, Sindhu continued to work on more complete NAT models by writing dedicated IPv4 NAT code hooked into the netfilter framework.  Two primary modes of NAT were implemented, a static one-to-one NAT and a dynamic NAT with port translation.  A NAT helper object was also written, along with a few examples and documentation for the models.  While there remain a few loose ends to complete before merging to ns-3-dev, we are hopeful to be able to complete and merge these models in the next ns-3 release cycle.
By Tom Henderson, Mentor

Mudit Gupta spent his summer designing and developing interfaces aimed at integrating ns-3 with an HLA (High Level Architecture) Federation. The first issue Mudit was faced with was how to communicate between the various federates. Although ns-3 is compliant with the latest LLVM compilers, the HLA components are not. Hence it was impossible to simply integrate the C++ HLA interfaces into ns-3. The chosen workaround was to build a Java ns-3 Federate object acting as a stub and communicating with the main ns-3 code through “normal” sockets. This solution is sub-optimal and the plan is to move to proper C++ interfeces as soon as the HLA implementations (i.e., Portico or CERTI) will be updated to support LLVM. The second problem was to synchronize the ns-3 time reference with the Federation one. The solution relies on a modification of the real-time ns-3 scheduler, with a virtual clock advanced by the RTI Federation messages rather than the hardware PC clock. The last issue Mudit worked on was the interaction between the Federates. The ns-3 module is able to receive Objects from the Federates along with their Attributes but such Objects and Attributes are currently unspecified, as there is no actual ns-3 module using them. A proof-of-concept module has been developed, showing how to receive the Object and its Attributes.
By Tommaso Pecorella, Mentor

We hope Dizhi, Sindhujha, and Mudit will continue their involvement with the ns-3 project, and community outside of Google Summer of Code as well. On a concluding note, we’d like to thank Google once again for accepting us into the program this year. We look forward to applying for Google Summer of Code 2013!

By Lalith Suresh, ns-3 Organization Administrator

Get Ready to Vote with Google

(Cross-posted from the Official Google Blog and the Politics and Elections blog)

Every four years in the United States, people prepare to head to the polls and increasingly search for information about how to register to vote, where to vote and who is on their ballot. Even though it is 2012, important voting information is disorganized and hard to find on the Internet. To help voters research candidates and successfully cast their ballot on Election Day, we’ve launched our new Voter Information Tool. 

You can enter your address to find information on your polling place, early vote locations, ballot information with links to candidates’ social media sites and voting rules and requirements. The tool is easy to embed on any website and is open source so developers can modify it to create custom versions. We're working with a number of media partners to ensure the tool is accessible across the web, and partners like Foursquare and AT&T are doing great work building apps on our Civic Information API. 

We hope this tool will help make getting to the polls and casting your ballot as simple as possible.

Posted by Jesse Mwaura, Google Politics & Elections Team

Lenses, Folds, and Traversals: Haskell Meet-up

The Google San Francisco office hosted a talk by Edward Kmett called "Lenses, Folds, and Traversals" which was well attended by the local Bay Area Haskell Users group as well as Google employees on Thursday, October 18,  2012. Almost 100 people turned up to hear why the power is in the dot (as explained by Gavin Bierman, Erik Meijer and Wolfram Schulte in their 2002 paper "The essence of data access in Cω: The power is in the dot"). This paper pushed the boundaries of our understanding of how to build data-access features for programming languages that go beyond naıve access via simple APIs) by greatly generalizing member access (e.g. over choice types). Although this paper resorted to the creation of a new language to support innovations in data-access (and concurrency) this talk by Edward showed how generalized member-access could be achieved using just a regular library in Haskell as exemplified by his lens library.

Edward presented a beautifully crafted set of APL-like combinators that allow programmers to navigate through structured data and perform generalized operations for access and modification. The core concept in the library is of a lens which groups together a 'setter' and a 'getter':

data Lens a b = Lens { set :: a -> b -> a
                    , view :: a -> b
                    }

where informally the idea is that a lens give you a way to extract from an object (of type a) some information (of type b) by using the view function and also to let you perform the reverse i.e. using the set function, take an object of type a and a value of type b and return a new object of type a with the sub-information about type b suitably incorporated. The actual definition of lens in Edward's library is different (it is cast in terms of a functor). Lenses are essentially composable functional references. The talk showed how lenses can be composed using the ordinary function composition operator (.) in Haskell. The talk then illustrated a scheme for performing lens-based folds and traversals in a highly composable manner and climaxed with the impossible: the overloading of function application. For the grand finale, Edward explained how lenses are just the functors, foldables, traversals and functions that (hard-core Haskell) people already know how to use.

Here is a specific example which shows the result of computing an expression using Haskell's command-line interpreter:

ghci> over (mapped._2) succ [(1,2),(3,4)]
[(1,3),(3,5)]

Informally, here we see the successor function succ applied to the second element of every structure contained by the top-level structure i.e. 2 is mapped to 3 and 4 is mapped to 5. The over function is used to modify the target of a lens (or all the targets of setters or traversals) with a function. The mapped function is a setter that can be used to map over all the values in a functor. The _2 operation accesses the second field of a tuple.

You can view Edward's slides and an earlier talk on the subject. Alternatively, @PLT_Borat could have just told you that "Costate Comonad Coalgebra is the equivalent of Java's member variable update technology for Haskell".

By Satnam Singh, Google

Ashier: Automating Terminal Interactions with Templates

We are pleased to announce the open source release of Ashier, a utility that allows users to automate terminal interactions with templates.

Have you ever returned from lunch, only to find that the program you started an hour ago had made no progress because it was waiting for you to answer a trivial question?  Terminal interaction automation is a technique that automatically enters commands and answers program prompts in a terminal even when you are away from the keyboard.

Ashier offers several advantages over traditional tools (such as expect): a template language that replaces regular expressions in many situations, support for nested multiline textual data parsing, and the ability to integrate with response logic implemented in any language.

Ashier aims to make terminal interaction automation simpler and more accessible to users — so you can spend less time at your computer and still get more work done.

By Chuan-kai Lin, Site Reliability Engineering Team

Celebrating Dart’s birthday with the first release of the Dart SDK

Working with a combination of small scripts to create large web apps is no easy task.  This is why a year ago we released a technology preview of Dart, a project that includes a modern language, libraries and tools for building complex web applications. Today, after plowing through thousands of bug reports and feature requests from the web community, a new, more stable and comprehensive version of Dart is now available and ready to use.


Dart is carefully designed so that you can build your web apps without having to choose between static and dynamic languages. With Dart, you can start coding simple scripts on your own and scale seamlessly to building large apps with hundreds of thousands of lines of code.  These complex apps can also be supported by a big team with a significantly reduced debugging and maintenance overhead.

With this first developer-oriented version of the Dart SDK, we’ve made several improvements and added many features:

• A faster Dart Virtual Machine that outperforms even V8 on some Octane tests.
• A new Dart to JavaScript translator that generates fast and compact output.
• An HTML library that works transparently on modern browsers.
• A library to interoperate with JavaScript code.
• An easy to use editor.
• Pub, a new package manager
• Dartium, a Chromium build with native Dart support.
• A server-side I/O library.
• A language specification describing the Dart semantics, including new features.

Over the following months, we will continue to work hard to evolve the SDK, improve Dart’s robustness and performance, and fine-tune the language while maintaining backwards compatibility.

You can download the Dart Editor from dartlang.org. It comes with a copy of the open-source SDK and Dartium. Thanks again for all your feedback - keep it coming.

Posted by Lars Bak, Software Engineer

Introducing Supersonic Query Engine

We are happy to announce the open source release of Supersonic, a query engine library, that is extremely useful for creating a column oriented database back-end.

Supersonic’s main strength lies in its speed. It is a cache-aware engine which exploits several low-level optimization techniques to ensure second-to-none execution times and high throughput. By making use of SIMD instructions and efficient pipelining we make columnar data processing very fast.

The engine supports a wide variety of typical columnar operations as well as many other specialized functionalities. We believe it to be a useful tool for those of you working on new DBMS solutions to help handle your data more effectively.

For some pointers on how to get started with the library visit the project page on Google Code. We also host a discussion group where you can get up to speed on the latest development news.

By Supersonic Team

Opening doors for students this summer

This summer, openSUSE had another great experience participating in the Google Summer of Code. While working on the list of ideas for our fifth year of Google Summer of Code projects, we decided to encourage students to apply not only for openSUSE-specific projects, but also for projects that would be useful to our upstreams and to other distributions. We love working with other organizations, which is why we always try to push for more collaboration.

Nine of our students successfully completed their projects, and we’d like to share their accomplishments with you below.

Beautiful 1-Click Install, by Saurabh Sood
One of openSUSE’s features is the “one click installer” which needed a visual refresh.  Saurabh made a great deal of progress on the installer, as you can see in some of the screenshots on his weblog, and we’re all looking forward to getting it into the main distribution.

Complete AppStream/Software-Center, by Matthias Klumpp
Matthias tackled the challenge of making the PackageKit-based software center work nicely on several distributions by improving the performance of PackageKit, adding new support for parallel transactions and making backend API changes. He also made improvements to the software center itself making it super fast and able to work better on non-Ubuntu distributions. On top of that, he created a new library to handle the appstream metadata that will be used by several other projects.

openSUSE Karma plugin for openSUSE Connect, by Priyanka M
openSUSE has a social network called “Connect”.  This summer, Priyanka wrote a karma plugin that functions similarly to StackOverflow’s reputation system.  openSUSE users get karma points for things like Bugzilla activity, or promoting openSUSE in social media.

osc2 client, by Marcus Hüwe
The Open Build Service is a service openSUSE provides to allow people to easily build packages for a variety of distributions like openSUSE, SLES, Fedora, and Ubuntu.  The command line client for interacting with this service is osc.  This summer, Marcus continued his previous Google Summer of Code work on the second version of the client.

Popularity Contest for RPM (popcorn), by Akshit Khurana
Akshit spent his summer improving Popcorn, which is inspired by Debian’s “popcon”, a system of (voluntarily) tracking how popular packages are in the install base, which is very useful information for a distro.

Redesign fdisk to be more extensible and implement GPT support, by Davidlohr Bueso
Davidlohr worked upstream on refactoring and modernising fdisk. He managed to clean up and modularize a lot of complicated fdisk code, which will make it easier in the future to support more disklabels and add more functionality, as well as creating a planned libfdisk library.

Scanny, by Piotr Niełacny
Scanny is a Ruby on Rails security scanner, which parses Ruby files, looks for suspicious patterns, and produces a report.  Piotr added new security checks, designed and implemented a proper command-line interface, and tested the project on several real-world applications which led to various performance gains and other improvements.

Upstream/Downstream Tracker, by N.B. Prashanth
Distributions usually have tools for tracking new versions upstream, but the existing tools are all pretty much limited to one distribution, and have some limitations.  The goal for this project was to take some existing work done for openSUSE, and turn it into the backend of a more complete system. A rails web application was also created to be able to see the upstream versions as well as administer the packages to be tracked.

Writable snapshot support for ext4, by Yongqiang Yang
Yongqiang worked on adding writable snapshot support to ext4. The code written during the project is available here, and people can test it with packages from the openSUSE Build Service repository.

It has been an exciting summer for all of us - students, mentors and administrators, and the openSUSE community. openSUSE will benefit greatly from the results of these projects, and we do hope the larger free software community will also enjoy what our students produced. Many thanks to Google for organizing Google Summer of Code, and we look forward to another wonderful summer next year!

By Matt Barringer and Vincent Untz, openSUSE Organization Administrators

ReFr: A New Open-Source Framework for Building Reranking Models

We are pleased to announce the release of an open source, general-purpose framework designed for reranking problems, ReFr (Reranker Framework), now available at: http://code.google.com/p/refr/.

Many types of systems capable of processing speech and human language text produce multiple hypothesized outputs for a given input, each with a score. In the case of machine translation systems, these hypotheses correspond to possible translations from some sentence in a source language to a target language. In the case of speech recognition, the hypotheses are possible word sequences of what was said derived from the input audio. The goal of such systems is usually to produce a single output for a given input, and so they almost always just pick the highest-scoring hypothesis.

A reranker is a system that uses a trained model to rerank these scored hypotheses, possibly inducing a different ranked order. The goal is that by employing a second model after the fact, one can make use of additional information not available to the original model, and produce better overall results. This approach has been shown to be useful for a wide variety of speech and natural language processing problems, and was the subject of one of the groups at the 2011 summer workshop at Johns Hopkins’ Center for Language and Speech Processing. At that workshop, led by Professor Brian Roark of Oregon Health & Science University, we began building a general-purpose framework for training and using reranking models. The result of all this work is ReFr.

From the outset, we designed ReFr with both speed and flexibility in mind. The core implementation is entirely in C++, with a flexible architecture allowing rich experimentation with both features and learning methods. The framework also employs a powerful runtime configuration mechanism to make experimentation even easier. Finally, ReFr leverages the parallel processing power of Hadoop to train and use large-scale reranking models in a distributed computing environment.

By Dan Bikel and Keith Hall, Research Scientists at Google

Who’s New in Google Summer of Code 2012: Part 6

For our final installment of our summer series featuring new organizations that participated in Google Summer of Code 2012 we have wrap up posts from OWASP, Outercurve and 52°North.

Being part of Google Summer of Code this year was truly an awesome experience both for students and OWASP mentors alike. OWASP is an open source community dedicated to enabling organizations to conceive, develop, acquire, operate, and maintain applications that can be trusted. However, it is usually challenging getting students to develop code for OWASP projects because application security is considered a difficult, and sometimes complex subject. 

Google Summer of Code acted as a window between OWASP and the academic world, enabling students to work first-hand with highly skilled application security professionals. 

The following OWASP projects took part in Google Summer of Code 2012: 

OWASP Zed Attack Proxy

• Cosmin Stefan rewrote the existing ZAP spider, making it it much faster and more accurate. He also enhanced ZAP to manage HTTP sessions, which will be a key feature that we can build on for future releases.
• Guifre Ruiz added a new Ajax spider to ZAP by integrating with crawljax project, which will allow ZAP to spider AJAX applications much more effectively. He not only had to understand the ZAP code but also delved into the crawljax code and has submitted a series of enhancements back to the crawljax project as well.
• Robert Koch (c/o Mozilla) added Web Sockets support to ZAP, exceeding the capabilities of any other  security tool in this aspect, free or commercial.

"This is the first year I've been involved in Google Summer of Code, and I'm delighted to say it has been an overwhelming success. We had 3 students working on the Zed Attack Proxy project, and I have been very impressed with the quality of their work. They all required a lot less hand-holding than I expected - they all just got stuck in, while still keeping their mentors up to date and asking questions when necessary. They were all very dedicated and produced code which has significantly enhanced the project. While I'm obviously hoping that they will carry on contributing, even if they don't I'll still consider this to have been very worthwhile and will be very keen to participate next year." - Simon Bennetts, mentor 

OWASP AppSensorRauf Butt extended the AppSensor project by building a SOAP-based web service in Java to allow multiple Java applications to access the same AppSensor detection engine. 

"The Google Summer of Code was a good experience for the AppSensor project. We had lots of needs to cover, so our scope was initially too large. Through the course of the first half of the summer, it became evident that this was the case. Through helpful discussions with our student as well as experiences related from those who had been through the process before, we were able to narrow our focus and produce a more defined project that will serve as the basis for the next version of AppSensor." - John Melton, mentor 

OWASP Hackademics “Pragya Gupta developed a new frontend for the OWASP Hackademic Challenges which implements CMS-like functionality, simplifies installation and facilitates the automated addition of new challenges. The additional functionality will significantly help professors and tutors in real class environments.” - Konstantinos Papapanagiotou, mentor 

Kudos to the Open Source Team at Google and special thanks to Carol Smith who guided us through this initiative! 

By Fabio Cerullo, OWASP Google Summer of Code 2012 Org Administrator

----------

The Outercurve Foundation provides software IP management and project development governance to enable and encourage organizations to develop software collaboratively in open source communities.  Outercurve was accepted as a mentoring organization for our first Google Summer of Code in 2012, being awarded two slots for student projects.  

Of the project proposals we received, a number of students stood out in discussing and evolving their proposals with Outercurve mentors during the submissions period.  The two projects we chose were: 

• Irina Grosu from Romania with her project to add modules to Outercurve Mayhem to integrate Microsoft Office, the Google API, Bluetooth connectivity, and Skype.   
• Cristina Turken, also from Romania, worked on creating a GUI-based desktop application that allows software publishers to create an Outercurve CoApp package, (instead of using the command line tools).  

While we were very excited to participate in the Google Summer of Code, we had a problem: these were not the only two excellent proposals we received from students. The Outercurve Mayhem and Outercurve CoApp projects each wanted to support at least one more student proposal. The project leaders (finding budget at Microsoft) and Outercurve jointly funded two more student proposals. 

The two "Outercurve" Summer of Code projects accepted were:  

• Henrik Juvonen in Finland wanted to build CoApp for Visual Studio so that CoApp library packages (native C/C++ and .NET) can be discovered, installed, and updated without ever having to leave the development environment.
• Anirudh Ranganath India wanted to add a number of events and responses to Mayhem for monitoring charging states, noting USB device connect/disconnect, and an Alarm event, as well as putting the computer in different power modes (sleep, hibernation, shut down) and setting the volume level.

Over the next few weeks we will showcase all of our students’ work. I’d like to thank Google for the opportunity to participate, the project leaders for finding the funds to co-sponsor additional student projects, and our mentors.  I would especially like to thank our students for all their hard work. I hope they all found the summer worthwhile and will continue to participate with their Outercurve Foundation projects of choice. 

By Stephen R. Walli, Outercurve Foundation Technical Director 

---------- 

The open source software initiative 52°North is an open international network of partners from research, industry and public administration. Its main purpose is to foster innovation in the field of Geoinformatics through a collaborative R&D process. The R&D process takes place within 52°North R&D communities, which develop new concepts and technologies such as for managing near real-time sensor data, integrating geoprocessing technologies into SDIs, making use of GRID and Cloud technologies. The developers evaluate new macro trends, such as the Internet of Things, the Semantic Web or Linked Open Data, and find ways to unfold their use in practice. 

As a Google Summer of Code first-timer, 52°North was thrilled to be chosen as a mentoring organization and particularly proud to welcome four students to work on different projects in the fields of Sensor Web, Web of Things and Geoprocessing this summer. 52°North’s overall goals with the student projects this summer was to improve the usability of the products and extend the user base to new domains. 

• Sarah Harvey’s project, On-demand transformation of Open Street Map Data into common GIS format, brings the power of the 52°North WPS to the popular Open Source Mapping project. It deals primarily with providing bindings for OpenStreetMap data so that they may be used within the Web Processing Service (WPS).
• Alexander Kmoch enabled easy integration of user implemented encodings for observations to the 52°North Sensor Observation Service (SOS). In his “Exchangeable Encodings for SOS” project, he developed and integrated a plugin mechanism into an SOS server for plain CSV (comma separated values), as well as WaterML 2.0 response formats.
• In his project, GIS link to the Web of Things, Sidhant Hasija created a link between the rising Web of Things with standard GIS by implementing the GeoServices REST Specification for the Arduino-powered SenseBox devices.
• Shubham Sachdeva focused on 52°North Sensor Observation Services installation and subsequent maintenance issues. In his project, "SOS Administrator", he developed an initial configuration and installation wizard, as well as a powerful administrative back-end, both browser-based and platform independent.

By Daniel Nüst, 52°North Google Summer of Code Organization Administrator

We will start posting more Google Summer of Code wrap-ups from returning organizations later this week.

By Stephanie Taylor, Open Source Programs