Google Summer of Code Veteran Orgs: Twitter

For our fifth guest post from veteran Google Summer of Code organizations the mentors and organization administrators from Twitter discuss their student’s projects below. 

For the second time, @TwitterOSS participated in the Google Summer of Code. Unlike many Google Summer of Code participating organizations that focus on a single ecosystem, we work on a variety of projects that span multiple programming languages and communities.

We worked on three projects with three amazing students over the summer.

Matrix optimizations for Scalding
Tomas Tauber worked closely with his mentor, Oscar Boykin, to improve the performance of Scalding by adding matrix optimizations (see the commits). For example, how should we multiply A*B*C? Perhaps (A*B)*C takes a lot longer than A*(B*C) due to the sizes of the matrices. What about matrices with huge skew, such as Twitter’s follower graph where some users have millions of followers, but most have only a handful? By optimizing at the Matrix API layer, we can easily reap the benefits at higher layers. This project added a scheduler to the formulas users write with Matrices, and performs the computation in the optimal order, where optimal is in terms of intermediate data size and formula tree-depth. See the performance results for more information.

Asynchronous DNS support for the Netty Project

Mohamed Bakkar worked with the lead of the Netty project Trustin Lee to add a built-in asynchronous DNS resolver. Instead of using the blocking DNS resolver provided by the JDK, the new resolver will prevent applications built on top of Netty from their performance being impacted by slow or overloaded DNS servers. As a result, Netty applications that rely on DNS should have a positive performance impact.

Authentication support for Apache Mesos
Ilim Ugur worked with Mesos committer Vinod Kone to add an authentication stage in Mesos before letting frameworks and slaves talk to the master(s) thereby making the communication between the modules forming Mesos (masters, slaves and frameworks) more secure.

As part of Google Summer of Code, students and mentoring organizations receive a stipend. We are donating our portion of the stipend to the Software Freedom Conservancy which is a 501(c)(3) organization that helps provide a non-profit home and infrastructure for open source projects like Git and Selenium.

We really enjoyed the opportunity to take part in Google Summer of Code. Thanks again to our three students, mentors and to Google for the amazing program.

By Chris Aniszczyk, Head of Open Source at Twitter