Updates to Our Project Hosting Service

By Leslie Hawthorn, Open Source Team

Users of Google Code Project Hosting rejoice! The Subversion component of this service has now been upgraded to version 1.5, so you'll likely find merging your branches much easier. For full details, check out Ben Collins-Sussman's write up on the Google Code Blog.

Updates from Googlers Contributing to Open Source Projects

By Leslie Hawthorn, Open Source Team

You may recall some of our previous posts about Google employee contributions to Open Source during their 20% time. While many engineers spend their 20% time on releasing code created internally at Google, many more spend their time contributing to external projects just to scratch their own itch. We're pleased to bring you some updates about what our engineers have been doing over the past few months:

For all you version control geeks out there, you'll be interested to know that Ben Collins-Sussman has been working on rewriting Subversion's HTTP protocol. While the rewrite will still be WebDAV compatible, he's busy removing all of the DeltaV formalities that cause numerous extra requests. Once complete, users should see much faster network traffic when speaking to an Apache server. For more details, check out the write up on Ben's blog.

Continuing his work on WHOPR, a scalable whole program optimizer for GCC, Diego Novillo reports that the complier can now build several Google applications with link-time optimizations enabled. Diego and the rest of the WHOPR team are handling code generation bugs and performance problems with generated code. They expect to be showing steady performance improvements over the next couple of months.

Frank Mayhar recently submitted a patch to remove the journaling dependency from ext4. The patch has been accepted and should be merged very soon. You can find more details and the actual patch submitted in the kernaltrap.org list archives.

Shiki Okasaka continued development on the ES Operating System, releasing some new source tarballs. Included in the latest release were a number of contributions made by Google Summer of Code students. The switchover to the new Web IDL standard from OMG IDL for the system interface definitions is in preliminary stages and some progress has also been made on the implementation of the project's TCP/IP stack. The latest release also contains the Web IDL based preliminary RPC runtime for x86 linux, allowing for testing and combination of multiple ES software components running in separate processes on Linux, a substitute for the ES kernel natively running on a PC.

Tim Hockin contributed to a number of different systems software projects. Work continued on the recently released prettyprint project. Tim has been exploring languages to use for prettyprint's runtime, Javascript being the top contender for now. He has also worked on improvements to ACPID, including a lot of code clean up and a minor release. Tim also has a patch in his review queue to add netlink support to this completely flexible, totally extensible daemon for delivering ACPI events. He's also done some more work to whip the MCE Daemon into shape and is looking at promoting it for various distros.

These are just a few of the many contributions that have taken place over the past few months, and we'll be bringing you regular news about what Googlers are up to in the wide world of Open Source. Happy Hacking!

A Gaggle of Googlers Are Going to OSCON Next Week

By Cat Allman, Open Source Programs

The Annual O'Reilly Open Source Convention (OSCON ) is returning to Portland, Oregon, USA next week from July 21-25, and like swallows to Capistrano, Googlers will be there in force.

Between speakers, tutorial instructors, demo-ers of cool stuff in our booth, #116, and attendees, we expect to have upwards of 25 Googlers in attendance. Sessions with our speakers and instructors include:

On MONDAY, July 21st
PHP Extension Writing, Marcus Boerger, and Practical Test-driven Development, Josh McAdams.

On TUESDAY, July 22nd
An Open Source Startup in Three Hours, Gavin Doughtie, Porting to Python 3.0, Anthony Baxter, and People for Geeks, a panel including Brian Fitzpatrick, Ben Collins-Sussman, plus Tuesday evening is the annual Google O’Reilly Open Source Awards.

On WEDNESDAY, July 23rd
Subversion Worst Practices, Ben Collins-Sussman and Brian Fitzpatrick, Code Reviews for Fun and Profit, Alex Martelli, An Open Source Project Called 'Failure', a panel with Ben Collins-Sussman and Brian Fitzpatrick, Google XML Pages (GXP), Laurence Gonsalves and Harry Heymann, and The Google Open Source Update, Chris DiBona and Leslie Hawthorn.

On THURSDAY, July 24th
PLUTO: PL/SQL Unit Testing for Oracle, Josh McAdams, General Lightning Talks, lead by Anthony Baxter, Do You Believe in the Users?, Ben Collins-Sussman and Brian Fitzpatrick, CSS for High Performance JavaScript UI, Gavin Doughtie, Even Faster Web Sites, Steve Souders, and (The Lack of) Design Patterns in Python, Joe Gregorio.

On FRIDAY, July 25th
Open Source and Standards, Joe Gregorio

If you have questions about Google and Open Source, come on down. Hope to see you there.

Export a Git Project to Google Code

By Benjamin Lynn, Google Developer Programs

Git and Subversion can work together in many ways. Previously, we showed how Git users can work with a project primarily managed by Subversion. In this post, we describe one way to handle the converse: Google Code acts as a stable read-only Subversion mirror of a Git project. In this model, patches are first applied to the central Git repository and exported to Google Code later.

Instead of merely providing a link to your repository, why not widen your audience with just a handful of commands? Open up your Git-hosted project to all Subversion users, whose patches can be integrated via Git.

We presume some familiarity with Git, though blindly typing the commands below should produce acceptable results.

1. Create Subversion-aware Git clone

Naturally, your official source tree lives on some Git-capable server, which we denote by $GIT_REPO. After creating a new Google Code project, initialize an intermediary repository and fetch the Git tree:

$ git svn clone --username you https://your-project.googlecode.com/svn/trunk
$ cd trunk
$ git fetch $GIT_REPO

The Subversion repository must be nonempty. A new Google Code project contains one revision by default, but if you reset it, you should also create a first revision.

Create a temporary branch for the fetched repository, and tag its head:

$ git branch tmp $(cut -b-40 .git/FETCH_HEAD)
$ git tag -a -m "Last fetch" last tmp

2. Apply initial commit

Unfortunately, Git treats the initial commit specially, and in particular, cannot rebase it. Work around this as follows:

$ INIT_COMMIT=$(git log tmp --pretty=format:%H | tail -1)
$ git checkout $INIT_COMMIT .
$ git commit -C $INIT_COMMIT

3. Rebase and submit

Apply all the other commits to the temporary branch, and make it the new master branch:

$ git rebase master tmp
$ git branch -M tmp master

Lastly, commit the changes to Google Code:

$ git svn dcommit

To more faithfully represent deleted subdirectories and copies of unmodified files, run dcommit with the options --rmdir and --find-copies-harder. Be aware the latter option can be expensive.

4. Update Google Code

Later, export Git repository updates to Google Code as follows:

$ git fetch $GIT_REPO
$ git branch tmp $(cut -b-40 .git/FETCH_HEAD)
$ git tag -a -m "Last fetch" newlast tmp
$ git rebase --onto master last tmp
$ git branch -M tmp master
$ git svn dcommit
$ mv .git/refs/tags/newlast .git/refs/tags/last

For simplicity, we've exported directly to Google Code. It may be faster to first export to a local Subversion repository, and then mirror it to Google Code via svnsync.

Develop with Git on a Google Code Project

By Benjamin Lynn, Google Developer Programs

Do you often work offline? Wish you could make local commits that you can reorganize and upload later? Would you like to have your own copy of the entire history of the project which you can peruse at your leisure?

Do you want to serve code and its history to others? Have changes you want to share but it's too early for the public?

Working on several issues in parallel? Need to save some of those experimental changes after all? Need to create, merge, clone and otherwise manipulate branches cheaply and offline?

You can do all this, and more, with Git, a version control system rapidly growing in popularity. You can readily find Git tutorials, but since I'm writing this post, I'll shamelessly plug my own guide to Git!

Although Google Code natively speaks Subversion, you can easily use Git during development. Searching for "git svn" suggests this practice is widespread, and we too encourage you to experiment with it.

We focus on importing a complete Git repository from a Google Code project organized in the recommended fashion. The git-svn manpage thoroughly describes how to handle other cases such as nonstandard layouts, importing only a few revisions, sharing exported repositories, and so on.

1. Import

First we perform the equivalent of a svn checkout. In an empty subdirectory, run:

$ git svn clone --username your-name -s https://your-project.googlecode.com/svn
 # older versions of git: replace "-s" with "-Ttrunk -bbranches -ttags"

Like a Subversion checkout, you now have a local copy of your project. Unlike a Subversion checkout, you also have a local copy of the entire history of the project. Try:

$ git log          # print summary of history
$ git diff HEAD^^  # diff against two revisions ago
$ gitk             # graphical history browser
$ qgit             # Qt-based history browser

These read from local disk, and work even when you're offline.

2. Develop

You now have a fully fledged version control system at your fingertips. You can checkpoint locally. You can create, merge, and destroy branches cheaply. You can checkout long-lost ancient code. You can stockpile and reorganize your commits.

Any Git tutorial teaches these abilities. We'll content ourselves with simple examples.

First, the basics. Edit code as usual, but if you add or remove files, type:

$ git add FILENAME...

or

$ git rm FILENAME...

There's no need to inform Subversion as git-svn will do so later. In fact, we only talk to Subversion via git-svn, and never run pure svn commands.

The only other git command you must know is:

$ git commit -a

which saves the current state of your project locally. You can see them with git log. Commit early and commit often!

Now for a couple of tricks. Let's say you've made several commits and suppose you want to undo the last one:

$ git reset --hard HEAD^

Or suppose you want to get a certain file from five commits ago:

$ git checkout HEAD~5 foo.c

We've barely scratched the surface. There are countless other features worth learning, particularly Git's extraordinary lightweight branches.

3. Update

Periodically, you should get online and fetch the latest changes from Google Code with:

$ git svn rebase   # think "svn update"

4. Export

Submit your commits to Google Code with:

$ git svn dcommit  # think "svn commit"

This converts each of your Git commits into Subversion commits and uploads them, provided your repository is up-to-date. To keep working, go back to step 2.

Stay Tuned!

We've seen it's easy to setup Git with a Google Code project. If distributed version control systems sound intimidating, the above is a great way to get your feet wet. You can always delete the directory and go back to Subversion.

But what if you keep your project in a Git repository, and you want to export it to Google Code? So you'd have a canonical read-only Subversion mirror of your project?

Exporting a Git project to Google Code requires only a handful of commands. We'll show you how in a future post.

This Week's Top 10's: Universities for Google Summer of Code 2008

By Leslie Hawthorn, Open Source Team

Last Monday, we brought you news on the Top 10 countries from which we're welcoming this year's Google Summer of Code™ students. Many thanks to everyone from the community for their suggestions about which program statistics they'd like to see next. By popular demand, this week we're showcasing data about which schools our Summer of Coders are attending.








Many thanks to David Anderson, currently on his third internship with Google and organization administrator for the Subversion project, for his help in getting these statistics together. As always, we welcome your comments; let us know what information from Google Summer of Code you'd like to see featured.

"Look! Actual Code!"

By Jason Robbins, Open Source Team

In one Dilbert cartoon, Dogbert confronts a long-winded technology "guru" by showing him some actual code, which blows him away. It's funny because it's true: some long, abstract discussions can turn into short, concrete ones when you can point to code.

We recently launched a new source code browsing tool as part of Google Code's project hosting feature. This new tool makes it easy to navigate through a project's Subversion repository. Key features include: fast directory browsing tree, syntax highlighting, history of changes, and easy-to-read diffs. See it yourself under the "Source" tab of any project that we host. For example: Google Gears source code.

We hope you find that the details make this tool a joy to use. For example, it's easy to get straight to the source code of a file, yet still see some of its recent history right on the same page. And, you can flip through revisions of a file with just a click. Stay tuned as the tool matures and evolves into something even more exciting in the months ahead.

Special thanks go to Google intern Jenan Wise for his passion for great software and attention to detail on this tool.

Best Practices: Repository Resets

By Ben Collins-Sussman, Open Source Team

Over in the google-code-hosting discussion group, we see a lot of users asking us to 'reset' their Subversion repositories back to an empty revision 0, tossing away all data and history. Sometimes there are good reasons for doing this; however, much of the time we're getting these requests for the wrong reasons.

Users who are new to Subversion often get the idea that the global revision number has some sort of special meaning — that it somehow measures the "maturity" of the software. In fact, the revision number is just a dumb counter that measures moments in time; it actually has no special meaning. If you accidentally mess up a code import or don't like the way things are arranged then just rearrange things with 'svn mv' or delete everything with 'svn rm' and start over! There's no need for the repository's history to be "pure." The repository is simply tracking changes over time (even restarts and mistakes), not describing the evolution of flawless software.

There are typically only three legitimate reasons for resetting a repository:

1. Something really sensitive, confidential, or illegal was accidentally committed, and now exists in the history forever.


2. Some HUGE garbage file was accidentally committed, and now a ridiculous amount of disk quota is wasted.


3. A project owner wants to replicate a prior repository into the repository, using the 'svnsync' tool (which requires an empty repository at revision 0).

If your request fits one of these categories, then please drop the Project Hosting team a note on the public googlegroup, or, if it's a sensitive matter, mail us privately. We'll ask you to remove everything in the repository, thus proving you're a legitimate project owner and not someone forging emails, and then we'll reset your repository.

Another tip: when a project is first created, it begins life at revision 1, with empty /trunk, /branches, and /tags directories. A project at revision 1 can be reset to revision 0 by project owners themselves; you'll see a bubble on the right side of the 'Source' tab advertising this fact. (This feature makes it easy for newly-created projects to prep the repository for receiving data from 'svnsync'.) Once a repository is changed and goes beyond revision 1, however, resetting a repository is no longer self-serve; you'll have to ask the administrative team for a reset.

One final tip: the wiki feature saves wiki pages into the /wiki directory of the Subversion repository. This means that if you make a wiki change while svnsync is populating your repository, you'll break the sync halfway through. You'll have to request a repository reset and start the sync all over. Be careful!