Wicked Good XPath: a faster JavaScript XPath library

We are proud to release Wicked Good XPath, a Google-authored pure JavaScript implementation of the DOM Level 3 XPath specification. We believe it to be the fastest XPath implementation available in JavaScript.

To use Wicked Good XPath, simply download the wgxpath.install.js file and include it on your webpage with a script tag. For example:
<script src="wgxpath.install.js"></script>

Then call wgxpath.install() from your JavaScript code, which will ensure document.evaluate, the XPath evaluation function, is defined on the window object. To install the library on a different window, pass that window as an argument to the install function.

Despite the growing popularity of CSS selectors, XPath remains a useful mechanism to locate elements in an HTML document. It has particularly heavy usage in the context of frontend web testing tools like Selenium and Web Puppeteer. Sometimes there is simply no way to reference an element on the page other than with an XPath expression.

For those who have never used XPath, here is a taste:

On a Google search results page, the XPath expression //li[@class=”g”][3] identifies the third search result. Here is a snapshot from the Chrome extension XPath Viewer showing the third result selected when that expression is used.

A key challenge when using XPath, however, is that it is not natively supported on every browser. Most notably, Internet Explorer does not provide native XPath support for HTML documents. As a result, users turned towards pure JavaScript implementations of XPath. In 2005, Google engineers released AJAXSLT, which included a correct, but slow, XPath evaluator. Running web tests on IE using AJAXSLT was time-consuming.

Fast-forward to 2007, Cybozu Labs released JavaScript-XPath, a new JavaScript XPath implementation that was 10 times faster than AJAXSLT. The web testing tools began using it instead, and life was good... for a while. The JavaScript-XPath quickly fell out of maintenance, and bugs became tough to get fixed. Also, since it wasn't written in Google Closure, it was tricky for us Googlers to integrate into our JavaScript applications. A rewrite was necessary.

However, we went beyond merely porting the library to Google Closure and fixing a couple bugs. We identified some significant additional performance improvements, such that our version runs about 30% faster than the original. On top of that, the Closure compiler was able to minify our code down to a mere 25K, 40% smaller than JavaScript-XPath's 42K. Finally, the code is structured and documented in a way that we believe will make future maintenance quicker and easier.

We would like to thank our two Google interns, Michael Zhou and Evan Thomas, for doing the bulk of the development on this project.

By Greg Dennis and Joon Lee, Google Engineers

Test your Web Apps on Chrome with ChromeDriver

ChromeDriver is a tool for testing websites with Google Chrome that implements the open source WebDriver wire protocol so it can easily be integrated with an existing WebDriver test suite. For those who aren't familiar with WebDriver, you may want to refer to our initial post about the project. Simply put, WebDriver presents an object-based API for automating the web from a real users perspective, such as clicking elements on a page and typing into text fields.

The WebDriver API is available for many popular browsers.  Each browser has its own driver, with ChromeDriver, of course, supporting the WebDriver API for Google Chrome. Unlike other drivers which are maintained by the open source Selenium/WebDriver team, ChromeDriver is developed by Chromium, the open source project that Google Chrome is based on.

Besides a new ChromeDriver release this past week, we wanted to share info about a new website dedicated to the project: http://code.google.com/p/chromedriver. This site will serve as the central location for all things relating to ChromeDriver. You can use the new site to:

• Download ChromeDriver
• Find information on getting started with and using ChromeDriver
• Submit ChromeDriver-specific bugs and feature requests

ChromeDriver works with the current stable, beta, and dev versions of Google Chrome. Older versions of Google Chrome are not supported and are not guaranteed to be compatible with ChromeDriver.  Consult the release wiki for more information on our release and support policy.

Thanks for testing with Google Chrome!

By Ken Kania, Chromium Developer

Test Your Mobile Web Apps with WebDriver - A Tutorial

Mobile testing has come a long way since the days when testing mobile web applications was mostly manual and took days to complete. Selenium WebDriver is a browser automation tool that provides an elegant way of testing web applications. WebDriver makes it easy to write automated tests that ensure your site works correctly when viewed from an Android or iOS browser.

For those of you new to WebDriver, here are a few basics about how it helps you test your web application. WebDriver tests are end-to-end tests that exercise a web application just like a real user would. There is a comprehensive user guide on the Selenium site that covers the core APIs.

Now let’s talk about mobile! WebDriver provides a touch API that allows the test to interact with the web page through finger taps, flicks, finger scrolls, and long presses. It can rotate the display and provides a friendly API to interact with HTML5 features such as local storage, session storage and application cache. Mobile WebDrivers use the remote WebDriver server, following a client/server architecture. The client piece consists of the test code, while the server piece is the application that is installed on the device.

Get Started

WebDriver for Android and iPhone can be installed following these instructions. Once you’ve done that, you will be ready to write tests. Let’s start with a basic example using www.google.com to give you a taste of what’s possible.

The test below opens www.google.com on Android and issues a query for “weather in san francisco”. The test will verify that Google returns search results and that the first result returned is giving the weather in San Francisco.

 public void testGoogleCanGiveWeatherResults() { 
 // Create a WebDriver instance with the activity in which we want the test to run. 
 WebDriver driver = new AndroidDriver(getActivity()); 
 // Let’s open a web page 
 driver.get("http://www.google.com"); 

 // Lookup for the search box by its name 
 WebElement searchBox = driver.findElement(By.name("q")); 
 // Enter a search query and submit 
 searchBox.sendKeys("weather in san francisco"); 
 searchBox.submit(); 

 // Making sure that Google shows 11 results 
 WebElement resultSection = driver.findElement(By.id("ires")); 
 List<WebElement> searchResults = resultSection.findElements(By.tagName("li")); 
 assertEquals(11, searchResults.size()); 
 // Let’s ensure that the first result shown is the weather widget 
 WebElement weatherWidget = searchResults.get(0); 
 assertTrue(weatherWidget.getText().contains("Weather for San Francisco, CA")); 
} 

Now let's see our test in action! When you launch your test through your favorite IDE or using the command line, WebDriver will bring up a WebView in the foreground allowing you to see your web application as the test code is executing. You will see www.google.com loading, and the search query being typed in the search box.

We mentioned above that the WebDriver supports creating advanced gestures to interact with the device. Let's use WebDriver to throw an image across the screen by flicking horizontally, and ensure that the next image in the gallery is displayed.

 WebElement toFlick = driver.findElement(By.id("image")); 
// 400 pixels left at normal speed 
Action flick = getBuilder(driver).flick(toFlick, 0, -400, FlickAction.SPEED_NORMAL) 
   .build(); 
flick.perform(); 
WebElement secondImage = driver.findElement(“secondImage”); 
assertTrue(secondImage.isDisplayed()); 

Next, let's rotate the screen and ensure that the image displayed on screen is resized.

 assertEquals(landscapeSize, secondImage.getSize()) 
((Rotatable) driver).rotate(ScreenOrientation.PORTRAIT); 
assertEquals(portraitSize, secondImage.getSize()); 

Let's take a look at the local storage on the device, and ensure that the web application has set some key/value pairs.

 // Get a handle on the local storage object 
LocalStorage local = ((WebStorage) driver).getLocalStorage(); 
// Ensure that the key “name” is mapped 
assertEquals(“testUser”, local.getItem(“name”)); 

What if your test reveals a bug? You can easily take a screenshot for help in future debugging:

 File tempFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); 

High Level Architecture

WebDriver has two main components: the server and the tests themselves. The server is an application that runs on the phone, tablet, emulator, or simulator and listens for incoming requests. It runs the tests against a WebView (the rendering component of mobile Android and iOS) configured like the browsers. Your tests run on the client side, and can be written in any languages supported by WebDriver, including Java and Python. The WebDriver tests communicate with the server by sending RESTful JSON requests over HTTP. The tests and server pieces don't have to be on the same physical machine, although they can be. For Android you can also run the tests using the Android test framework instead of the remote WebDriver server.

Infrastructure Setup

At Google, we have wired WebDriver tests to our cloud infrastructure allowing those tests to run at scale and making it possible to have them run in our continuous integration system. External developers can run their mobile tests either on emulators/simulators or real devices for Android and iOS phones and tablets.

Android emulators can run on most OSes because they are virtualized, so we run them on our generic cloud setup. Though there are many advantages to using Android emulators because they emulate a complete virtual device (including the virtual CPU, MMU, and hardware devices), it makes the test environment slower. You can speed up the tests by disabling animations, audio, skins, or even by running in the emulator headless mode. To do so, start the emulator with the options --no-boot-anim, --no-audio, --noskin, and --no-window. If you would like your tests to run even faster, start the emulator from a previously created snapshot image. That reduces the emulator startup time from 2 minutes to less than 2 seconds!

iOS simulators can't be virtualized and hence need to run on Mac machines. However, because iOS simulators don't try to emulate the virtual device or CPU at all, they can run as applications at "full speed," this allows the test to run much faster.

Stay tuned for more mobile feature in Selenium WebDriver, and updates on the Selenium blog. For questions and feedback not only of the Android WebDriver but also its desktop brethren, please join the community.

By Dounia Berrada, Engineer on the EngTools team

Introducing WebDriver

WebDriver is a clean, fast framework for automated testing of webapps. Why is it needed? And what problems does it solve that existing frameworks don't address?

For example, Selenium, a popular and well established testing framework is a wonderful tool that provides a handy unified interface that works with a large number of browsers, and allows you to write your tests in almost every language you can imagine (from Java or C# through PHP to Erlang!). It was one of the first Open Source projects to bring browser-based testing to the masses, and because it's written in JavaScript it's possible to quickly add support for new browsers that might be released

Like every large project, it's not perfect. Selenium is written in JavaScript which causes a significant weakness: browsers impose a pretty strict security model on any JavaScript that they execute in order to protect a user from malicious scripts. Examples of where this security model makes testing harder are when trying to upload a file (IE prevents JavaScript from changing the value of an INPUT file element) and when trying to navigate between domains (because of the single host origin policy problem).

Additionally, being a mature product, the API for Selenium RC has grown over time, and as it has done so it has become harder to understand how best to use it. For example, it's not immediately obvious whether you should be using "type" instead of "typeKeys" to enter text into a form control. Although it's a question of aesthetics, some find the large API intimidating and difficult to navigate.

WebDriver takes a different approach to solve the same problem as Selenium. Rather than being a JavaScript application running within the browser, it uses whichever mechanism is most appropriate to control the browser. For Firefox, this means that WebDriver is implemented as an extension. For IE, WebDriver makes use of IE's Automation controls. By changing the mechanism used to control the browser, we can circumvent the restrictions placed on the browser by the JavaScript security model. In those cases where automation through the browser isn't enough, WebDriver can make use of facilities offered by the Operating System. For example, on Windows we simulate typing at the OS level, which means we are more closely modeling how the user interacts with the browser, and that we can type into "file" input elements.

With the benefit of hindsight, we have developed a cleaner, Object-based API for WebDriver, rather than follow Selenium's dictionary-based approach. A typical example using WebDriver in Java looks like this:

// Create an instance of WebDriver backed by Firefox
WebDriver driver = new FirefoxDriver();

// Now go to the Google home page
driver.get("http://www.google.com");

// Find the search box, and (ummm...) search for something
WebElement searchBox = driver.findElement(By.name("q"));
searchBox.sendKeys("selenium");
searchBox.submit();

// And now display the title of the page
System.out.println("Title: " + driver.getTitle());

Looking at the two frameworks side-by-side, we found that the weaknesses of one are addressed by the strengths of the other. For example, whilst WebDriver's approach to supporting browsers requires a lot of work from the framework developers, Selenium can easily be extended. Conversely, Selenium always requires a real browser, yet WebDriver can make use of an implementation based on HtmlUnit which provides lightweight, super-fast browser emulation. Selenium has good support for many of the common situations you might want to test, but WebDriver's ability to step outside the JavaScript sandbox opens up some interesting possibilities.

These complementary capabilities explain why the two projects are merging: Selenium 2.0 will offer WebDriver's API alongside the traditional Selenium API, and we shall be merging the two implementations to offer a capable, flexible testing framework. One of the benefits of this approach is that there will be an implementation of WebDriver's cleaner APIs backed by the existing Selenium implementation. Although this won't solve the underlying limitations of Selenium's current JavaScript-based approach, it does mean that it becomes easier to test against a broader range of browsers. And the reverse is true; we'll also be emulating the existing Selenium APIs with WebDriver too. This means that teams can make the move to WebDriver's API (and Selenium 2) in a managed and considered way.

If you'd like to give WebDriver a try, it's as easy as downloading the zip files, unpacking them and putting the JARs on your CLASSPATH. For the Pythonistas out there, there's also a version of WebDriver for you, and a C# version is waiting in the wings. The project is hosted at http://webdriver.googlecode.com, and, like any project on Google Code, is Open Source (we're using the Apache 2 license) If you need help getting started, the project's wiki contains useful guides, and the WebDriver group is friendly and helpful (something which makes me feel very happy).

So that's WebDriver: a clean, fast framework for automated testing of webapps. We hope you like it as much as we do!

by Simon Stewart, Engineering Productivity Team

London OS Jam 10: Off-Topic Is the New On-Topic

By Zak Cohen and Neil Dunn, Software Engineering Team

Another Thursday. Another London Google Open Source Jam. This time the format couldn't have been simpler: Come to Google. Have some beer, have some pizza. Talk about whatever you like.

Here's the lightning talk roundup. 5 minutes apiece:

• Simon Stewart - Stinky Code Project - How to ruin an open source project.


• Sam Mbale - Open Source Social Networking.


• John Ripley - Writing Vorbis from scratch. John's rewriting Vorbis. From scratch. It sounds great.


• Douglas Squirrel - A day in the life of a check-in. What should happen when someone checks in code. Another energetic superlist from Squirrel (with pictures!).


• Chris Read - Build tools - Good vs Bad. Chris wants to build a new build tool. We helped him, sorta.


• DJ Walker-Morgan - Talking to the press about your open source project. Some eye-opening advice from DJ about publicising your project.


• Nicolas Roard - Gears - Extending the browser!


• Ade Oshineye - NBL - Ade thinks it's server-side Javascript.


• William Fulton - Using OpenOffice Spreadsheets for C-style macro processing - Hacking for code generation.


• Joe Walnes - Lightspeed talk - AJAX web apps. Do it properly: Bookmarkable URLs and MVC design.


• Simon Stewart. Again - Super lightspeed talk - The roadmap for integrating Selenium and Webdriver.


• Steev - Minerva home automation.


• Neil Dunn - Lightspeed talk - The framework chase. Is that new framework really going to simplify your project?


• Rob Tweed - Mumps. The Swiss Army Knife of databases.

John Ripley. Rewriting Vorbis. From Scratch.

Since 5 minutes can seem like a really long time we also introduced something new: Lightspeed talks! Pow! Don't have enough to fill 5 minutes? Don't have slides? Don't worry! Stand up and say your thing. 30 seconds. 5 seconds. A single word perhaps!

Talks start conversations, conversations start projects, projects are good!

Joe really wanted to spread the message about MVC for AJAX web apps.

Don't know about the London Open Source Jam yet? Where have you been? OS Jam is a free for all, open to anyone, geek night where open source hackers get together to discuss a topic close to their hearts.

Rock on OS Jam! See you next time...

(Update, Nov 7th: Fix spelling of William's last name. Sorry!)

Selenium Ice: A "Cool" New Browser Extension for Internet Explorer

By Jason Huggins, Software Engineering Team and Selenium Co-Creator

I'm happy to announce "Selenium Ice", a new tool for developers who test web applications in Internet Explorer with Selenium. This first version of Ice is limited in scope — it simply adds a new namespace ("window.ice") to every web page, for which developers can write code in C# that is callable from JavaScript. In the future, Ice will be used by the Selenium project team as a building block for providing more native control over the IE browser process. With this native access, Selenium will be able to more reliably catch and close unexpected dialog windows, provide better information when JavaScript errors occur, or do things that require interaction with the operating system, like handling file uploads and downloads. Ice can also serve as a base for linking in faster XPath support in IE — an oft-requested feature among Selenium users.

The Ice project was developed using Visual C# 2008 Express Edition, so no development licensing fees are required beyond access to a machine running Windows. There's an NAnt script for continuous integration, a WiX script for automated creation of a Microsoft Installer (.msi), and a system test script written in Python that can launch IE and ensure that the extension works correctly.

The code is all Open Source and available under the Apache 2.0 license. You can check out our project site on Google Code and may want to join our Selenium Ice discussion group. We always love to hear from you, so let us know what you think of our cool new code!

Selenium Users Event Coming Up

By Jason Huggins, Jennifer Bevan and Simon Stewart, Software Engineering Team

Since the Google Test Automation Conference last year, it has been no secret that Google uses Selenium for functional testing of web applications. We've also constructed a Selenium farm for the distributed and speedy testing of select applications on a range of operating system and browser combinations. We all work at Google and contribute to the project in our 20% time, so we are proud to announce that we are hosting an open evening for users of Selenium at the Googleplex in Mountain View, CA, USA on Monday, February 25th at 6:30 PM. If you are interested in using Selenium for web app testing and sharing your thoughts with other Agile developers, please join us for an evening of discussion, presentations and lightning talks. You'll hear from representatives of several Selenium projects, including Selenium core, Selenium Remote Control, Selenium IDE and Selenium Grid. Some of these folks will have come from as far afield as London and Tokyo, and we hope you'll take this opportunity to explore Selenium's future with them. Google's Open Source Team will be providing hors d'oeuvres and a delightful selection of soft drinks.

We hope to see you there. If you're planning to join us, please make sure to pre-register.