Drupal Usability Test Conclusions: A Missing Conceptual Foundation

Earlier this year we announced that we would be conducting a Drupal usability study that we would live stream so viewers could watch as participants worked with Drupal 7. Becky Gessler and I are excited to announce our analysis of the results that we will also present at DrupalCon Denver to the Drupal community in a “core conversation” session with Jen Lampton called “User eXperience for Open Source: How to Galvanize a Community."

After the study concluded we posted a YouTube playlist with each participant’s session. We went through each video and wrote down problems that each participant faced along with direct quotes, then consolidated and grouped these together by issue area. This document was posted on Drupal.org and members of the Drupal community have been filing issue reports to work on fixing specific problems.

While we learned about a lot of specific interface-level problems, we also saw broad trends that called for a higher-level analysis. We witnessed new users feeling confused, overwhelmed, uncertain and unaware of Drupal’s capabilities. Becky and I set to work preparing a summary of our findings. Out of the data, four layers of usability issues emerged:

• Conceptual: Most problems that new users encounter boil down to a missing understanding of how Drupal works, particularly the interaction between content, content types, and fields.
• Flow: While trying to complete tasks, new users frequently felt lost inside Drupal because they couldn’t tell what they were looking at, how they got there, or where they were supposed to go next.
• Terminology: Drupal is filled with terminology that just doesn’t make sense to new users.
• Interface: New users don’t trust Drupal because many interfaces are not intuitive.

We are excited for discussion in the Drupal community about how to best tackle the problems outlined in these layers. You can find our complete analysis in this report.

New users from our study did have some positive impressions of Drupal as well. Once they understood how it works, they were impressed by the power and extensibility that Drupal offers. This study also demonstrated that Drupal 7 brought significant usability improvements over Drupal 6, particularly with the ease of user interaction with administrative tools and the Views creation wizard.

We are excited by the success of this study. We look forward to engaging the Drupal community in discussion at DrupalCon Denver during our core conversation, and we are confident that by working together as a community, we can shape Drupal to be more supportive and helpful for new users.

By Garen Checkley, Google Search Quality Team

Drupal Usability Study at Google livestreamed to 100+

I am excited to announce what a huge success the Drupal usability studies were last week! We livestreamed 8 usability sessions using both Google+ Hangouts on Air and Livestream.com and had over 115 people watching live! The exciting conversations taking place in the #drupal-usability IRC channel proved how powerful live events are; the community truly came together through a process of collaborative brainstorming and experiencing the struggles of new users collectively as they happened.

Our usability lab setup was flawless thanks to the masterful help of my friend and coworker Garen Checkley. Jen Lampton from Chapter Three served as the help desk support for study participants during the session. I moderated the sessions and walked participants through different tasks that we created along with the help of Bojhan Somers and Angie Byron.

We divided our study participants into 2 groups to study; one using vanilla Drupal 7 to observe where new users had trouble understanding the the basic system, and one working with a custom Drupal 7 install that already had specific contrib-modules installed and enabled to gain insight into usability issues with specific interfaces, especially the hugely popular module Views."

Currently we are beginning the process of turning our findings into actionable items that we can put into issue queues. The Drupal Usability group will work hard to act on the results of this study to improve Drupal. Recorded videos of the sessions can now be viewed on our YouTube playlist.

Additionally, I have posted a video of my presentation “User Experience for You and Drupal too!” with Jen Lampton from Drupal Camp San Diego (SANDCamp) in an article on my blog. Be sure to check out the video for an overview of fundamental principles of user experience and how UX has evolved in the Drupal community.

When our findings and results from the study become more refined, Garen will write another blog post on this blog to share our findings and some more links. Until then, make sure to get involved with the Drupal Usability group if you are interested in helping to act on the findings of these studies and continue to make Drupal better!

By Becky Gessler, Google Search Quality team

Drupal usability study at Google

Drupal is an open source content management system with thousands of active community members behind it. A popular solution for both small and large scale websites, Drupal is extremely flexible and offers thousands of add-on modules.  Drupal’s user experience (UX) layer, however, can be daunting and frustrating for beginners to learn. I am working on an exciting project in conjunction with the Drupal User Experience team and the Google Open Source team to help determine some of the key UX issues new users of Drupal encounter. The usability study will have participants (all Googlers) building a website and will help to gain insight into the stumbling blocks users encounter along the way.

The usability study will be streamed live and available for everyone to watch. The usability study is planned to take place February 1-3. Details about the live stream will be posted in the comments section below in the coming days. You can follow the discussion about this study on the Drupal.org wiki page.

Saturday, January 28 at Drupal Camp San Diego (SANDCamp) I will be presenting a talk called “Usability Studies for you and Drupal too!” on the fundamental principles of user experience and an introduction to the usability study. Jen Lampton from Chapter Three is co-presenting with me to talk about why UX is so important to Drupal, what the Drupal UX team has discovered through past studies, and how to get involved with the project.

Stay tuned for another post on the results, and make sure to check back on the Drupal.org wiki for details on how to watch live!

By Becky Gessler, Google Search Quality team

Former student pays it forward as a mentor for Google Code-in

Charlie Gordon was a student participant in the Google Highly Open Participation Contest, an earlier version of Google Code-in that pairs high school students with mentors in open source projects. Here he discusses his recent experience as a Mentor for the 2010-2011 Google Code-in.

There were so many great stories from Google Code-in that it's hard to pick out just a few. I'm always so excited to get young people involved in open source - I remember how excited I was when I participated in the Google Highly Open Participation Contest myself back in 2007-2008. The program opened many exciting doors for me, including going to Drupal conferences, becoming involved in Drupal core development, and even eventually going to college at MIT, where I am now. It brings me great joy to be able to help open the same doors for the next generation of bright young open source enthusiasts.

Now for the stories! I'm always impressed when students, who are used to doing only what they're specifically told to do for school assignments, go above and beyond. Especially in the format of a contest where the incentives lead students to do the bare minimum, I find those students who put in a real effort to go above and beyond are those who really care about the open source project, and are those who are most likely to stick with it after the contest ends. Here are some of these students:

A student from Thailand, "chalet16", worked on five complex coding tasks for Drupal's style guide module. Over many iterations with the tasks' mentor, the quality of chalet16's code improved, and he also became more comfortable with using drupal.org's issue queue, a key part of our development process. Not only were the tasks well done, but when chalet16 encountered software bugs in parts of the code that he wasn't working directly on, chalet16 had no problem taking the initiative and opening new issues for the bugs, even submitting additional patches to those issues that fixed the problem. That's exactly the kind of participation that we want to attract to our project, and to open source software development in general.

Despite not having very much experience in PHP, “aantn,” a student from Israel, found ways to contribute to the Drupal project, such as by writing a Python script to survey test coverage statistics for Drupal core and the top 50 or so modules. We now have a much better idea of where and to what extent our automated testing system is being used, and making such statistics public will hopefully encourage module maintainers to write more test coverage for their code.

We specified a task that called for a Drupal core patch to allow a "full preview" button on the content submission form. Halfway through the task, Romanian student "bluman," encountered big roadblocks to that patch. After discussing the issue, we realized that the task would not work as a patch for Drupal core, but would have to be rewritten from the ground up as a contributed module. However, rather than give up, bluman worked hard to redo the task as a module, and came up with fantastic results.

Finally, we set a task to translate a small to medium sized module into another language. Another student from Romania, “ungureanuvladvictor," went way above and beyond the task description, translating one of the largest and most widely used contributed Drupal modules into Romanian. The translation file had more than 1,000 translations in it, a very impressive feat for such a short period of time. I had to scramble to find a Romanian speaker willing to review such a large translation!

Really, there are so many impressive students out there.

By Charlie Gordon, Drupal.org mentor

We always hope that the students who participate in programs like Google Code-in and Google Summer of Code develop a bond with the open source community they work with during the program and continue to contribute to the community long after the contest ends. Charlie Gordon is a prime example; he not only continues to be actively involved in the Drupal community but he is also sharing his experience and wisdom with younger developers and helping them become contributors to open source. Great job, Charlie!

By Stephanie Taylor, Open Source Programs Office

GNOME Usability Hackfest

Google recently sponsored the GNOME Usability Hackfest, which took place in London. With over 30 GNOME design and usability experts attending on some days, it was an unusually large, exciting and dynamic event. As GNOME 3.0 is just around the corner, people took advantage of the opportunity to build bridges within the GNOME usability community, and re-think the desktop paradigm.

Highlights for me included:

• GNOME 3.0 and Shell Design Planning by William Jon McCann and Seth Nickell
  
• Talking accessibility with Willie Walker
• Charline Poirier's Empathy Usability Report and Icon Usability Study
  
• HIG Planning
• Card sorting exercises to get a better understanding of how to organize applications and settings
  
• Usability improvements for GTK+.
  
• Discussions about how to improve Nautilus usability
  

In the last few days at the event, I spent a fair amount of time cleaning up the GNOME Usability Project Wiki so it is more clear and straightforward. I also did a lot of coordination with the Dev8D conference organizers to arrange for GNOME speakers at their event, and made arrangements for Antonio Roberts from the Dev8D community to attend the GNOME Usability Hackfest and participate on Thursday the 25th.

The GNOME Usability team has posted a great deal of blog posts, articles, and photos highlights about the work done at this hackfest and as more attendees post their notes, they will be updated onto the GNOME Planet blog aggregator. You can see my blog for more photos and a full report on the event. Many thanks to Google for sponsoring this event.

By Brian Cameron, GNOME Foundation Board Secretary

Spreading the Summer Love

By Leslie Hawthorn, Open Source Team

Google Summer of Code™ is in full swing, and the outpouring of community support has been phenomenal. Less than two weeks after announcing the program for 2008, our program information flyer was translated into over 35 languages, from Croatian to Kinyarwanda. We've also heard from several students and mentors that they're working on spreading the summer love to their local communities. Here are a few highlights:

• Adriano Monteiro Marques and some of his co-conspirators from the Umit project toured several universities in Brasil to let students know about the program. They also created a program video in both English and Brasilian Portuguese.


• Alex Pantaleev added a call for students to check out the program to his last class at Ohio State University.


• The Beijing LUG kindly localized our program presentation and are busy spreading the word about Summer of Code in Chinese universities; the Free Software Initiative of Japan was also kind enough to localize the presentation and spread the word.


• The folks at DevSeed are hosting an information session in Washington, D.C., USA to help students learn more about contributing to Drupal through Summer of Code.


• Louis Landry from the Joomla! project writes that today they'll be sending information to various Computer Science Departments throughout the United States about participating in Summer of Code.


• Dr. Titus Brown was kind enough to organize an impromptu meet up session at Michigan State University for all takers, with a hopeful eye towards finding a few more Python contributors.

Numerous other students have written to let us know that they've blogged about the program, encouraged their friends to apply and given talks at their universities and local user groups.

Many thanks to all of you for your efforts to help us grow the Summer of Code community. If you're out and about evangelizing the program, we'd love to hear from you; post a comment and let us know what you've been doing.

Drupal's Runner-Ups

By Leslie Hawthorn, Open Source Team

As we mentioned last week, many of the mentoring organizations participating in the Google Highly Open Participation Contest had a difficult time choosing their Grand Prize Winners. All of our projects had stellar participants, and we're pleased to join them in sharing kudos with their unofficial "runner-ups."

Angela Byron and Adam Light from the Drupal project sent us these accolades:

Jimmy Berry

Jimmy Berry was a pleasure to work with over the course of GHOP. He has a real knack of coming in, tackling a difficult task that's been long-neglected (such as SimpleTest coverage for most of Drupal core, or the new Git back-end for our Version Control API module), and knocking it completely out of the park. Jimmy rocketed his way up the Drupal learning curve and is now one of our best new contributors.

Edward Yang

Edward Z. Yang already has open source development experience, as the developer for the HTML Purifier project. During his time with the Drupal project over the course of GHOP, he has shown a penchant for impeccable code quality, and takes incredible pride in his work, always going above and beyond (often way above and beyond) what has been asked for by the task description, as particularly evidenced by his work on the Coder Format module.

Charlie Gordon

If GHOP were based solely on community participation, Charlie Gordon would far and away take the prize. In addition to suggesting tasks and helping to answer other students' questions, he also spear-headed efforts to setup the Drupal community's successor to the GHOP contest, DROP. Oh yeah, and he also kicked butt on several GHOP coding tasks, and made very short work of everything from writing SimpleTests to porting Revision Moderation module to 6, and even creating and maintaining the new Citation Filter module.

Wilson Lee

Wilson Lee took on a wide variety of GHOP tasks ranging from documentation to code to marketing. Kourge did a particularly awesome job on a marketing
presentation for Drupal 6 (.zip) and a Core Hooks Cheatsheet (.zip) which will be of great use to Drupal module developers everywhere. Kourge was also frequently found in our IRC channels and helped other students find answers to their questions.

Dmitri G.

Dmitri G. was actually too *young* to participate in GHOP, being only 12 years old, so wasn't eligible to claim any tasks or prizes. However, he more than made up for that by helping to mentor and guide the GHOP students throughout the duration of the contest. So we'd like to name him as honorary GHOP runner-up.