The Usability of GNOME

My Usability Test Results

The value of a usability test is finding areas where the program is difficult to use, so developers can improve the interface in the next version. To do that, you need to identify the tasks that were difficult for most users to perform.

You easily can see the usability test results by using a heat map. This kind of visualization neatly summarizes the results of the usability test. In the heat map, each scenario task is represented in a separate row, and each block in the row represents a tester's experience with that task (Figure 1). Green blocks indicate tasks that testers completed with little or no difficulty, and yellow blocks signify tasks that presented moderate difficulty. Red boxes denote tasks where testers experienced extreme difficulty or where testers completed tasks incorrectly. Black blocks indicate tasks the tester was unable to complete, while white boxes indicate tasks omitted from the test, usually for lack of time.

Figure 1. Usability Heat Map

The "hot spots" in the heat map show tasks that were difficult for testers.

1) Interestingly, all participants experienced significant issues with changing the default font in gedit (GNOME 3.12). A significant number of testers were unable to accomplish a similar task in Notes. In observing the tests, the testers typically looked for a "font" or "text" action under the gear menu. Many participants referred to the gear menu as the "options" or "settings" menu because of a previous affiliation with the gear icon and "settings" in other Mac OS X and Windows applications (Figure 2).

Figure 2. Header Bar for gedit, Showing the Gear Menu

These participants expected that changing the font was an option in the program, and therefore searched for a "font" action under the gear or "options" menu. Part of this confusion stemmed from thinking of the text editor as though it were a word processor, such as Microsoft Word, which uses items in menus or on a toolbar to set the document font. This behavior often was exhibited by first highlighting all the text in the document before searching for a "font" action.

2) In the Nautilus file manager, testers also experienced serious difficulty in creating a bookmark to a folder. In GNOME, this task is usually achieved by clicking into the target folder then selecting "Bookmark this Location" from the gear menu, or by clicking and dragging the intended folder onto "Bookmarks" in the left pane (Figure 3).

Figure 3. Dragging a Folder to Create a Bookmark in Nautilus

However, testers initially addressed this task by attempting to drag the folder onto the GNOME desktop. When interviewed about this response, almost all participants indicated that they prefer to keep frequently accessed folders on the desktop for easier access. Most testers eventually moved the target folder into the Desktop folder in their Home directory, and believed they successfully completed the task even though the target folder did not appear on the desktop.

3) Testers also experienced difficulty when attempting "find and replace text" in gedit. In this task, testers employed the "Find" feature in gedit to search for text in the document. Experimenting with "Find", testers said they expected to replace at the same time they searched for text. After several failed attempts, testers usually were able to invoke the "Find and Replace" action successfully under the gear menu.

Although the overall GNOME desktop was not part of the usability test, many testers experienced difficulty with the GNOME "Activities" hot corner. In the GNOME desktop environment, the Activities menu reveals a view of currently running programs and a selection of available programs. Users can trigger the Activities menu either by clicking the "Activities" word button in the upper-left corner of the screen or by moving the mouse into that corner (the "hot corner"). Although testers generally recovered quickly from the unexpected "hot corner" action, this feature caused significant issues during the usability test.

General Issues

The open-source software usability challenge is cultural. To date, usability has been antithetical to open-source software philosophy, where most projects start by solving a problem that is interesting to the developer, and new features are incorporated based on need. Crafting new functionality takes priority, and open-source developers rarely consider how end users will access those features.

However, a key strength of open-source software development is the developer-user relationship. In open-source software projects, the user community plays a strong role in testing new releases. Unfortunately, testers cannot rely on the typical user-testing cycle to provide sufficient user interface feedback. Developers can learn a great deal simply by observing a few users interacting with the program and making note where testers have problems. This is the essence of a usability test.

Usability tests need not be performed in a formal laboratory environment, and developers do not need to be experts in order to apply usability testing methodology to their projects. Developers need only observe users interacting with the program for usability issues to become clear. A handful of usability testers operating against a prototype provides sufficient feedback to make informed usability improvements. And with good usability, everyone wins.

______________________

Jim Hall is an advocate for free and open-source software, best known for his work on the FreeDOS Project. Jim earned his MS in Scientific and Technical Communication from the University of Minnesota, focusing on the usability of open-source software.