13 Reviewing, Evaluating, and Testing Documents and Websites

Conducting Usability Evaluations

Printed Page 345-348

Conducting Usability Evaluations

What is a usability evaluation? To evaluate the usability of a draft, you ask someone to study the draft, looking for ways to improve its usability. That person then communicates his or her impressions and suggestions, either in writing or in an interview.

You can perform usability evaluations of existing or prototype documents or sites. A prototype is a model that is built to simulate the look and feel of an item before it is produced commercially. In technical communication, a prototype is typically an early draft of a document, website, or software program. A prototype can range in sophistication from a simple drawing of a computer screen to a fully functioning system that looks exactly like a commercial product. Figure 13.2 shows an array of free blank templates that can be revised and used to create a home page of a website.

Figure 13.2 A Website Showing Sample Templates

Wix provides hundreds of different web templates, complete with images, geared to particular subjects, such as businesses, restaurants, and photography. This Figure shows a few of the blank templates offered by the company. The user would download a blank template and plug information into it, creating a working prototype. With this prototype, the user could then evaluate how well the design works.

Reprinted by permission of Wix.com.

Most types of formal usability evaluations involve three categories of people in addition to the writer:

Users. In technical communication, users are people who use a document, site, or program, usually as part of their jobs. They’re your primary audience, so they are an important source of feedback. They can be current or future users; they can be novice, experienced, or expert users. They are probably not people who work with you for the company that makes the product, because such people are likely to have specialized knowledge that would make them atypical.
Subject-matter experts (SMEs). An expert in the subject of the document, website, or software can be very useful in evaluating a draft. For instance, a database engineer is presumably an SME in database software programs. This person probably could see more—and different—potential problems in a new database program than a typical user could. He or she might also be the person in charge of carrying out the usability evaluation.
Usability experts. An expert in ergonomics, human-computer interaction, usability engineering, or cognitive psychology typically designs the usability evaluation. That is, he or she determines which questions to ask about the draft and how to most effectively and efficiently obtain answers. He or she might also carry out the evaluation. Or, a usability expert might evaluate a draft himself or herself.

Although there are many varieties, usability evaluations usually take one of five major forms:

Read more about interviewing and about writing questionnaires in Ch. 6..

Surveying or interviewing users. Evaluators survey or interview users to learn about the strengths and weaknesses of a document or site. These techniques sometimes reveal problems that can be fixed; for instance, you might learn that your users would really like to have a printed list of keyboard shortcuts to tape to the office wall. More often, however, these techniques provide attitudinal information; that is, they reveal users’ attitudes about aspects of using the draft.
Observing users. To understand how people use an existing document or site, evaluators go to their workplaces and observe them as they work. Observations can reveal, for example, that typical users are unaware of a feature that you assumed they used. This insight can help you see that you need to make that feature easier to find and use. Arrange the visit beforehand, and bring food to establish good will.
Interviewing SMEs and usability experts. An evaluator might ask an expert to study the draft for usability and then interview that person, asking general questions about the strengths and weaknesses of the draft or focused questions about particular aspects of the draft. One well-known version of an expert evaluation is called a cognitive walk-through, in which the evaluator asks an expert to carry out a set of tasks, such as signing up for RSS (rich site summary or really simple syndication) on a blog, a prototype, or an existing site. The evaluator watches and notes the expert’s actions and comments. Another version of an expert evaluation is called a heuristic evaluation. A heuristic is a guideline or desirable characteristic, such as that every page of a website should include an easy-to-find link to the home page. A heuristic evaluation, then, is an assessment of how well a draft adheres to a set of guidelines. After an expert conducts a cognitive walk-through or a heuristic evaluation, the evaluator interviews the expert.
Conducting focus groups. A focus group is a meeting at which a group of people discuss an idea or product. Typically, the people are current or prospective users. Let’s say your company sells a software program called FloorTraxx, which helps people design custom floors. A focus group might consist of FloorTraxx customers and perhaps other people who have indicated an interest in designing custom floors for houses. The moderator would lead a discussion that focused on what the customers liked and disliked about the product, whether they were satisfied with the results, and what changes they would recommend in an updated version. The moderator would also seek to learn what information the prospective customers would need before deciding to purchase the product.
Using a commercial usability service. Companies such as UserTesting.com offer usability testing of websites. You specify how many “users” you wish to have evaluate your site, their demographics (such as age, sex, web experience, and nationality), the context in which they are to use the site, a set of simple tasks they are to carry out, and a set of questions (such as “What do you like best about the site?”). You then receive a brief report from each person who evaluated your site and a video of the person thinking aloud while trying to carry out the tasks. Although such usability services claim that they are performing usability testing, in fact they are performing basic evaluations; real usability testing always involves real users. Real usability testing, as described in the next section, provides more detailed information because the testing team conducts the test in a controlled laboratory environment and can interact more extensively with test participants.

If your users include people from other cultures, be sure to include people from these cultures in your interviews and focus groups. If possible, use interviewers from the culture of the people you are interviewing. Vatrapu and Pérez-Quiñones (2006) have shown that people from other cultures are sometimes reluctant to criticize a draft for fear of embarrassing the interviewer. When the interviewer is from the same culture, however, people are more forthcoming.

After completing any usability evaluation, you need to gather the important information that you learned and share it with others in your company through a presentation, a website, or a collection of documents on the company intranet.