Observing the User Experience

Elizabeth Goodman and Mike Kuniavsky

Buy on Amazon

A research plan consists of three major parts: why you’re doing the research (the goals), when you’re going to be doing it (the schedule), and

how much it’s going to cost (the budget). These are in turn broken up into practical chunks such as report formats and timetables.

the first step is to make a list of issues of how the product’s user experience affects the goals of the company. Each issue represents a goal for the research program; it focuses the research plan and helps uncover how the product can be improved for the greatest benefit to the company.

  1. Collecting issues and presenting them as goals 2. Prioritizing the goals 3. Rewriting the goals as questions to be answered

The key questions each person (or department) should answer are as follows: 1. In terms of what you do on a day-to-day basis, what are the goals of the product? 2. Are there ways that it’s not meeting those goals? If so, what are they? 3. Are there questions you want to have answered about it? If so, what are they?

There shouldn’t be more than half a dozen or so “big” questions and a dozen or so smaller, more specific ones.

Based on what you know about the company priorities, group the questions into clusters by technique, and make a rough schedule.

In my experience, useful times for calculating the duration of a qualitative user research project such as a usability test (including project management time and typical inefficiencies) are roughly as follows:

(as of spring 2003 in San Francisco) they tend to fall around $100 per person per 90-minute session for most research.

a demographic profile is one that describes a person’s physical and employment characteristics; a Web use profile describes someone’s Web experience; and a technological profile describes their experience with computer technology in general.

Screeners vary from project to project and from recruiter to recruiter, but there are some general rules that apply to most. 1. Stick to 20 questions. There’s a reason that game exists. It’s possible to find out almost anything about someone in 20 questions. Most target audiences can be defined in 10 to 15 questions, and if the people are prescreened through your database, you can get away with fewer than 5. 2. Make it short. It should be possible to get through a whole screener in five to ten minutes. 3. Be clear and specific. The person responding to the question should know exactly what kinds of answers are expected. 4. Never use jargon. Use simple, straightforward, unambiguous language. 5. Ask for exact dates, quantities, and times. This eliminates the problem of one person’s “occasionally” being another’s “all the time.” 6. Every question should have a purpose. Each question should help determine whether this person is in the audience or not. Don’t ask questions that are incidental or “nice to know” since answers to them will not be useful in recruiting and take everyone’s time. Nice-to-know questions can be asked at the beginning of the test. 7. Order questions from general to specific,

Open-ended questions like this serve two purposes. They give the recruiter an idea of how articulate a potential participant is, and they can collect information that’s not easily formatted as a multiple-choice question. Save them for the end, and don’t put more than one in any given screener

Questions should be nonjudgmental. The person answering the question should not think that you’re expecting a specific answer or that any answer is wrong.You

Questions should be focused on a single topic. A question that has an “and” or an “or” linking two ideas leads to ambiguity since it’s often unclear which part of the question is being answered.

Keep questions open-ended.