π Lecture 39 β Behavior & Form β Part VII
π Overview: This lecture focuses on the principles of visual interface and information design, providing a comprehensive guide on how to create clear, functional, and aesthetically pleasing user interfaces. It covers topics from visual structure and alignment to the use of imagery, color, text, and the critical role of consistency and standards, aiming to equip designers with the knowledge to create effective and user-friendly digital products.
ποΈ Topics Covered
This lecture explores the principles of visual interface design, including how to provide visual structure and flow through alignment, grids, symmetry, and spatial harmony; how to use cohesive and contextually appropriate imagery, such as function-oriented icons and visualizations of behavior; the importance of integrating style and function for branding; the seven grand principles of visual information design according to Edward Tufte; best practices for the use of text and color; and finally, the benefits, risks, and proper application of interface consistency and standards.
π Lecture Summary
39.1 Provide visual structure and flow at each level of organization
Interfaces are composed of visual and behavioral elements grouped into panes and screens. It is critical to maintain a clear visual structure to help users navigate. Grouping can be achieved by position, alignment, color, texture, size, or shape. Several important attributes help define a crisp visual structure.
π Definition β Alignment: The arrangement of visual elements so they line up with each other, creating a sense of order and organization for the user.
π Formula: Grouped elements should be aligned both horizontally and vertically, following a regular grid structure to combat ambiguity.
π Example: For controls stacked vertically, labels should be left-justified for easier scanning. A related group of checkboxes, radio buttons, or text fields should all be aligned according to the same regular grid to maintain a clean layout.
Symmetry and Balance: Symmetry is a useful tool for providing visual balance, while asymmetrical balance is harder to achieve in user interfaces due to screen real-estate constraints. Two common types are vertical axial symmetry and diagonal axial symmetry, the latter being the most frequent in dialog boxes. Sovereign applications typically achieve balance through a grid at the top level but use symmetry for internal elements.
Spatial Harmony and White Space: This considers the interface as a whole. Certain proportions, like the Golden Section ratio, are more pleasing to the eye. Proper dimensioning of functional regions and a proper amount of white space between elements is critical. Avoid indecisive proportions (e.g., "almost a square"); make them bold, crisp, and exact.
π‘ Why this matters: A well-structured interface reduces user cognitive load, making the application feel organized and easy to navigate.
Use cohesive, consistent, and contextually appropriate imagery: Icons and other illustrative elements must be carefully designed, with a good understanding of personas, their mental models, and cultural issues (e.g., red is not a warning color in China). Visual elements should be part of a cohesive visual language, where similar elements share attributes like position, size, and style to form a cohesive whole.
Function-oriented Icon: Designing icons for abstract functions is challenging. Guidelines include representing both the action and an object acted upon (e.g., a document with an X for Cut), keeping icons simple, reusing elements, and using ToolTips or text labels for clarity.
Associating Visual Symbols to Objects: Creating unique symbols for object types aids user recognition, and these symbols should be used wherever the object is represented.
Rendering Icons and Visual Symbols: Icons should be simple and schematic, not photo-realistic. Photo-realistic icons draw undue attention, render poorly at small sizes, and lack visual cohesion. They are like all-capitalized text; the differences are not sharp.
Visualizing Behaviors: Use visual elements (e.g., diagrams, thumbnails) to show users the results of interface functions, rather than just describing them with words. For example, the Word Page Setup dialog shows a Preview of the page layout. Text is still important for precise values.
Integrate style and function comprehensively and purposefully: Stylistic elements must be applied from a global perspective, ensuring functional aspects are in harmony with the visual brand.
Form versus Function: Stylized visual elements should not affect the basic shape, visual behavior, or visual affordance of controls. Educational and entertainment applications allow for more stylistic experimentation, but basic affordances must be preserved.
Branding and the user interface: Brand value is the sum of all user interactions with a company. A positive first impression in the first five minutes is critical, built on visual design. Long-term relationships are built by delivering on that promise through consistent behavior (interaction design).
39.2 Principles of Visual Information Design
Information design guru Edward Tufte asserts that good visual design is "clear thinking made visible." He identifies two key problems: displaying multidimensional information on a 2D surface and the low resolution of computer displays. He proposes seven "Grand Principles" to maximize the effectiveness of information display.
- Enforce visual comparisons: Provide a way for users to compare related variables or "before and after" scenarios to make information more valuable (e.g., previews in Photoshop).
- Show causality: Clarify cause and effect in information graphics and use modeless visual feedback to inform users of consequences.
- Show multiple variables: Display multiple related variables simultaneously without sacrificing clarity, allowing users to turn them on/off.
- Integrate text, graphics, and data in one display: Avoid separate keys or legends, as they add cognitive load.
- Ensure the quality, relevance, and integrity of the content: Only display information that helps users achieve goals relevant to their context.
- Show things adjacently in space, not stacked in time: For changes over time, show them adjacently (like a cartoon strip) rather than superimposed, unless animation is available.
- Don't de-quantify quantifiable data: While using graphs, also display the raw numbers (e.g., a pie chart for disk space with the exact number of kilobytes).
π Example: The Challenger disaster is cited as a classic example where data was not organized to clearly show the causality between air temperature and O-ring failure.
39.3 Use of Text and Color in Visual Interfaces
Text and color are indispensable elements of the visual language of user interfaces.
Use of text: For navigation, text words are best considered as visual elements and should be short and easily recognized. Avoid ALL CAPS as they are harder to read because familiar pattern-matching hints are absent. Minimize text reading for navigation; use visual symbols for context and text for differentiation. When text must be read, ensure high contrast, choose an appropriate typeface (sans-serif like Arial for brief text, serif like Times for paragraphs), and phrase clearly with the fewest words necessary.
π Example: "WORDS TYPED IN ALL CAPITAL LETTERS ARE HARDER TO READ than upper/lowercase β the familiar pattern-matching hints are absent in capitalized words."
Use of color: Color is a powerful design tool. It draws attention, improves navigation speed, and shows relationships.
π Definition β Chromostereopsis: A visual effect where colors from extreme ends of the spectrum (like red and blue) appear to "vibrate" when placed adjacently, making them difficult to read.
Common misuses of color include:
- Using too many colors (more than seven degrades performance).
- Using complementary colors adjacently, causing chromostereopsis.
- Excessive saturation, which looks garish.
- Inadequate contrast (difference only in hue, not saturation or brightness).
- Inadequate attention to color impairment (color blindness in ~10% of males). Important information should not rely solely on red and green hues.
π Example: Red text on a blue background (complementary colors) is extremely difficult to read due to chromostereopsis.
39.4 Consistency and Standards
Consistency in look, feel, and behavior across modules and products is a key goal for usability organizations.
Benefits of interface standards (according to Jakob Nielsen):
- For users: Improved ability to learn interfaces, higher productivity, and fewer errors.
- For vendors: Reduced customer training and support costs, reduced development time, and improved code reuse.
Risks of interface standards:
- The product is only as good as the standard itself.
- Standards primarily emphasize the syntax (visual look and feel) but not the deeper behaviors or logical structure, as they lack context-specific knowledge.
Standards, guidelines, and rules of thumb: Standards should evolve with technology and understanding. They are best treated as detailed guidelines or rules of thumb. Following them too rigidly can result in a poor fit for a specific application.
When to violate guidelines: Standards should be violated when there is a very good reasonβspecifically, when a new idiom is clearly seen to be significantly better by most people in the target user audience. This is how toolbars, tabs, and other innovations came into existence.
Consistency and standards across applications: For companies with multiple software titles, consistency is important for visual branding. However, consistency does not imply rigidity. If different products serve distinct personas with different goals, a single standard may be inappropriate. The "spirit of the law" should guide design, not the "letter of the law."
π Example: Microsoftβs attempt to enforce a similar menu structure in Word, Excel, and PowerPoint was arguably too rigid, as PowerPointβs user model is different.
β Key Takeaways
This lecture is critical for understanding how to design professional, usable interfaces. The key takeaways are: 1) Visual structure, achieved through alignment, grids, symmetry, and white space, is fundamental for user navigation and comfort. 2) Imagery, including icons, must be part of a cohesive visual language that is culturally appropriate, contextually relevant, and simple; visualizing behaviors is often more effective than text alone. 3) Tufte's seven principles for information design provide a powerful framework for presenting data clearly, emphasizing comparisons, causality, and integration. 4) The use of text and color is a core design tool; avoid all-caps, ensure high contrast, and be aware of color blindness and perceptual artifacts like chromostereopsis. 5) Consistency and standards are valuable but should be treated as guidelines to be thoughtfully violated when a new, measurably better idiom emerges that serves user goals.
π§ Quick Revision Questions
- What are the key attributes that help define a crisp visual structure in an interface?
- Explain the "squint test" in the context of visual balance and symmetry.
- List the seven "Grand Principles" of visual information design as defined by Edward Tufte.
- What are the primary risks and limitations of relying too heavily on interface standards?
- Describe the βchromostereopsisβ effect and give an example of a color combination that causes it.
π Lecture 40 β Observing User
π Overview: This lecture focuses on observation as a key evaluation method in Human-Computer Interaction. It covers how to watch and listen to users in both controlled and natural environments, explores different observation frameworks, and explains how to collect, analyze, and present observational data to inform design decisions.
ποΈ Topics Covered
The lecture covers when and what to observe during product development, how to observe in controlled environments versus the field, data collection techniques including notes, audio, video, diaries, and interaction logging, participant observation and ethnography, and finally how to analyze, interpret, and present both qualitative and quantitative observational data to feed findings back into design.
π Lecture Summary
40.1 What and when to observe
Observing is useful at any time during product development. Early in design, observation helps understand users' needs. Later, observation examines whether the developing prototype meets users' needs. Depending on the type of study, evaluators may be onlookers, participant observers, or ethnographers. The degree of immersion varies across a broad outsider-insider spectrum, determined by the study's goals and practical/ethical issues.
40.2 How to observe
The same basic data-collection tools are used for laboratory and field studies (direct observation, notes, video), but used differently. In the laboratory, emphasis is on details of what individuals do, while in the field, context is important with focus on how people interact with each other, technology, and their environment.
In controlled environments: Key practical issues include deciding where users will be located (e.g., usability labs with wall-mounted cameras), testing equipment beforehand, and having informed consent forms and scripts ready to make users comfortable.
In the field: Events can be complex and rapidly changing. Experts use frameworks to structure observation. One simple practitioner's framework focuses on three items:
- The Person β Who is using the technology?
- The Place β Where are they using it?
- The Thing β What are they doing with it?
A more detailed framework by Goetz and LeCompte (1984) encourages attention to context, people, and technology: Who is present? What is happening? When does the activity occur? Where is it happening? How is the activity organized?
Colin Robson (1993) suggests another framework: Space, Actors, Activities, Objects, Acts, Events, Goals, Feelings.
A comprehensive checklist for field observation includes: stating study goals clearly, selecting a framework, deciding how to record events, reviewing notes within 24 hours, separating opinion from fact, being prepared to refocus the study, gaining acceptance and trust, handling sensitive issues, working as a team, checking notes with informants, and looking at situations from different perspectives.
Participant observation and ethnography: Being a participant observer or ethnographer requires acceptance into the group. An example is Nancy Baym's (1997) work where she joined an online soap opera community for over a year. She told the community what she was doing and offered to share findings, gaining trust and support. She kept all messages as data and adapted interviewing and questionnaires.
The distinction between ethnography and participant observation is blurred. Some believe ethnography is an open interpretive approach; others like David Fetterman (1998) see a stronger role for theoretical underpinning: "before asking the first question... the ethnographer begins with a problem, a theory or model, a research design, specific data collection techniques, tools for analysis, and a specific writing style." Ethnographic study allows multiple interpretations of realityβit is interpretive. Data collection and analysis often occur simultaneously.
Fetterman's checklist for ethnography includes: identify a problem or goal and ask good questions; be there to observe; collect a variety of data (notes, pictures, audio, video, artifacts); move between broad picture and specific questions; analyze data using a holistic approach where observations are understood within broad context.
40.3 Data Collection
Data collection techniques (notes, audio recording, video recording) are used individually or in combination, often supplemented with photos.
Notes plus still camera: Taking notes is the least technical but can be difficult and tiring. Handwritten notes are flexible but must be transcribed; this transcription can be the first step in data analysis. A laptop is an alternative but more obtrusive.
Audio recording plus still camera: Less intrusive than video, allows mobility. Two main problems: lack of visual record (handled with a small camera) and transcribing data. Many studies use the recording to remind evaluators of important details and as a source of anecdotes.
Video: Captures both visual and audio data but can be intrusive. Small handheld digital cameras are mobile and inexpensive. Problems: attention becomes focused through the lens, missing other things; sound may get muffled in noisy conditions. Analysis can be very time-consumingβover 100 hours of analysis for one hour of video is common for detailed analyses.
40.4 Indirect observation: tracking users' activities
Sometimes direct observation is not possible. Diaries and interaction logs are used to track users' activities indirectly.
Diaries: Provide a record of what users did, when, and what they thought. Useful when users are scattered and unreachable. Diaries are inexpensive, require no special equipment, and are suitable for long-term studies. Templates can be created online to standardize entry format. Robinson and Godbey (1997) used diaries to record how Americans spent time on various activities. Barry Brown and colleagues (2000) collected diaries from 22 people with handheld cameras to examine information captureβ381 activities were recorded, providing contextual information for designing new digital cameras and handheld scanners.
Interaction logging: Records key presses, mouse movements, etc. Often synchronized with video and audio logs to help evaluators analyze behavior. Logs are time-stamped, allowing calculation of time spent on tasks. Explicit counters record visits to websites and can justify maintenance and upgrades. For example, McLaughlin et al. (1999) analyzed server logs of an interactive art museum over seven months, discovering the site was busiest on weekday evenings. Nonnecke and Preece (2000) compared messages posted with list membership to investigate lurking behavior. An advantage of logging is unobtrusiveness, but ethical concerns must be considered. Large volumes of data can be logged automatically, requiring powerful analysis tools like WebLog (Hochheiser and Shneiderman, 2000).
40.5 Analyzing, interpreting, and presenting the data
Most observational data is qualitative, involving interpreting what users did or said by looking for patterns. The DECIDE framework suggests identifying goals and questions first before selecting techniques, as goals determine which data is collected and how it will be analyzed.
First step: "eyeball" the data to see what stands out. Then proceed to analyze according to goals and questions. Three types of data are discussed:
Qualitative analysis to tell a story: The power lies in telling a convincing story illustrated with powerful examples. Main activities: review data after each session, record themes coherently with examples, record date and time of each analysis session, check understanding with participants, iterate until the story faithfully represents observations, and report findings in oral and written form.
Quantitative data analysis: Video data from usability laboratories is annotated as observed. Evaluators mark the video and record remarks when errors or unusual behavior occur. Performance times are calculated and compared across prototypes. Interaction log data is used similarly. Simple statistics (means, standard deviations, T-tests) are applied. Categorized data may also be quantified and analyzed statistically.
Feeding the findings back into design: Results can be reported through clearly written reports with overviews, detailed content lists, anecdotes, quotations, pictures, and video clips. Both qualitative and quantitative data analysis are useful because they provide alternative perspectives.
β Key Takeaways
Observation is a versatile evaluation method useful throughout product development, from understanding needs early to testing prototypes later. The choice between controlled laboratory observation and field observation depends on whether the focus is on individual task details or contextual social interactions. Frameworks like the Person-Place-Thing model and Goetz and LeCompte's checklist help structure field observations systematically. Data collection methods range from simple note-taking to video recording and interaction logging, each with trade-offs between intrusiveness, richness, and analysis time. The most critical skill is analyzing qualitative data to tell compelling, evidence-based stories that feed directly into design improvements.
π§ Quick Revision Questions
- What are the three main items in the simple practitioner's observation framework for field studies?
- Why is it recommended to review observation notes within 24 hours?
- What is the key difference between laboratory observation and field observation in terms of focus?
- What are two main advantages and two main disadvantages of using video recording for observation?
- How does interaction logging help evaluators understand user behavior, and what ethical concern does it raise?
π Lecture 41 β Asking Users
π Overview: This lecture introduces techniques for gathering user feedback by directly asking users, including interviews and questionnaires. It covers the basics of questionnaire design, how to conduct interviews, heuristic evaluations, and walkthroughs, and explains how to collect, analyze, and present data. Understanding these methods is crucial for evaluating usability and gathering user requirements in human-computer interaction.
ποΈ Topics Covered
The lecture covers four main methods for asking users and experts about system usability: interviews (unstructured, structured, semi-structured, and group interviews), questionnaires (design, administration, and online versions), heuristic evaluation using usability principles, and walkthroughs (cognitive and pluralistic). Each method is presented with its appropriate use cases, strengths, limitations, and practical guidelines for implementation.
π Lecture Summary
41.1 Introduction
Interviews and questionnaires are well-established techniques in social science research, market research, and human-computer interaction. They are used in "quick and dirty" evaluation, usability testing, and field studies to ask about facts, behavior, beliefs, and attitudes. These techniques can be structured or flexible, and they are often combined with observation in field studies. This lecture focuses specifically on interviewing techniques and also covers expert-based evaluation methods like heuristic evaluation and cognitive walkthroughs, which predict how usable interfaces are.
41.2 Asking users: interviews
Interviews can be thought of as a "conversation with a purpose" (Kahn and Cannell, 1957). There are four main types: open-ended or unstructured, structured, semi-structured, and group interviews. The first three types are named according to how much control the interviewer imposes by following a predetermined set of questions. The fourth involves a small group guided by an interviewer who facilitates discussion of specified topics.
The most appropriate approach depends on the evaluation goals, the questions to be addressed, and the paradigm adopted. For gaining first impressions about a new design idea, an informal, open-ended interview is often best. For feedback about a particular design feature, a structured interview or questionnaire is often better because the goals and questions are more specific.
Developing questions and planning an interview When developing interview questions, plan to keep them short, straightforward, and avoid asking too many. Key guidelines include:
- Avoid long questions because they are difficult to remember
- Avoid compound sentences by splitting them into two separate questions. For example, instead of "How do you like this cell phone compared with previous ones that you have owned?" say "How do you like this cell phone? Have you owned other cell phones?"
- Avoid using jargon that the interviewee may not understand
- Avoid leading questions such as "Why do you like this style of interaction?" which assumes the person did like it
- Be alert to unconscious biases and strive for neutrality in your questions
Asking colleagues to review questions and running a pilot study helps identify problems in advance.
When planning an interview, think about interviewees who may be reticent or in a hurry. Include these steps:
- An Introduction β interviewer introduces themselves, explains purpose, reassures about ethics, asks about recording
- A warm-up session β easy, non-threatening questions first
- A main session β questions presented in logical sequence, with difficult ones at the end
- A cool-off period β few easy questions to defuse tension
- A closing session β interviewer thanks interviewee
Further advice for conducting interviews includes: dress similarly to interviewees, prepare an informed consent form, make sure recording equipment works, and record answers exactly without making cosmetic adjustments.
π Definition β Unstructured (Open-ended) Interviews: Interviews more like conversations focusing on a particular topic, where the format and content of answers are not predetermined. The interviewee is free to answer as fully or briefly as they wish.
Unstructured interviews are at one end of the spectrum of interviewer control. They generate rich data and allow exploration of things the interviewer may not have considered. However, they produce a lot of data that is time-consuming and difficult to analyze, and the process is impossible to replicate. Typically, the evaluator makes notes or records the session and later notes main issues of interest.
Key points for unstructured interviews: have an interview agenda, be prepared to follow new lines of enquiry, pay attention to ethical issues, work on gaining acceptance, respond with sympathy but avoid putting ideas into respondents' heads, indicate the beginning and end of the session, and start to analyze data as soon as possible after the interview.
π Definition β Structured Interviews: Interviews that pose predetermined questions similar to those in a questionnaire. Responses may involve selecting from a set of options. The questions are closed, requiring a precise answer, and the same questions are used with each participant for standardization.
Structured interviews are useful when the study's goals are clearly understood and specific questions can be identified. Questions need to be short and clearly worded. Responses may involve selecting from a set of options read aloud or presented on paper. Questions should be refined by review and pilot study.
π Definition β Semi-structured Interviews: Interviews that combine features of structured and unstructured interviews, using both closed and open questions. The interviewer has a basic script for guidance but probes the interviewee for more information.
Semi-structured interviews use both closed and open questions. For consistency, the interviewer has a basic script so the same topics are covered with each interviewee. The interviewer starts with preplanned questions and then probes the interviewee to say more until no new relevant information is forthcoming. For example: "Which websites do you visit most frequently? Why? Tell me more about x? Anything else?"
It is important not to preempt an answer by phrasing a question to suggest a particular answer is expected. The body language of the interviewer can have a strong influence. The interviewer needs to accommodate silence and not move on too quickly. Probes are devices for getting more information, especially neutral probes such as "Do you want to tell me anything else?"
π Definition β Focus Group: A form of group interview frequently used in marketing, political campaigning, and social sciences research, normally involving three to 10 people who share certain characteristics.
Group interviews or focus groups allow diverse or sensitive issues to be raised that would otherwise be missed. The method assumes individuals develop opinions within a social context. A preset agenda guides discussion but allows flexibility for a facilitator to follow unanticipated issues. The facilitator encourages quiet people to participate and stops verbose ones from dominating.
Focus groups appear to have high validity because the method is readily understood. They are low-cost, provide quick results, and can be easily scaled. Disadvantages include the need for a skillful facilitator, difficulty getting people together, and problems scheduling participants.
Other sources of interview-like feedback include: telephone interviews (good for people you cannot meet but you cannot see body language), online interviews via email or chat (good for sensitive issues), video-conferencing systems, customer help lines, consumer groups, and online customer communities. Retrospective interviews can be done during field studies to check understanding.
Data analysis and interpretation Analysis of unstructured interviews can be time-consuming. A coding form may be developed, either predetermined or developed during data collection. Comments may be clustered around themes with anonymous quotes used to illustrate points. Tools like NUDIST and Ethnography can be useful for qualitative analyses. Data from structured interviews is usually analyzed quantitatively as in questionnaires.
41.3 Asking users: questionnaires
π Definition β Questionnaire: A well-established technique for collecting demographic data and users' opinions, similar to interviews, that can have closed or open questions.
Questionnaires are similar to interviews and can have closed or open questions. Effort and skill are needed to ensure questions are clearly worded and data can be analyzed efficiently. One advantage of questionnaires is they can be distributed to a large number of people. Structured interviews are easier and quicker in situations where people will not stop to complete a questionnaire.
Designing questionnaires Many questionnaires start by asking for basic demographic information (e.g., gender, age) and details of user experience (time or years using computers, level of expertise). This background information helps find out the range within the sample group. Following general questions, specific questions that contribute to the evaluation goal are asked. If the questionnaire is long, questions may be subdivided into related topics.
Checklist for designing a questionnaire:
- Make questions clear and specific
- When possible, ask closed questions and offer a range of answers
- Consider including a "no-opinion" option for opinion questions
- Think about ordering β general questions should precede specific ones
- Avoid complex multiple questions
- When using scales, make sure the range is appropriate and does not overlap
- Make ordering of scales intuitive and consistent, and be careful with using negatives
- Avoid jargon and consider different versions for different populations
- Provide clear instructions on how to complete the questionnaire
- Balance white space with compactness to avoid deterring participation
Question and response format Different types of questions require different types of responses, including discrete responses (e.g., "Yes" or "No"), ranges, or single preferred opinions.
Check boxes and ranges: The range of answers to demographic questions is predictable. A common design error occurs when ranges overlap. For example, specifying ranges as 15-20, 20-25 will cause confusion for people who are 20 years old. Making ranges 14-19, 20-24 avoids this problem.
Whether the interval must be equal depends on what you want to know. For example, for an e-commerce site selling life insurance, target population might be 21-65 years, so ranges could be: under 21, 21-65, and over 65. For ten-year cohort groups, ranges would be: under 21, 22-31, 32-41, etc.
Administering questionnaires Two important issues are reaching a representative sample and ensuring a reasonable response rate. For large surveys, potential respondents need to be selected using a sampling technique. Interaction designers tend to use small numbers (often fewer than 20 users). 100% completion rates are often achieved with small samples, but with larger populations, 40% return is generally acceptable.
Ways to encourage a good response include: ensuring good design, providing a short overview section, including a stamped self-addressed envelope, explaining why the questionnaire needs to be completed, contacting respondents through follow-up letters/phone calls/emails, and offering incentives.
Online questionnaires Online questionnaires are effective for reaching large numbers quickly. There are two types: email and web-based. Email allows targeting specific users but is usually limited to text. Web-based questionnaires are more flexible, allowing check boxes, pull-down menus, help screens, graphics, immediate data validation, and enforcement of rules.
Advantages of online questionnaires:
- Responses are usually received quickly
- Copying and postage costs are lower or nonexistent
- Data can be transferred immediately into a database for analysis
- Time required for data analysis is reduced
- Errors in questionnaire design can be corrected easily
A big problem with web-based questionnaires is obtaining a random sample of respondents. There is some evidence suggesting response rates may be lower online than with paper questionnaires.
41.4 Asking experts: heuristic evaluation
π Definition β Heuristic Evaluation: An informal usability inspection technique developed by Jakob Nielsen in which experts, guided by a set of usability principles known as heuristics, evaluate whether user-interface elements conform to the principles.
Heuristic evaluation involves experts guided by a set of usability principles called heuristics. The original set was derived empirically from an analysis of 249 usability problems. The latest set of heuristics includes:
- Visibility of system status β Are users kept informed about what is going on? Is appropriate feedback provided within reasonable time?
- Match between system and the real world β Is the language simple? Are words, phrases and concepts familiar to the user?
- User control and freedom β Are there ways of allowing users to easily escape from places they unexpectedly find themselves in?
- Consistency and standards β Are ways of performing similar actions consistent?
- Help users recognize, diagnose, and recover from errors β Are error messages helpful? Do they use plain language and suggest a way of solving the problem?
- Error prevention β Is it easy to make errors? If so, where and why?
- Recognition rather than recall β Are objects, actions and options always visible?
- Flexibility and efficiency of use β Have accelerators (shortcuts) been provided for experienced users?
- Aesthetic and minimalist design β Is any unnecessary or irrelevant information provided?
- Help and documentation β Is help information provided that can be easily searched and followed?
For evaluating commercial websites, Nielsen suggests the HOME RUN heuristics: High-quality content, Often updated, Minimal download time, Ease of use, Relevant to users' needs, Unique to the online medium, Net-centric corporate culture.
Different sets of heuristics are needed for different products (toys, WAP devices, online communities, wearable computers). Using a set of heuristics, expert evaluators role-play typical users and note problems. Empirical evidence suggests that five evaluators usually identify around 75% of total usability problems.
π‘ Why this matters: Heuristic evaluation is a quick and cost-effective method for identifying usability problems without involving users directly, making it ideal for early design evaluation when prototypes may not be ready for user testing.
41.5 Asking experts: walkthroughs
π Definition β Walkthroughs: An alternative approach to heuristic evaluation for predicting users' problems without doing user testing, involving walking through a task with the system and noting problematic usability features.
Walkthroughs involve walking through a task with the system and noting problematic usability features. Most walkthrough techniques do not involve users. Others, such as pluralistic walkthroughs, involve a team including users, developers, and usability specialists.
Cognitive walkthroughs π Definition β Cognitive Walkthrough: A method involving simulating a user's problem-solving process at each step in the human-computer dialog, checking to see if the user's goals and memory for actions can be assumed to lead to the next correct action.
Cognitive walkthroughs focus on evaluating designs for ease of learning. The steps involved are:
- Identify characteristics of typical users, develop sample tasks focusing on aspects of the design to be evaluated, and produce a description or prototype of the interface along with a clear sequence of actions needed for the task.
- A designer and one or more expert evaluators come together to do the analysis.
- Evaluators walk through action sequences for each task, trying to answer these questions:
- Will the correct action be sufficiently evident to the user? (Will the user know what to do?)
- Will the user notice that the correct action is available? (Can users see the button or menu item?)
- Will the user associate and interpret the response from the action correctly? (Will users know from feedback whether the action was correct or not?)
- As the walkthrough is done, a record is compiled including assumptions about problems, notes about side issues, and a summary of results.
- The design is then revised to fix the problems.
It is important to document the cognitive walkthrough, using a standardized feedback form and documenting the severity of problems.
Strengths: Focuses on users' problems in detail, users do not need to be present, and a working prototype is not necessary. Limitations: Very time-consuming and laborious, and has a narrow focus that is useful for certain types of systems but not others.
Pluralistic walkthroughs π Definition β Pluralistic Walkthrough: A type of walkthrough in which users, developers, and usability experts work together to step through a task scenario, discussing usability issues associated with dialog elements.
Pluralistic walkthroughs involve a team that includes users, developers, and usability experts. Each group is asked to assume the role of typical users. The steps are:
- Scenarios are developed as a series of hard-copy screens representing a single path through the interface.
- Scenarios are presented to the panel of evaluators, and panelists individually write down the sequence of actions they would take to move from one screen to another.
- When everyone has written down their actions, the panelists discuss the actions they suggested. Typically, representative users go first, then usability experts, and finally developers.
- The panel moves on to the next round of screens until all scenarios have been evaluated.
Benefits: Strong focus on users' tasks, performance data is produced, and the approach lends itself well to participatory design practices by involving a multidisciplinary team where users play a key role. Limitations: Requires getting all experts together at once, proceeds at the rate of the slowest, and only a limited number of scenarios can be explored due to time constraints.
β Key Takeaways
The most critical things to remember from this lecture are the four main methods for gathering user feedback: interviews (unstructured for rich exploratory data, structured for specific questions, semi-structured for a balance, and focus groups for group dynamics), questionnaires (carefully designed with clear questions, appropriate scales, and no overlapping ranges), heuristic evaluation (using Nielsen's 10 heuristics to identify usability problems without users, with five evaluators finding about 75% of problems), and walkthroughs (cognitive for learning-focused analysis and pluralistic for collaborative evaluation with users). Each method has specific strengthsβinterviews provide depth, questionnaires provide breadth, heuristic evaluation is quick and expert-driven, and walkthroughs focus on task-based analysis. The golden rules include avoiding leading questions, piloting before full use, ensuring representative sampling, designing for good response rates, and documenting problems with severity ratings for effective design revision.
π§ Quick Revision Questions
- What are the four main types of interviews described in this lecture, and what distinguishes each type in terms of interviewer control and question format?
- How can you avoid a common design error when creating age range options in a questionnaire, and what is the problem with overlapping ranges?
- What are Nielsen's 10 heuristics for heuristic evaluation, and approximately what percentage of usability problems can five evaluators typically identify using this method?
- What are the three key questions evaluators must answer during a cognitive walkthrough, and how do these questions relate to the user's learning process?
- What are the advantages and limitations of using online questionnaires compared to paper-based questionnaires, particularly regarding response rates and sampling?