๐ Lecture 13 โ The Computer
๐ Overview: This lecture shifts focus from human aspects to computer aspects of Human Computer Interaction. It provides a comprehensive overview of input and output devices, examining their design, usability, and suitability for different tasks and users. Understanding these devices is crucial for designing systems that are safe, effective, efficient, and enjoyable to use.
๐๏ธ Topics Covered
This lecture begins by defining input devices and their selection criteria, emphasizing the importance of matching devices to user physiology, task, and environment. It then explores various text entry devices such as keyboards (QWERTY, alphabetic, Dvorak, chord), phone pads with T9, handwriting, and speech recognition. The lecture continues with a detailed discussion of positioning and pointing devices including the mouse, touch pad, trackball, joystick, touch screen, stylus, and eyegaze. It concludes with an overview of display devices (CRT, LCD, digital paper), sound output, tactile feedback, physical controls, and environmental sensing.
๐ Lecture Summary
13.1 Input devices
Input is concerned with recording and entering data into a computer system and issuing instructions. Input devices can be defined as a device that, together with appropriate software, transforms information from the user into data that a computer application can process. The key aim in selecting an input device is to help users carry out their work safely, effectively, efficiently, and enjoyably. The most appropriate device will match the physiology and psychological characteristics of users, be appropriate for the tasks to be performed (e.g., continuous movement for drawing, discrete movement for selecting from a list), and be suitable for the intended work and environment.
Frequently, no single optimal device can be identified, and trade-offs must be made. Many systems use two or more input devices together, which must be complementary and well coordinated. There must also be adequate and appropriate system feedback to guide users. Feedback can be visual (text appearing, cursor moving), auditory (an alarm, keys clicking), or tactile (using a joystick), often in combination.
๐ Definition โ System Feedback: Responses from the system to guide, reassure, inform, and correct userโs errors. It can be visual, auditory, or tactile.
13.2 Text entry devices
Keyboard The most common method of entering information is through a keyboard. Broadly defined, a keyboard is a group of on-off push buttons. It is a discrete entry device, sensing one of two or more discrete positions, as opposed to continuous entry devices which sense in a continuous range. The physical design of individual keys and their grouping arrangements are important. For example, keys that are too small cause difficulty in hitting them accurately. Membrane keyboards are sealed and can withstand grease, dirt, and liquids, making them suitable for environments like production floors, but they lack tactile feedback. Alterations in key arrangement affect speed and accuracy. Research shows trained typists look ahead and process text in chunks of two to three words for alphabetic text and three to four characters for numerical material.
๐ Definition โ Discrete Entry Device: An input device that senses one of two or more discrete positions (e.g., keyboard keys, touch-sensitive switches). ๐ Definition โ Continuous Entry Device: An input device that senses in a continuous range (e.g., pens with digitizing tablets, moving joysticks).
QWERTY keyboard
This standard alphanumeric layout, named after the first letters in the uppermost row, became a commercial success in 1874. The arrangement was chosen to reduce keys jamming in manual typewriters, not for optimal typing. For example, the letters s, t, and h are far apart even though they are frequently used together.
Alphabetic keyboard This layout arranges letters alphabetically. Despite expectations, it is not faster for properly trained typists and makes little difference to the speed for novice or occasional users. It is sometimes used in pocket electronic organizers to dissuade touch-typing on a very small keyboard.
Dvorak Keyboard Patented in 1932, the Dvorak board was designed based on the frequency of letter usage and patterns in English. All vowels and the most frequent consonants are on the home row, so approximately 70% of common words are typed on this row. It promotes faster operation by tapping with alternate hands. Dvorak claimed this arrangement reduces between-row movement by 90%. Despite its benefits, it has never been commercially successful due to the cost of replacing existing keyboards and retraining users.
๐ Example: On a Dvorak keyboard, the vowels a, o, e, and u and the most frequent consonants are typed on the home row, reducing finger travel.
Chord keyboards On chord keyboards, several keys must be pressed at once to enter a single character, similar to playing a flute. Few keys are required, so they can be very small and operated with one hand. They are useful where space is limited or one hand is occupied. Training is required to learn the finger combinations. They are used for mail sorting and recording court transcripts.
Phone pad and T9 entry
Mobile phones use a keypad with digits 0-9, not a full keyboard. For text input, numeric keys are pressed multiple times. For example, the 3 key has def; pressing it once gives d, twice gives e, three times gives f. The T9 algorithm uses a large dictionary to disambiguate words by typing each letter only once. For example, 3926753 becomes example as it is the only real word that matches.
๐ Example: To type the word "example" using T9, you press 3926753. The phone's dictionary recognizes this sequence as "example" because alternative letter combinations (like ewbosld) are not real words.
Handwriting recognition Handwriting is a familiar activity, making it an attractive text entry method. However, current technology is often inaccurate and individual differences in handwriting are enormous. The most significant information is in the stroke informationโthe way the letter is drawnโso devices must capture this. Online recognition is easier than reading text on paper. A key limitation is speed, as it is difficult to write more than 25 words per minute, which is half the speed of a decent typist. Pen-based systems are popular in mobile computing for taking notes and sketching, as they can be small and accurate, unlike small keyboards.
Speech recognition Speech input offers several advantages: it is a natural form of communication, does not require hands, and aids disabled users. However, it suffers from several problems: it is limited to specialized tasks, can be inaccurate in distinguishing similar-sounding words, is subject to background noise, and natural language is hard for computers to interpret. Systems range from isolated word recognition (requiring pauses between words) to continuous speech recognition (allowing faster, more natural input). Speaker-dependent systems require training, while speaker-independent systems accommodate a large range of speaking characteristics but are less reliable.
๐ Definition โ Continuous Speech Recognition System: A system capable of recognizing words spoken in a natural, flowing manner without pauses between words, allowing faster data entry.
13.3 Positioning, Pointing and Drawing
Pointing devices are input devices used to specify a point or path in space. Examples include the mouse, touch pad, trackball, joystick, touch screen, and eye gaze.
Mouse The mouse is a small, palm-sized box with a weighted ball. As the box is moved, the ball rotates inside, and this rotation is detected by rollers. It is an indirect input device because a transformation maps horizontal desktop motion to vertical screen alignment. Left-right motion is directly mapped, while up-down is achieved by moving the mouse towards or away from the user.
๐ Definition โ Indirect Input Device: An input device for which a transformation is required to map its movement to the screen's output (e.g., mouse to screen cursor).
Foot mouse A foot-operated device, akin to an isometric joystick, that allows hands to be dedicated to the keyboard. It is a rare device.
Touch pad Touchpads are touch-sensitive tablets, usually 2-3 inches square, operated by stroking a finger over their surface. Because they are small, they may require several strokes to move the cursor across the screen. This can be improved by acceleration settings in the software, where the pad-to-screen distance ratio varies with the speed of finger movement.
๐ Example: If you move your finger slowly on a touchpad, it maps to small screen movements. If you move your finger quickly, the same distance on the pad moves the cursor a long distance on the screen.
Trackball and thumbwheel A trackball is an upside-down mouse with a weighted ball in a static housing. It is compact, requires no additional space, but is hard for drawing. Thumbwheels have two orthogonal dials to control the cursor position; they are cheap but slow and difficult to use for diagonal movement. Single thumbwheels are often included on mice for scrolling documents.
Joystick and track point The joystick is an indirect input device consisting of a stick in a box. An absolute joystick has its stick position correspond to the cursor position. An isometric joystick uses pressure on the stick to control cursor velocity, and the stick returns to center when released. A track point is a smaller version used on laptops, often a rubber nipple in the center of the keyboard.
Touch screens Touch screens allow input by touching the screen, making it a bi-directional device (input and output). Advantages are ease of learning, durability, and direct interaction. Disadvantages include greasy marks on the screen, inaccuracy (making small selections and drawing difficult), and arm fatigue from reaching to a vertical screen.
Stylus and light pen A stylus is a pen-like stick for pointing and drawing, popular in PDAs. A light pen is an older technology connected to the screen by a cable. It detects a burst of light from the screen phosphor during the display scan, allowing it to address individual pixels, making it more accurate than a touch screen.
Eyegaze Eyegaze systems control the computer by tracking eye movement. A low-power laser is shone into the eye, and the reflection changes as the eye angle alters. The system determines the direction of the gaze. It is fast and accurate but can be expensive. It is fine for selection but not for drawing because the eye does not move in smooth lines, and it can be difficult to distinguish a deliberate gaze from an accidental glance.
Cursor keys
Four keys on the keyboard (up, down, left, right) are used to control the cursor. There is no standardized layout, but the most common is the inverted T. They were more heavily used in character-based systems.
13.4 Display devices
Cathode ray tube The CRT is a television-like screen. An electron gun emits a stream of electrons, which are focused and directed by magnetic fields to hit a phosphor-coated screen, causing it to glow. The beam is scanned from left to right, top to bottom. Color is achieved using three electron guns for red, green, and blue phosphors. CRTs are cheap with fast response times and high color capability but are bulky.
Liquid Crystal Display LCDs are light, flat plastic screens using liquid crystal technology. They are smaller, lighter, and consume far less power than CRTs. They are matrix addressable, meaning individual pixels can be accessed without scanning. They have no radiation problems and are common in notebook and portable computers.
๐ Definition โ Matrix Addressable: A display technology where individual pixels can be accessed directly without the need for scanning the entire screen.
Digital paper A new form of display that is thin, flexible, and can be written to electronically. It retains its contents even when removed from an electrical supply.
Sound output Auditory signals are used in conjunction with screen displays for system output and feedback. Sounds like beeps, clicks, and tones convey information. For example, keyboards can emit a click when a key is pressed, speeding up performance.
13.5 Touch, feel and smell
The sense of touch and feel is used for tactile feedback. Technology for simulating textures is just beginning to become available.
13.6 Physical controls
Dedicated control panels are designed for a single device and use, unlike generic desktop computer controls. For example, a microwave uses a flat plastic control panel because it is used in a kitchen with greasy hands. The smooth surface has no gaps for food to accumulate and is easy to clean.
๐ Example: A microwave has a smooth, flat control panel with no gaps, making it easy to clean, while a washing machine has large buttons that act as both controls and displays, as hands are less greasy.
13.7 Environment and bio sensing
Many sensors exist in our environment, such as for automatic doors and energy-saving lights. The vision of ubiquitous computing suggests our world will be filled with such devices that monitor our behavior.
โญ Key Takeaways
The choice of input and output devices is a critical design decision in HCI, requiring a trade-off between user characteristics, task requirements, and environmental constraints. For text input, the QWERTY keyboard remains dominant despite its suboptimal design, while alternatives like Dvorak offer ergonomic benefits but face adoption barriers. For pointing, the mouse is a standard, but other devices like touch screens and trackpads are better suited for specific contexts (e.g., public kiosks, laptops). System feedbackโvisual, auditory, and tactileโis essential for guiding users and preventing errors. Finally, emerging technologies like speech recognition and eyegaze offer new interaction paradigms, particularly for users with disabilities, though they still face significant limitations in accuracy and naturalness.
๐ง Quick Revision Questions
- Explain the three key factors to consider when selecting an input device for an interactive system.
- Compare and contrast the QWERTY, Dvorak, and alphabetic keyboard layouts, focusing on their design principles and usability.
- Describe the difference between an absolute joystick and an isometric joystick.
- What are the primary advantages and disadvantages of using a touch screen as an input device?
- Explain the difference between speaker-dependent and speaker-independent speech recognition systems and give an example of a problem each might face.
๐ Lecture 14 โ Interaction
๐ Overview: This lecture introduces the concept of interaction in Human Computer Interaction, explaining how humans and computers communicate. It covers key terminology, theoretical models of interaction, and various interaction styles, providing a foundational understanding of how users engage with computer systems.
๐๏ธ Topics Covered
The lecture defines core terms of interaction including domain, task, and goal, then presents Donald Normanโs model of interaction with the seven stages of action and the gulfs of execution and evaluation. It introduces an interaction framework breaking the system into four components (System, User, Input, Output) and discusses how frameworks relate to HCI, focusing on ergonomics, physical aspects of interfaces, industrial interfaces, glass interfaces, indirect manipulation, and the physical environment. Finally, it describes various interaction styles including command line interface, menus, natural language, question/answer and query dialog, form-fills and spreadsheets, WIMP, point and click, and three-dimensional interfaces.
๐ Lecture Summary
14.1 The terms of Interaction
Domain defines an area of expertise and knowledge in some real-world activity. Examples include graphic design, authoring, and process control in a factory. A domain consists of concepts that highlight its important aspects. In a graphic design domain, important concepts are geometric shapes, a drawing surface, and a drawing utensil.
Tasks are the operations to manipulate the concepts of a domain. For example, constructing a specific geometric shape with particular attributes on the drawing surface is a task within the graphic design domain.
Goal is the desired output from a performed task. A related goal would be to produce a solid red triangle centered on the canvas. So, the goal is the ultimate result you want to achieve after performing specific tasks.
14.2 Donald Normanโs Model
Donald Normanโs Model of interaction describes how a user chooses a goal, formulates a plan of action, which is then executed at the computer interface. When the plan has been executed, the user observes the computer interface to evaluate the result and determine further actions.
The two major parts, execution and evaluation, of the interactive cycle are subdivided into seven stages. For example: you decide you need more light (goal), form an intention to switch on the desk lamp, specify actions to reach over and press the switch, execute the action, perceive the result (light on or off), interpret this based on knowledge of the world, and evaluate against the original goal.
๐ Definition โ Gulf of execution: the difference between the userโs formulation of the actions to reach the goal and the actions allowed by the system. If the actions allowed correspond to those intended by the user, the interaction will be effective.
๐ Definition โ Gulf of evaluation: the distance between the physical presentation of the system state and the expectation of the user. If the user can readily evaluate the presentation in terms of their goal, the gulf of evaluation is small.
๐ก Why this matters: These two gulfs represent common interface problems. Good interface design aims to minimize both gulfs to make interaction more intuitive and effective.
14.3 The interaction framework
The interaction framework breaks the system into four main components: the System, the User, the Input, and the Output. Each component has its own language. The systemโs language is the core language (describing computational attributes of the domain relevant to system state), and the userโs language is the task language (describing psychological attributes of the domain relevant to user state). Input and output together form the interface.
There are four steps in the interactive cycle, each corresponding to a translation between components: articulation (user formulates task and articulates it in input language), performance (input language is translated into core language as operations), presentation (system state is rendered as output features), and observation (user observes output and assesses results relative to the original goal).
14.4 Frameworks and HCI
The ACM SIGCHI Curriculum Development Group presents a framework placing different areas relating to HCI. Ergonomics addresses issues on the user side of the interface, covering input and output as well as the userโs immediate context. Dialog design and interface styles are placed along the input branch, while presentation and screen design relate to the output branch.
Ergonomics (or human factors) is the study of the physical characteristics of interaction: how controls are designed, the physical environment, and the layout and physical qualities of the screen. Physical aspects of interface include:
- Arrangement of controls and displays: Users should group sets of controls and parts of the display logically for rapid access, especially in safety-critical applications.
- Industrial Interface: Industrial interfaces may require rapid assimilation of multiple numeric displays varying in response to the environment, raising additional design issues.
- Glass interfaces vs. dials and knobs: A glass interface (computer screen) can be cheaper and more flexible for complex systems, allowing information to be shown in multiple forms.
- Indirect manipulation: Industrial interfaces are intermediaries between operator and the real world, requiring feedback at two levels: immediate feedback that actions were received, and monitoring of equipment effects.
The physical environment of the interaction considers where the system will be used and by whom. Health issues include physical position (comfortable reach and support), temperature (extremes affect performance), lighting (adequate without glare), noise (maintain comfortable levels), and time (control usage duration).
Use of color: The human visual system has limitations regarding color, including the number of distinguishable colors and relatively low blue acuity. Colors should be distinct, not affected by contrast changes, and correspond to common conventions and user expectations, though color conventions are culturally determined.
14.5 Interaction styles
Interaction is communication between computer and human. For successful enjoyable communication, interface style has its own importance. Common interface styles include:
-
Command line interface: The first interactive dialog style, providing direct access to system functionality using function keys, single characters, abbreviations, or whole-word commands. They are powerful, flexible with options/parameters, and useful for repetitive tasks.
-
Menu: Options available to the user are displayed on screen and selected using mouse or keys. Since options are visible, they rely on recognition rather than recall. Menus are often hierarchically ordered; grouping and naming provides cues for finding options.
-
Natural Language: Attractive but difficult because the ambiguity of natural language makes it very difficult for a machine to understand.
-
Question/answer and query dialog: Question/answer leads users through interaction step by step โ easy to learn but limited in functionality. Query languages construct queries to retrieve information from databases using natural-language-style phrases but require specific syntax and knowledge of database structure.
-
Form-fills and spreadsheets: Form-filling interfaces present a display resembling a paper form with slots to fill in, allowing easy movement and correction facilities. Spreadsheets comprise a grid of cells, each containing a value or formula. Modern example: MS Excel; historical examples: VISICALC and Lotus 123.
-
The WIMP Interfaces: Stands for Windows, Icons, Menus, and Pointers. This is the default interface style for most interactive computer systems today, especially in PC and desktop workstation environments.
-
Point and Click interface: In multimedia systems and web browsers, virtually all actions take only a single click. Pointing at a city on a map and clicking opens tourist information; pointing at a word shows a definition.
-
Three-dimensional interfaces: Increasing use of 3D effects in user interfaces. The most obvious example is virtual reality (VR), but simpler techniques include giving ordinary WIMP elements a 3D appearance using shading.
โญ Key Takeaways
The lecture establishes that interaction is the core of HCI โ the communication between human and computer users. Normanโs model with its seven stages of action and the concepts of gulf of execution and gulf of evaluation are critical for understanding how users interact with systems and for identifying interface problems. The interaction framework with its four components (System, User, Input, Output) and four translations (articulation, performance, presentation, observation) provides a systematic way to analyze interaction. Ergonomics and physical considerations including health issues, environmental factors, and color use directly impact user performance and safety. Finally, the various interaction styles (from command line to WIMP to 3D interfaces) each have distinct advantages and limitations that make them appropriate for different contexts and user types.
๐ง Quick Revision Questions
- What are the four main components in the interaction framework, and what are the four translations involved in the interactive cycle?
- Explain the difference between the gulf of execution and the gulf of evaluation in Normanโs model. Provide an example of each.
- What are the seven stages of action in Normanโs model of interaction? Use the desk lamp example to illustrate them.
- List and briefly describe at least five different interaction styles covered in this lecture.
- What physical environment and health issues should be considered in ergonomic design of interfaces?
๐ Lecture 15 โ Interaction Paradigms
๐ Overview: This lecture provides a detailed examination of WIMP interfacesโWindows, Icons, Pointers, and Menusโalong with related widgets like toolbars, buttons, and dialog boxes. It then explores the evolution of interaction paradigms, from time-sharing and personal computing to modern concepts like ubiquitous computing and the World Wide Web, showing how technological advances have reshaped human-computer interaction.
๐๏ธ Topics Covered
This lecture covers two main sections: first, a detailed breakdown of the WIMP interface components including windows, icons, pointers, menus, keyboard accelerators, buttons, radio buttons, check boxes, toolbars, palettes, and dialog boxes. The second section introduces interaction paradigms, discussing time-sharing, video display units, programming toolkits, personal computing, WIMP interface history, the metaphor, direct manipulation, language versus action paradigms, hypertext, multi-modality, computer-supported cooperative work, the World Wide Web, ubiquitous computing, and sensor-based interaction.
๐ Lecture Summary
15.1 The WIMP Interfaces
The lecture begins by revisiting and expanding on the four key features of the WIMP interface: windows, icons, pointers, and menus. Beyond these, many additional interaction objects and techniques are commonly used, including toolbars, menus, buttons, palettes, and dialog boxes. Collectively, these elements are called widgets, and they form the toolkit for interaction between user and system.
Windows Windows are areas of the screen that behave as if they were independent terminals. They can contain text or graphics, be moved or resized, and allow multiple tasks to be visible simultaneously. When one window overlaps another, the back window is partially obscured and refreshed when exposed again. Overlapping windows can obscure vital information, so windows may also be tiled (adjoining without overlapping) or placed in a cascading fashion (each new window offset slightly down and to the left). Scrollbars allow users to move window contents up/down or side to side, making the window behave like a real window onto a larger world. A title bar at the top identifies the window, and corner boxes aid resizing, closing, or maximizing. Some systems also allow windows within windows, such as in Microsoft Office where each application has its own window and each document has a sub-window.
Icons Windows can be closed and lost forever, or they can be shrunk to a much-reduced representation called an icon. By allowing icons, many windows can be available on the screen simultaneously, ready to be expanded by clicking. Iconifying a window suspends that dialog temporarily, saving screen space and serving as a reminder that the dialog can be resumed. Icons can also represent other system aspects (e.g., a wastebasket, disks, programs). Icons can be realistic, stylized, or arbitrary symbols, though arbitrary symbols can be difficult for users to interpret.
Pointers The pointer is essential because WIMP interaction relies heavily on pointing and selecting. The mouse provides input, though joysticks and trackballs are alternatives. The user sees a cursor on the screen controlled by the input device. Different cursor shapes distinguish modes (e.g., arrow for normal, cross-hairs for drawing). Cursors also indicate system activity (e.g., an hourglass cursor when the system is busy). Pointer cursors have a hot-spot, the exact location to which they point.
Menus A menu presents a choice of operations or services available at a given time. Since recall is inferior to recognition, menus provide visual cues in an ordered list. The pointing device highlights items as the pointer moves over them, and selection usually requires pressing a mouse button or keyboard key. Cascading menus allow refinement by opening another menu adjacent to the selected item. Main menus can be permanently visible as a menu bar (often at screen/window top) or hidden as pop-up menus that appear upon request, often context-sensitive. Pull-down menus require pressing a button to drag down, while fall-down menus appear automatically when the pointer enters the title bar. Pie menus arrange options in a circle with the pointer in the center, making selection distance equal for all items, though they take more screen space. The major problems with menus are deciding what items to include and how to group them; menu labels should reflect function, items grouped by function, groupings consistent across applications, and items ordered by importance and frequency of use.
Keyboard accelerators Keyboard accelerators are key combinations that have the same effect as selecting a menu item, allowing expert users to avoid moving off the keyboard. Accelerators are often displayed alongside menu items so frequent use makes them familiar.
Buttons Buttons are individual, isolated regions within a display that can be selected to invoke specific operations. They are designed to resemble push buttons on a control panel. "Pushing" a button invokes a command, usually indicated by a textual label or icon.
Radio Buttons Radio buttons are toggle buttons grouped together to allow selection of one feature from a set of mutually exclusive options (e.g., font size in points).
Check boxes Check boxes are collections of toggle buttons used when a set of options is not mutually exclusive (e.g., bold, italic, underline), indicating the on/off status of each option.
Toolbars Toolbars are collections of small buttons with icons, placed at the top or side of a window, offering commonly used functions. They are similar to menu bars but allow more functions to be displayed simultaneously because icons are smaller than text. Users can often customize the toolbar's content or choose from predefined sets.
Palettes Palettes are a mechanism for making the set of possible modes and the active mode visible to the user. A mode changes the interpretation of user actions (e.g., keystrokes). A palette is usually a collection of icons reminiscent of each mode's purpose, like an artist's palette for paint colors. Users may be able to "tear off" menus or drag toolbars to create palettes placed anywhere on the screen.
Dialog boxes Dialog boxes are information windows used to bring the user's attention to important information (e.g., an error, warning) or to invoke a sub-dialog for a specific task (e.g., saving a file). They factor out auxiliary task threads from the main task dialog.
15.2 Interaction Paradigms
An interaction paradigm is a particular philosophy or way of thinking about interaction design, intended to orient designers to the kinds of questions they need to ask. For many years, the prevailing paradigm was the desktop, with a single user using a GUI/WIMP interface. Recent trends move beyond the desktop with wireless, mobile, and handheld technologies.
Time sharing In the 1940s-1950s, advances were in hardware. By the 1960s, J.C.R. Licklider at ARPA financed research to channel growing computing power. A major contribution was time-sharing, where a single computer supported multiple users. Previously, batch sessions required submitting jobs on punched cards. Time-sharing made programming interactive, creating a subculture of "hackers."
Video display units In the 1950s, researchers experimented with video display units (VDUs). In 1962, Ivan Sutherland's Sketchpad program demonstrated two key ideas: computers could be used for more than data processing, extending the user's ability to abstract and visualize; and one creative mind could make a significant contribution to computing history.
Programming toolkits Douglas Engelbart aimed to use computers to complement human problem-solving. His idea of teaching humans through computers contrasted with the prevailing view that computers were complex and only for the intellectually privileged.
Personal computing Programming toolkits increased productivity for skilled users. The 1970s saw computing aimed at the masses. LOGO was a graphics programming language for children, using a computer-controlled turtle. Alan Kay envisioned small, powerful, single-user machinesโpersonal computers. He worked at Xerox PARC on Smalltalk, a visually based programming environment.
Window systems and the WIMP interface Personal computing focused on single users, but humans think about multiple things at once. A system that forces linear progression does not match human working patterns. The WIMP interface (windows, icons, menus, pointers) became commonplace, first appearing commercially in April 1981 with Xerox's 8010 Star Information System.
The metaphor Metaphor is used to teach new concepts in terms of familiar ones. The Xerox Alto and Star used the desktop metaphor for file manipulation tasks. The spreadsheet is another successful metaphor. However, metaphors can have dangers after the initial honeymoon period (e.g., the typewriter metaphor for word processors, where space is a character, not a passive movement). Metaphors also carry cultural bias that may not apply across national boundaries.
Direct Manipulation Direct manipulation, a term coined by Ben Shneiderman in 1982, describes graphics-based interactive systems. Its features include:
- Visibility of objects of interest
- Incremental action with rapid feedback
- Reversibility of all actions
- Syntactic correctness of all actions
- Replacement of complex command languages with direct manipulation of visible objects
The first commercial success demonstrating direct manipulation for the public was the Apple Macintosh in 1984. The direct manipulation interface for the desktop metaphor makes documents and folders visible as icons; moving a file is mirrored by dragging its icon.
Language versus action Direct manipulation makes some tasks easier but others more difficult. The action paradigm (direct manipulation) involves performing actions at the interface. The language paradigm involves giving instructions via indirect language, with the interface acting as an interlocutor or agent between user and system. The action paradigm is easier for simple tasks; the language paradigm allows describing generic procedures for repeatable tasks. Programming by example combines both, where the user performs actions and the system records them as a generic script.
Hypertext In 1945, Vannevar Bush described the memex, an information storage/retrieval apparatus mimicking human ability to create random associative links. In the 1960s, Nelson coined hypertext to reflect non-linear text structure, inspired by the memex. Traditional text is linear; hypertext allows random, associative browsing.
Multi-modality A multi-modal interactive system relies on multiple human communication channels. Each different channel is a modality. Traditional systems use keyboard and mouse (visual and haptic channels). Genuine multi-modal systems use simultaneous use of multiple channels for both input and output, mimicking natural human information processing.
Computer-supported cooperative work (CSCW) The 1960s saw the first computer networks. As networks became widespread, individuals reconnected their workstations, leading to computer-supported cooperative work (CSCW) . These systems allow interaction between humans via the computer, and designers must consider the society within which users operate. A fine example is electronic mail (email) .
The World Wide Web The World Wide Web (WWW or web) is built on top of the Internet, offering an easy-to-use, graphical interface to information. The Internet is a collection of computers linked by data connections using common transmission protocols. The web adds its own network protocol, a standard markup notation (HTML), and a global naming scheme. Web pages can contain text, images, movies, sound, and most importantly, hypertext links to other pages.
Ubiquitous computing In the late 1980s, Mark Weiser at Xerox PARC initiated a program to move HCI away from the desktop into everyday life. Ubiquitous computing (or pervasive computing) aims to create a computing infrastructure that permeates our physical environment so we no longer notice the computer. The electric motor is a good analogy: it was once large and noticeable, but now it is ubiquitous and invisible.
Sensor-based and context-aware interaction Weiser's dream was "computers anymore." There are increasing technologies that embed computation unobtrusively into daily life, from mobile devices to more pervasive environments.
โญ Key Takeaways
Students must remember the four core components of the WIMP interfaceโwindows, icons, pointers, and menusโand understand how each functions and their associated issues (e.g., window overlap, icon recognition, menu grouping). The concept of interaction paradigms is critical: these are philosophies that guide design, and they have evolved from time-sharing to personal computing, direct manipulation, and beyond. The metaphor is a powerful but potentially dangerous tool for teaching new concepts. Direct manipulation features (visibility, rapid feedback, reversibility, syntactic correctness, and direct action) contrast with language-based interaction paradigms. Finally, modern paradigms like hypertext, CSCW, the World Wide Web, and ubiquitous computing represent fundamental shifts in how we think about and design human-computer interaction.
๐ง Quick Revision Questions
- What are the four key features of the WIMP interface, and what is the primary purpose of each?
- Explain the difference between pull-down menus, fall-down menus, and pop-up menus.
- What are the five features of a direct manipulation interface as defined by Ben Shneiderman?
- What is the "metaphor" in HCI, and what are its potential dangers?
- What is the core idea behind ubiquitous computing, and how does the electric motor analogy help explain it?