π Lecture 06 β Online Focus Group
π Overview: This lecture covers the transition from traditional to online focus groups and provides a detailed examination of descriptive research design in marketing research. It explains the fundamental differences between cross-sectional and longitudinal studies, and provides a comprehensive overview of survey methods including personal interviews, telephone interviews, and mailed questionnaires.
ποΈ Topics Covered
The lecture begins with an explanation of online focus groups and their advantages/disadvantages compared to traditional focus groups, followed by applications of focus groups in marketing research and dos and don'ts. It then moves into descriptive research design, covering its purpose and classification into cross-sectional and longitudinal studies. The lecture concludes with a detailed examination of survey methods, including personal interviews (in-home, mall intercept, computer-assisted), telephone interviews, and mailed questionnaires/mail panels.
π Lecture Summary
Online Focus Group
Apart from the traditional focus group, it can now be conducted online. The online focus group size is usually four to six participants and can be conducted anywhere in the world. It continues for about an hour or so. Group dynamics are limited. It is inexpensive as compared to traditional focus group, but the data gathered through conventional focus group are more valuable than online focus group.
Advantages of Focus Group
Several advantages exist:
- Synergism: Several people put together to produce information
- Snowballing: A person's comment triggers a chain reaction
- Security: Group security; participants feel free to express themselves
- Spontaneity: Spontaneous responses, not planned, are accurate
- Speed: Many people contribute ideas
- Client observes the group: Through one-way glass
- Flexibility: Can change with the situation
Disadvantages of Focus Group
- Misjudgment: Subjective interpretation β a trained moderator is needed
- Cost: Per participant is high
- Sample: Not representative
- Moderation: Difficult to manage
- Domination of some members: Control of the moderator is needed
Application of Focus Group in Marketing Research
Focus groups can be used in almost any situation requiring some preliminary understanding and insight. In marketing research, they address issues like:
- Developing copy for advertisement
- Obtaining an impression about price of the product
- New product concept testing
- Understanding consumer perceptions, preference, and purchase behavior
- Finding new ideas about old products
- Generating hypotheses for research
Doβs and Donβts about Focus Group
- Be sure to get the right people in the group
- Avoid judging participants on physical appearance
- With focus groups, fewer is better than many
- Never do a focus group without planning
- Never lose sight of the objectives of the research for which the focus group is being conducted
- Hire a trained and qualified moderator
- Complete the report and submit it quickly
Descriptive Research Design
The major purpose of descriptive research is to describe marketing functions or characteristics. It provides answers to questions such as who, what, how, when, where as they relate to the research problem. Descriptive research is typically conducted to answer the following basic questions to formulate effective marketing strategies:
- Describe the characteristics of relevant groups such as consumers, market areas, salespersons, etc.
- Percentage of units in a population showing certain behavior. For example, percentage of heavy users of a brand.
- Product characteristics as perceived by the market/customers.
- Degree of association of different marketing variables. For example, association of income and buying quantity. Such association does not mean cause-and-effect relationship.
- To make predictions about the occurrence of marketing phenomena. For example, what will be the sales of Bata stores in Okara during December 2007?
Classification of Descriptive Research
Descriptive research can further be classified into two types: cross-sectional and longitudinal.
Cross-Sectional Studies Cross-sectional studies measure data from a sample at one point in time. The sample is like a cross-section of the population. If you have to measure the population at different times, every time a new sample should be taken. Cross-sectional studies are quite prevalent in marketing research. As data from the sample are collected only once, they are also called snapshot studies. Usually, cross-sectional studies use a large sample, which is why these are called survey research β the method with which people are most familiar.
Longitudinal Research Longitudinal research is a type of research design which involves a fixed sample measured repeatedly on the same variables. In cross-sectional design, the sample changes every time, but in longitudinal research, the sample remains the same over time. In cross-sectional studies, there is only one picture or snapshot, but in longitudinal studies, there are series of pictures which provide a view of the changes that have taken place over time.
The term "panel" is used to describe a longitudinal design. A panel contains a sample of respondents who have agreed to provide information at specified intervals over an extended period of time. A panel may be a group of customers, experts, households, or stores. Panel members are compensated for their participation by gifts, coupons, or cash.
There are two types of panels:
- Traditional panel: A fixed sample where the same variables are measured repeatedly.
- Omnibus panel: A fixed sample which is measured repeatedly, but the variables measured are different.
Traditional panel studies can be used to analyze how members switched from one brand to another from one time period to the next. Another use of longitudinal study is that of market tracking studies. Through such studies, changes over time can be measured. A marketer can track how their brand is doing compared to other brands by having representative data on brand market share.
π‘ Why this matters: Cross-sectional studies give a snapshot of the market at one moment, while longitudinal studies reveal trends and changes over time, which is critical for understanding brand loyalty and market dynamics.
Methods Used in Descriptive Design
There are basically two methods employed in descriptive research: survey and observation.
Survey method is based on questioning of respondents. Questions may be asked verbally, in written form, or through a computer. Questions may be asked about awareness, interactions, attitude, motivation, demographics, or lifestyle of the respondents.
Questions in a survey may be structured or unstructured. Structured questions, which are usually asked in survey research, are standardized and direct. Unstructured questions are open and do not have prearranged answer choices.
Advantages of Surveys
Comparatively, survey methods allow collection of a significant amount of data in an economical and efficient manner. Survey methods typically allow larger samples. There are at least four advantages:
- Standardization: Survey methods ensure that all respondents are asked the same questions. The same order of questions and same choices of answers are given to each respondent. The sequence of the questions is the same. In short, a standard questionnaire is administered to all respondents.
- Ease of Administration: Whether personal interview, telephonic interview, computer-assisted interview, or mailed questionnaire, survey methods are easy to administer. Mailed questionnaires are perhaps the simplest method. There are no tape recordings, note-taking, or analyzing projective or physiological data.
- Find Out "Unseen": Much of the unseen data can be found out through direct questions. For example, we can find out by asking respondents how many stores they visited before making a final purchase. Similarly, we can ask about income, family size, or occupation β information that was otherwise unobservable by the researcher. All unobservable information can be obtained through direct questioning.
- Suitable for Tabulation and Statistical Analysis: Survey methods are designed so that tables can easily be prepared and statistical packages like SPSS can be easily used. In contrast to qualitative research where samples are small and data prove frustrating when subjected to statistical analysis, large cross-sectional surveys perfectly suit these statistical procedures.
Classification of Survey Method
Survey methods can be classified based on the mode used to administer the questionnaire. These are:
- Personal interview
- Telephonic interview
- Mailed questionnaire
Personal Interview In personal interviews, respondents are interviewed face-to-face. The surveyor or interviewer reads the questions to the respondent and records the answers in writing, audiotape, or videotape form. This has been a primary method of survey for many years, but its use is declining due to advances in technology and rising costs. Nevertheless, personal interviews are still used due to their following advantages:
a. Feedback: Non-verbal cues give feedback to the interviewer as to whether the respondent is understanding the question or not. Clarification may be provided by the interviewer, or the question may be adjusted accordingly.
b. Rapport: Due to the personal presence of the interviewer, a rapport can be developed with the interviewee. Thus, the respondent starts relating to the interviewer and answers the questions more openly. Respondents are more truthful when they are face-to-face.
c. Quality control: For interviews, most of the time the researcher selects respondents based on certain distinguishing characteristics relevant to the research. Personal interviewers must ensure that respondents are selected correctly before the interview takes place. This helps maintain quality of the research.
d. Adaptability: Personal interviewers can easily adapt to the needs of different respondents. They can help an elderly respondent understand how to respond to "somewhat agree" or "strongly agree." Similarly, they can give examples to respondents on other issues. They can adjust the direction. If the interviewer does not understand the answer, they can ask probing questions to get clarification or depth in the answer.
Personal interviews have three main disadvantages: they are expensive, take more time, and lend themselves to interviewer's bias. The interviewer may ask leading questions or give hints to a specific answer, which is not desirable in research.
Categories of Personal Interview Personal interviewing may be categorized into three types: a. In-home personal interviews: Take place in the home or office of the respondent according to the convenience of the interviewer. The interviewer fixes an appointment for date, time, and place of the interview prior to going. b. Mall intercept personal interviews: Respondents are shoppers at shopping malls, and they are intercepted while shopping in the store or outside the store. Sometimes they are brought to an office facility built by the research firm in the shopping mall for this purpose. Mall intercepts take place in high-traffic shopping areas. Mall intercept has acquired a major role due to its ease of implementation. It is less expensive than depth interviews in home or office. This has a very low cost per interviewee. However, representativeness of mall intercept samples is always an issue. Some shoppers refuse to give an interview. Another disadvantage is that the environment of shopping malls is not as comfortable as an office or home; therefore, long questionnaires cannot yield meaningful information. c. Telephonic interview: Where physical contact with the respondent is not possible or is expensive, telephonic interviewing is an attractive option.
Advantages of telephonic interviews include speed and cost. Telephone is relatively inexpensive for collecting data. Telephone surveys are also quick β approximately one good interview per hour can be conducted.
Disadvantages include: First, you cannot demonstrate or show anything to the respondent on the phone; therefore, when showing an advertisement or package is important, telephone interviews are not a good alternative. Second, telephone interviews do not permit observation of body language, facial expression, or eye contact, so the interviewer is deprived of observational judgment available in face-to-face interviews. Third, the information obtained is more limited in quantity because people do not like to answer many questions on the phone and hang up quickly. Telephone interviews are a poor choice for surveys with many open-ended questions. They also have great potential for fake interviewees. An additional problem in countries like Pakistan is that most women respondents hesitate to be interviewed on the phone.
Computer-Assisted Interviewing (CAPI and CATI) Computers can assist interviewers in both personal and telephone interviews. Computer Assisted Personal Interview (CAPI) and Computer Assisted Telephone Interview (CATI) are being used in marketing research in advanced countries, but these are in the development stage in developing countries.
The Internet is also being used in marketing research. E-mail interviews or surveys are not uncommon. E-mail addresses are obtained, and questions are e-mailed to these addresses. Respondents type answers to close-ended questions at designated places and click on "reply" to send them back.
Mailed Questionnaire Another type of data collection is the mail interview, usually through mailed questionnaires or mail panels. In a typical mailed interview, questionnaires are sent to respondents by mail. The questionnaire is accompanied by a return envelope and a cover letter. Prior to that, respondents had been identified through an appropriate sampling method, and a mailing list was developed. The cover letter appeals to the respondent in an effective manner to return the questionnaire in the enclosed return envelope with postage stamps affixed by the researcher.
Mail surveys have advantages and disadvantages. Advantages include very low cost per respondent and convenience β respondents can fill out the questionnaire at their convenience. The main disadvantage is a very low response rate. People are more willing to participate in a survey for a worthy reason or incentive.
Mail Panel A variation of the mailed interview is the mail panel. A mail panel consists of a large sample of households who have agreed to participate in mailed questionnaires, product tests, or telephone surveys periodically. They either volunteer to do so or are provided with some incentive. Data from panels is updated periodically. The panel is pre-screened to ensure that panel members represent the target market or consumer of interest.
β Key Takeaways
The most critical points from this lecture are the distinction between cross-sectional and longitudinal research designs, where cross-sectional provides a single snapshot and longitudinal tracks changes over time using a fixed panel. Focus groups, whether traditional or online, serve exploratory purposes but have limitations in representativeness and moderation. Within descriptive research, survey methods dominate and include personal interviews (in-home, mall intercept, and computer-assisted), telephone interviews, and mailed questionnaires β each with distinct trade-offs in cost, speed, data quality, and response rate. The key advantages of surveys β standardization, ease of administration, ability to uncover unseen data, and suitability for statistical analysis β make them the most widely used method. Finally, the presence of interviewer bias in personal interviews and low response rates in mailed questionnaires are critical limitations to remember for exam scenarios.
π§ Quick Revision Questions
- What is the fundamental difference between cross-sectional and longitudinal research designs in terms of sampling?
- What are the four main advantages of survey methods mentioned in the lecture?
- What are the three categories of personal interviews, and what are the key disadvantages of personal interviews overall?
- What are the two types of panels used in longitudinal research, and how do they differ?
- Why is mail intercept interviewing popular despite its representativeness issues?
π Lecture 7 β Mailed Questionnaire
π Overview: This lecture covers mail-based survey methods and observation techniques in marketing research. It explains the design and administration of mailed questionnaires and mail panels, then provides a detailed comparison of all survey methods. The second half introduces observation methods including their classifications, types, advantages, and limitationsβcritical for collecting unbiased behavioral data.
ποΈ Topics Covered
The lecture covers mailed questionnaires and mail panels as data collection methods, followed by a comprehensive comparison table of personal interview, mall intercept, telephone interview, mail survey, and mail panels across factors like volume of data, speed, response rate, flexibility, accuracy, sensitivity, bias potential, time, expense, and sample control. It then introduces observation methods, explaining structured vs. unstructured, participant vs. non-participant, undisguised vs. disguised, and natural vs. contrived observation. Additional methods include human observation, mechanical observation, trace analysis, content analysis, and retail/wholesale audit, concluding with advantages and limitations of observation.
π Lecture Summary
Mailed Questionnaire
Another type of data collection is the mail interview, usually through mailed questionnaires or mail panels. In a typical mailed interview, questionnaires are sent to respondents by mail, accompanied by a return envelope and a cover letter. Prior to that, respondents are identified through appropriate sampling and a mailing list is developed. The cover letter appeals to the respondent to return the questionnaire in the accompanied return envelope with postage stamps affixed by the researcher.
The mail surveys have some advantages and disadvantages. Advantages include very low cost per respondent and convenience for the respondent to fill out the questionnaire at their convenience. The main disadvantage is a very low response rate. People are more willing to participate in a survey for a worthy reason or incentive.
Mail Panel
A variation of mailed interview is the mail panel. A mail panel consists of a large sample of households who have agreed to participate in mailed questionnaires and product tests or telephone surveys periodically. They either volunteer or are provided some incentive. Data from panels is updated periodically. The panel is prescreened to ensure that panel members represent the target market or consumer of interest.
Comparison of Survey Methods
The above mentioned survey methods are evaluated on different factors in the following table.
A Comparison of Different Survey Methods
| Factor | Personal interview | Mall intercept | Telephone interview | Mail survey | Mail panels |
|---|---|---|---|---|---|
| Volume of data | more | more | less | less | less |
| Speed | moderate | moderate/high | high | low | low |
| Response rate | high | high | moderate | low | moderate |
| Flexibility of data collection | high | high | moderate | low | low |
| Accuracy of data | very good | very good | good | very good | very good |
| Obtaining sensitive information | low | low | high | high | moderate/high |
| Possibility of interview bias | high | high | moderate | nil | nil |
| Time consumed | high | moderate | moderate | moderate | moderate |
| Expense | high | moderate | moderate | low | low/moderate |
| Sample control | high | moderate | moderate | low | moderate/high |
Observation Methods
Another method of data collection in descriptive research is observation, which records the behavior pattern of people, objects, or events in a systematic manner. Observation is limited to providing information about current behavior. There is no question or communication of observer with the people being observed. Observation can be structured and unstructured, disguised and undisguised, or natural vs. contrived.
π‘ Why this matters: Observation captures actual behavior, not reported behavior, eliminating recall errors and response bias common in surveys.
Structured vs. Unstructured
In structured observation, the researchers determine in advance what behaviors are to be observed and recorded. The researcher prepares a checklist of these behaviors and ignores all others. Structured observation is useful in conclusive research. In unstructured observation, the researcher observes all the episodes under study and records whatever they find interesting and relevant. No details of what will be observed are set in advance. Observer's bias in unstructured observation is potentially high, and this type is more suitable for exploratory research.
Participant vs. Non-Participant Observation
Participant observation is a method in which the observer or researcher participates in the process being observed. For example, they can be a customer, a worker, or a trainee. In non-participant observation, the observer is just an observer.
Undisguised and Disguised Observation
When the subjects are informed that they are being observed for some purpose, such observation is known as undisguised observation. When the subjects are unaware that they are being observed, this observation is called disguised observation. Sometimes the observer disguises the observation process by using one-way mirrors or hidden cameras.
Natural versus Contrived Observation
Behavior can be observed in natural or artificial settings. If the observation takes place in a natural environment, it is known as natural observationβfor example, a researcher observing the behavior of respondents while eating at McDonald's. It is called contrived observation when the respondent is brought into an artificial setting and observed. Tests done by varying shelf space, product flavors, and display locations fall under contrived observation.
Other Methods of Observation
The methods of observation based on mode of administration are classified as below:
-
Human Observation: In this method, the observer is the researcher themselves or a person hired for this purpose. The observer merely records what they observe.
-
Mechanical Observation: In this observation, a mechanical device rather than the human eye observes the phenomenon. Such mechanical devices include people meter, audiometer, on-site-cameras, turnstiles (to count people entering or leaving a building), and scanners. Scanners are good devices to collect information about consumer purchases by product category, price, quality, brand, and store type.
-
Trace Analysis: This is the observation of some traces of the event that has passed. Data collection is based on physical evidence or traces of past behavior. It is also known as unobtrusive method. Examples include:
- Erosion of tile in a building
- The magazines that were donated to charity showing the popularity of magazines
- Wear and tear of pages in a journal showing its readership
- Age and condition of the car in the parking lot to determine the affluence of customers
- Number of empties of Coke or Pepsi to determine the consumption of Coke or Pepsi
-
Content Analysis: It is the observation as well as analysis of various content of communication which is manifest. In marketing research context, applications of content analysis involve observation and analysis of the message of advertisement, radio and television programs, newspaper articles, and the like. In international marketing, content analysis has successfully been used in studying foreign cultures and cross-culture ads.
-
Retail and Wholesale Audit: In this method of observation, the researcher checks physical records or performs inventory analysis. This method is discussed in detail in the section on syndicated services.
Advantages and Limitations of Observation
Observation can be used to supplement and complement other research techniques to check on results obtained by others. Ideally, the subjects of observational research should not be aware that they are being observed, so they react in a natural manner. This provides the researcher insight into actual, not reported behavior. Subjects are not asked about somethingβthey are observed, so there is no chance for recall error. Data obtained through observation is up-to-date and correct if interpreted correctly.
In some cases, observation may be the only choice to obtain correct information. For example, children who cannot express their opinion about a new toy can be observed while playing or not playing with the new toy. Observation methods can successfully collect marketing intelligence in retail marketing, and employee behavior through "mystery shoppers"βtrained observers who pose as customers in competitors' stores.
Observation has limitations too. Due to small samples and subjective interpretation, results are usually considered tentative. A big limitation is that beliefs and internal conditions of the subjects cannot be observedβyou cannot tell what is going beneath the surface behavior. Observation methods are successful when such feelings are unimportant for the research.
Another problem is that observation is sometimes time-consuming and expensive, with potential for observer's bias. It is suggested that observation should not be used alone but in combination with other survey methods. It is estimated that not more than only one percent of marketing research projects rely solely on observational methods for obtaining primary data.
β Key Takeaways
The mailed questionnaire method offers low cost and respondent convenience but suffers from very low response rates, making mail panels a useful variation with prescreened households. The comparison table shows that personal interviews and mall intercepts have high response rates and flexibility but also high bias potential and expense, while mail surveys have nil bias and low expense but low response rates and sample control. Observation methods are essential for capturing actual behavior without recall error or response bias, with classifications including structured/unstructured, disguised/undisguised, natural/contrived, and human/mechanical/trace/content/audit approaches. The major limitation of observation is its inability to capture internal beliefs and feelings, and it should rarely be used aloneβonly about 1% of projects rely solely on observation.
π§ Quick Revision Questions
- What is the main disadvantage of mailed questionnaires, and how does a mail panel attempt to address this?
- According to the comparison table, which survey method has the highest speed and which has the highest response rate?
- What is the key difference between structured and unstructured observation, and which type of research is each best suited for?
- Give two examples of mechanical observation devices and two examples of trace analysis.
- Why is observation considered to provide "actual, not reported behavior," and what is its biggest limitation?
π Lecture 08 β Casual Research
π Overview: This lecture introduces causal research and the conditions required to establish causality between variables. It defines key experimental concepts like independent, dependent, and extraneous variables, details the components of experimental design, and explains the critical threats to internal and external validity. Understanding these concepts is essential for designing valid experiments in marketing research.
ποΈ Topics Covered
The lecture begins with the definition and three necessary conditions for causality: concomitant variation, time order, and absence of other causal factors. It then defines key experimental concepts including independent and dependent variables, test units, extraneous variables, and experiments. The lecture explains experimental design and introduces standard symbols. It covers the distinction between internal and external validity, enumerates seven specific threats to internal validity (history, maturation, testing effects, instrumentation, statistical regression, selection bias, and mortality), and concludes with four methods for controlling extraneous variables: randomization, matching, statistical control, and design control.
π Lecture Summary
Casual Research
Causality is defined as when the occurrence of X increases the probability of the occurrence of Y. Before making causal inferences, three conditions must be satisfied: concomitant variation, time order of occurrence of variables, and elimination of other possible causal factors.
π Definition β Concomitant Variation: the extent to which a cause (X) and an effect (Y) occur together or vary together in the way predicted by the hypothesis under consideration.
π Definition β Time Order of Occurrence of Variables: the causing event must occur either before or simultaneously with the effect; it cannot occur afterwards.
π Definition β Absence of Other Possible Causal Factors: the factor or variable being investigated should be the only possible causal explanation.
Definitions and Concepts
The lecture defines basic experimental concepts:
- Independent Variables: variables or alternatives that are manipulated (i.e., the levels are changed by the researcher) and whose effects are measured and compared. Also known as treatments, they may include price levels, package design, and advertising themes.
- Test Units: individuals, organizations, or other entities whose response to the independent variables or treatments is being examined. Test units may include consumers, stores, or geographic areas.
- Dependent Variables: variables that measure the effect of the independent variables on the test units. These may include sales, profits, and market shares.
- Extraneous Variables: all variables other than the independent variables that affect the response of the test units. These variables can confound the dependent variable measures, weakening or invalidating the results of the experiment. Examples include store size, store location, and competitive effort.
- Experiment: formed when the researcher manipulates one or more independent variables and measures their effect on one or more dependent variables, while controlling for the effect of extraneous variables.
Experimental Design
An experimental design is a set of procedures specifying:
- The test unit and how these units are to be divided into homogenous subsamples
- What independent variables or treatments are to be manipulated
- What dependent variables are to be measured
- How the extraneous variables are to be controlled
Definition of Symbols
A set of symbols is defined to facilitate discussion of extraneous variables and specific experimental designs:
- X = the exposure of a group to an independent variable, treatment, or event the effects of which are to be determined
- O = the process of observation or measurement of the dependent variable on the test units or group of units
- R = the random assignment of test units or groups to separate treatment
Additional conventions:
- Movement from left to right indicates movement through time
- Horizontal alignment of symbols implies all those symbols refer to one specific group, treatment, or control
- Vertical alignment of symbols implies all those symbols refer to activities or events that occur simultaneously
π Example: The symbol arrangement X Oβ Oβ means that a given group of test units was exposed to the treatment variable (X) and the response was measured at two different points in time (Oβ and Oβ).
Validity in Experimentation
When conducting an experiment, a researcher has two goals:
- Draw valid conclusions about the effects of independent variables on the study group
- Make valid generalizations to a larger population of interest
Internal validity concerns the first goal, while external validity concerns the second.
π Definition β Internal Validity: refers to whether the manipulation of the independent variables or treatments actually caused the observed effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity.
π Definition β External Validity: refers to whether the cause and effect relationships found in the experiment can be generalized. It is desirable to have an experimental design that has both internal and external validity, but in applied marketing research often we have to trade one type of validity for another.
π‘ Why this matters: Understanding the trade-off between internal and external validity is crucial for designing experiments that are both rigorous and applicable to real-world marketing decisions.
Threats to Internal Validity
Following are seven threats to internal validity in the form of extraneous variables:
- History (H): refers to specific events that are external to the experiment but occur at the same time as the experiment. These events may affect the dependent variable. For example, if general economic conditions decline during the experiment and the local area is particularly hard hit by layoffs and plant closings. The longer the time interval between observations, the greater the possibility that history will confound an experiment.
- Maturation (MA): refers to changes in the test units themselves that occur with the passage of time, involving people. Maturation takes place as people become older, more experienced, tired, bored, or uninterested. Stores change over time in terms of physical layout, dΓ©cor, traffic, and composition.
- Testing Effects: caused by the process of experimentation. The main testing effect (MT) occurs when a prior observation affects a latter observation.
- Instrumentation (I): refers to changes in the measuring instruments that are modified during the course. Instrumentation effects are likely when interviewers make pre- and post-treatment measurements. The effectiveness of interviewers can be different at different times.
- Statistical Regression (SR): effects occur when test units with extreme scores move closer to the average score during the course of the experiment. People with extreme attitudes have more room for change, so variation is more likely.
- Selection Bias (SB): refers to the improper assignment of test units to treatment conditions. If test units self-select their own groups or are assigned to groups on the basis of the researcher's judgment, selection bias is possible. For example, in a merchandising experiment where two different displays (old and new) are assigned to different department stores, the stores may not be equivalent and may vary with respect to store size, which is likely to affect sales regardless of which display was assigned.
- Mortality (MO): refers to the loss of test units while the experiment is in progress. This happens for many reasons, such as test units refusing to continue in the experiment. Mortality confounds results because it is difficult to determine if the lost test units would respond in the same manner to treatment as those that remain.
Controlling Extraneous Variables
Extraneous variables confound the results and are also called confounding variables. There are four ways of controlling extraneous variables:
- Randomization: refers to the random assignment of test units to experimental groups by using random numbers. Treatment conditions are also randomly assigned to experimental groups. Randomization may not be effective when the sample size is small.
- Matching: involves comparing test units on a set of key background variables before assigning them to the treatment conditions.
- Statistical Control: involves measuring the extraneous variables and adjusting for their effects through statistical analysis (ANCOVA).
- Design Control: involves the use of experiments designed to control specific extraneous variables.
β Key Takeaways
To establish causality, three conditions must be simultaneously met: concomitant variation, correct time order (cause before effect), and the elimination of other possible causal factors. A valid experiment requires the researcher to manipulate independent variables while controlling for extraneous variables that threaten internal validity. The seven major threats to internal validity are history, maturation, testing effects, instrumentation, statistical regression, selection bias, and mortality. Extraneous variables can be controlled through four methods: randomization, matching, statistical control (ANCOVA), and design control. There is often a necessary trade-off between internal validity (accuracy of causal conclusions within the experiment) and external validity (generalizability to the larger population).
π§ Quick Revision Questions
- What are the three necessary conditions for establishing causality in marketing research?
- Differentiate between independent variables, dependent variables, and extraneous variables in an experiment.
- Explain the threat of "selection bias" to internal validity and provide an example.
- How does "statistical regression" threaten the internal validity of an experiment?
- Name and briefly describe the four methods for controlling extraneous variables in an experimental design.