π Lecture 9 β Experimental Designs
π Overview: This lecture covers various experimental designs used in marketing research, including one-shot case studies, pretest-posttest control group designs, and Latin square designs. It also explains the differences between laboratory and field experiments, the limitations of experimentation, and introduces test marketing procedures. Finally, the lecture introduces fundamental concepts of measurement and scaling, including the four basic types of rating scales.
ποΈ Topics Covered
The lecture begins by describing several experimental designs: one-shot case study, static group design, pretest-posttest control group design, posttest-only control group design, randomized block design, and Latin square design. It then discusses experimental settings (laboratory vs. field experiments) and the limitations of experimentation (time, cost, administration). The topic of test marketing is covered, including standard, controlled, and simulated test markets. Finally, the lecture introduces measurement and scaling concepts, distinguishing between objective and subjective properties, and explains the four basic scales: nominal, ordinal, interval, and ratio.
π Lecture Summary
Experimental Designs
Various experimental designs are described below.
One-Shot Case Study
Also known as the after only design, the one-shot case study may be symbolically represented as X Oβ. Here, X represents the exposure of a group to an experimental treatment, and Oβ represents the measurement of the dependent variable after the treatment. This design lacks a control group and a pretreatment measure, making it very weak for establishing causality.
Static Group Design
The static group design involves two groups: an experimental group that receives the treatment and a control group that does not. It is symbolically represented as:
- X Oβ
- Oβ
π Definition β Treatment Effect (TE): The impact of the experimental manipulation on the dependent variable, calculated as the difference between the experimental and control group outcomes. π Formula: TE = Oβ - Oβ β The treatment effect is measured by subtracting the measurement of the control group (Oβ) from the measurement of the experimental group (Oβ). π Example: If Oβ (sales after an advertising campaign) = 100 units and Oβ (sales without the campaign) = 70 units, then TE = 100 - 70 = 30 units of sales attributed to the campaign.
Pretest-Posttest Control Group Design
In the pretest-posttest control group design, test units are randomly assigned to either the experimental or the control group, and a pretreatment measure is taken on each group. This design is symbolized as:
- R Oβ X Oβ
- R Oβ Oβ
Where R denotes random assignment, Oβ and Oβ are pretreatment measures, X is the treatment, and Oβ and Oβ are posttreatment measures.
π Definition β Treatment Effect (TE): In this design, the treatment effect accounts for both the change in the experimental group and any change in the control group, isolating the true impact of the treatment. π Formula: TE = (Oβ - Oβ) - (Oβ - Oβ) β The treatment effect is the difference between the change in the experimental group and the change in the control group. π Example: Experimental group: Oβ = 50, Oβ = 70 (change = +20). Control group: Oβ = 55, Oβ = 60 (change = +5). TE = (70-50) - (60-55) = 20 - 5 = 15. This design controls for most extraneous variables. Selection bias is eliminated by randomization.
Posttest-Only Control Group Design
This design is similar to the pretest-posttest control group design but does not include a pretreatment measure. Test units are randomly assigned to experimental or control groups, and only a posttreatment measure is taken. Randomization is assumed to ensure the groups are equivalent before the treatment.
Randomized Block Design
A randomized block design is useful when there is only one major external variable, such as sales, store size, or income of the respondent, that might influence the dependent variable. This design groups test units into blocks based on the external variable, then randomly assigns treatments within each block to control for that variable's effect.
Latin Square Design
A Latin square design allows the researcher to statistically control two non-interacting external variables as well as to manipulate the independent variable. It ensures that each level of the independent variable appears exactly once in each row and column, balancing the effects of the two external variables.
An example of Latin square design:
| Store patronage \ Interest in the store | High | Medium | Low |
|---|---|---|---|
| High | B | A | C |
| Medium | C | B | A |
| Low and none | A | C | B |
In this example, the independent variable (e.g., a store feature) is varied across three levels (A, B, C), while store patronage and interest in the store are the two controlled external variables.
Experimental Settings
There are two types of settings in which experiments are conducted.
Laboratory Experiment
It is an artificial setting for experimentation in which the researcher constructs the desired conditions. The laboratory environment offers a high degree of control because it isolates the experiment in a carefully monitored environment.
Field Experiment
An experimental location is set in actual market conditions. Field experiments are conducted in real-world environments, offering greater external validity but less control over extraneous variables.
Limitations of Experimentation
- Time: Experiments can be time consuming.
- Cost: Experiments are often expensive.
- Administration: Experiments can be difficult to administer. It may be impossible to control for the effects of the extraneous variables, particularly in a field environment.
Test Marketing
Test marketing, also called market testing, is an application of controlled experiment, done in limited but carefully selected parts of the marketplace called test markets. It involves a replication of a planned national marketing program in the test markets. Often, the marketing mix variables (independent variables) are varied in test marketing, and the sales (dependent variable) are monitored so that an appropriate national marketing strategy can be identified.
The two major objectives of test marketing are:
- To determine market acceptance of the product
- To test alternative levels of marketing mix variables.
Test marketing procedures may be classified as standard test markets, controlled and min-market tests, and simulated test marketing.
Standard Test Market
It is a test market in which the product is sold through regular distribution channels. For example, no special considerations are given to products simply because they are being test-marketed. The duration of the test depends on the repurchase cycle for the product, the probability of competitive response, cost considerations, the initial consumer response, and company philosophy. The test should last long enough for repurchase activity to be observed. If competitive reaction to the test is anticipated, the duration should be short. Recent evidence suggests that tests of new brands should run for at least 10 months.
Controlled Test Market
A test-marketing program conducted by an outside research company in field experimentation. The research company guarantees distribution of the product in retail outlets that represent a predetermined percentage of the market.
Measuring and Scaling
Measurement in marketing research is determining how much of a property/characteristic is possessed by an object. Measurement is to determine the intensity of some characteristic of interest to the researcher.
Now what are we really measuring? We are not measuring objects but we are measuring some propertiesβcalled attributes or characteristics. Thus we do not measure buyers but their characteristics like their preferences or perceptions. Objects in marketing research are consumers, brands, stores, advertisements, etc. Properties are characteristics of an object that can be used to distinguish one object from the other.
Properties may be: a. Objective properties which are physically verifiable characteristics such as age, income, number of cans purchased, store last visited, and so on. b. Subjective properties which cannot be directly observed and are mental constructs such as perceptions, attitudes. Subjective properties are observer-able and intangible into a rating scale. There are several examples of both these properties in marketing, e.g., market potential for new product, sales of existing product, demographic and psychographic characteristics of the buyers, effectiveness of a new advertising campaign, market share, and the like.
Rating Scales
Subjective properties, e.g., attitude, beliefs, intentions, preferences, etc., are measured with the help of rating scales usually called scales. In a scale, numbers are assigned to the amount of characteristic or βconstructβ being measured. The number varies according to the amount of characteristic available in the object. There are four basic scales.
- Nominal scale: A nominal scale is one in which numbers serve as labels to identify or categorize objects or events. All numbers are equal with respect to characteristics of objects. Each number is assigned to only one object and each object has only one number. The number in a nominal scale does not have any relationship with the amount of characteristic. For example, a unique number is given to each player in a football team but a player having number 8 does not play better twice than a player having number 4. Although nominal scales are used for the lowest form of measurement, yet nominal scales are frequently used in marketing research. Nominal level identifications are needed in marketing research to identify brands, store types, sales territories, customers, gender, geographic location, race, religion, buyer/non-buyer, heavy and light users, etc. The numbers assigned to such categories are mutually exclusive. Alphabets, even symbols, could be used instead of numbers in nominal scales. Nominal scales simply label objects and do not provide information on greater than or less than. Usually counting is the permissible operation in nominal scales. Thus, statistics like frequency distribution, percentages, mode, chi-square, etc., are used while analyzing data gathered by nominal scale. Average in these scales is meaningless.
β Key Takeaways
The most critical concepts from this lecture are the different experimental designs and their formulas for calculating treatment effects, particularly the pretest-posttest control group design which controls for extraneous variables through randomization and pretreatment measures. Understanding the distinction between laboratory and field experiments is essential, as is recognizing the limitations of experimentation including time, cost, and administration challenges. Test marketing proceduresβstandard, controlled, and simulatedβare key applications of controlled experiments in real market conditions. Finally, the foundation of measurement in marketing research requires distinguishing between objective and subjective properties, and understanding that nominal scales are the simplest measurement level where numbers serve only as labels without quantitative meaning.
π§ Quick Revision Questions
- What is the formula for calculating the treatment effect (TE) in a pretest-posttest control group design?
- How does a Latin square design differ from a randomized block design in controlling external variables?
- What are the two major objectives of test marketing?
- Why is calculating an average meaningless for data collected using a nominal scale?
- What is the key difference between objective and subjective properties in marketing research measurement?
π Lecture 10 β Rating Scales
π Overview: This lecture covers the four fundamental types of rating scales used in marketing research to measure subjective properties like attitudes, beliefs, and preferences. It explains the hierarchy of measurement from nominal to ratio scales, and details comparative scaling techniques including paired comparison, rank order, and constant-sum scales, which are essential for collecting and analyzing marketing data.
ποΈ Topics Covered
This lecture introduces the four basic rating scales: nominal, ordinal, interval, and ratio scales, explaining their nature, applications in marketing research, and permissible statistical analyses. It then covers comparative scaling techniques, specifically paired comparison, rank order, and constant-sum scales, detailing their procedures, advantages, disadvantages, and practical examples for measuring preferences and attribute importance.
π Lecture Summary
Subjective properties e.g. attitude, beliefs, intentions, preferences etc. are measured with the help of rating scales usually called scales.
In a scale, numbers are assigned to the amount of characteristic or βconstructβ being measured. The number varies according to the amount of characteristic available in the object. There are four basic scales.
1. Nominal scale
A nominal scale is one in which numbers serve as labels to identify or categorize objects or events. All numbers are equal with respect to characteristics of objects. Each number is assigned to only one object and each object has only one number. The number in a nominal scale does not have any relationship with the amount of characteristic. For example, a unique number is given to each player in a football team, but a player having number 8 does not play better twice than the player having number 4.
Although nominal scales are used for the lowest form of measurement, yet nominal scales are frequently used in marketing research. Nominal level identifications are needed in marketing research to identify brands, store types, sales territories, customers, gender, geographic location, race, religion, buyer/non-buyer, heavy and light users, etc. The numbers assigned to such categories are mutually exclusive. Alphabets, even symbols, could be used instead of numbers in nominal scales. Nominal scales simply label objects and do not provide information on greater than or less than. Usually counting is the permissible operation in a nominal scale. Thus statistics like frequency distribution, percentages, mode, chi-square, etc., are used while analyzing data gathered by a nominal scale. Average in these scales is meaningless.
π Definition β Nominal Scale: A scale where numbers serve as labels to identify or categorize objects without indicating any quantitative relationship.
2. Ordinal Scale
An ordinal scale defines ordered relationships among the objects measured. It indicates relative size difference between objects. An ordinal scale shows whether an object has more or less of the attribute but not as to how much less or more. It shows the relative position of the objects under measurement but not what the magnitude of the difference is. World ranking of cricket teams, finishing order of horse races, positions of students in the class, and social class are examples of ordinal scales. In marketing research, opinions, measurement of preferences, relative attitudes, and evaluation of quality of different brands of the same product etc. are through ordinal scales. In an ordinal scale, the difference of numbers indicates difference in rank and nothing else. The statistics commonly used in analyzing data gathered by an ordinal scale are percentile, median, and rank order correlation, etc.
π Definition β Ordinal Scale: A scale that defines an ordered relationship among the objects measured, indicating relative size difference but not the magnitude of the difference.
3. Interval scale
One problem with the ordinal scale is that it defines the order of the objects but it does not tell about what is the difference (or distance) between the objects. An interval scale shows that as the interval between the numbers on the scale represent equal increments of the attribute being measured, the differences can be compared. A difference between 25 and 26 is the same as between 26 and 27 which is the same as between 27 and 28. The most common example of an interval scale in life is that of a thermometer. But as you know, two types of thermometers, Celsius and Fahrenheit, do not have a fixed or true zero or freezing point. Both the zeros and units of measurements are different, although the amount of heat in various intervals on each thermometer will be the same. Let us illustrate it with figures. The amount of heat between 88Β° and 89Β° on Fahrenheit is the same as between 91Β°-92Β°, but the amount of heat between 88Β° and 89Β° on Fahrenheit is different from the amount of heat between 88Β° and 89Β° on Celsius. Statistical techniques that are used in nominal and ordinal scales can also be used in interval scales. In addition to that, statistics like mean, standard deviation, product moment correlation, etc., can also be used in interval scales.
π Definition β Interval Scale: A scale where the intervals between numbers represent equal increments of the attribute being measured, allowing for the comparison of differences.
4. Ratio scale
A ratio scale is one in which a true zero exists. True zero or absolute zero means that the number zero is assigned to the absence of the characteristic being measured. Thus we can compute the ratio of scale values. For example, it is possible to say how many times greater or smaller one object is than another. This is the only type of scale that allows making comparison of absolute quantities. We can say that the market share of company A is twice as much as that of company B. In market research, data on the number of customers, costs, sales, market share, and some other marketing variables are measured on ratio scales. All statistical techniques can be applied to analyze ratio scale data.
π Definition β Ratio Scale: A scale with a true absolute zero point, allowing for the computation of ratios and comparison of absolute quantities.
π Summary of the Four Basic Rating Scales:
| Scale Type | Nature | Application in Marketing Research |
|---|---|---|
| Nominal | Identification, labeling of objects | Classification by gender, location, social class; identification of stores, brands, etc. |
| Ordinal | Ordered relations according to more or less of the attribute | Ranking, preferences, merit list, positions in tournament |
| Interval | Interval between adjacent ranks are equal | Attitude measurement, index numbers, temperature |
| Ratio | Absolute zero exists; comparisons possible | Sales, income, age, units produced, costs, market share |
The scales used in marketing research can be divided into two types: Comparative and Non-comparative scales. In the first type, direct comparison of objects can be made with one another. Data in comparative scales have ordinal or rank order properties. Each object in non-comparative scaling is scaled independently of others in the set. The data in such scales is usually interval or ratio scaled. Likert scale, semantic differential or staple scales are the classification of itemized non-comparative scales.
Comparative scales
First type of comparative scaling is paired comparison scale.
Paired Comparison Scale
A paired comparative scale is a technique in which a respondent is presented with two objects at a time in the pair and asked to select one according to some criterion. It is like an ordinal scale in which two objects are ranked.
Example: Please indicate which one you like for your use among ten pairs of shampoo brands (A, B, C, D, E). The results can be compiled in a matrix where a '1' means the column brand is preferred over the row brand, and a '0' means the row brand is preferred over the column brand.
| A | B | C | D | E | |
|---|---|---|---|---|---|
| A | - | 1 | 1 | 0 | 0 |
| B | 0 | - | 0 | 0 | 0 |
| C | 0 | 1 | - | 0 | 0 |
| D | 1 | 1 | 1 | - | 1 |
| E | 1 | 1 | 1 | 0 | - |
| Sum | 2 | 4 | 3 | 0 | 1 |
In the above table, we see that figure 1 in the box means that the column brand is preferred to the corresponding brand in the row, and a zero in the box means that the brand in the row is preferred to the corresponding column. Data in this matrix can be analyzed by finding out percentages of preferences. In this case, brand A is preferred e.g., A is preferred by 2/10*100=20%, B=40%, C=30%, D=0% and E=10%. Thus you can find out the rank order of the five brands.
Paired comparison should be used if the number of brands is limited. In case the number of brands is large, then the exercise becomes unmanageable. Due to this difficulty, respondents cannot usually meaningfully rank more than five or six brands. Another problem with this technique is that the comparison of two objects at a time is seldom the way choices are really made in the marketplace. Thus a brand can be a first choice in a paired comparison situation but performs poorly in the actual marketplace. Paired comparison, however, is the most common method of testing taste.
π Example: For five brands (A, B, C, D, E) with paired comparison preference counts: A=2, B=4, C=3, D=0, E=1. Preference percentages are A=20%, B=40%, C=30%, D=0%, E=10%, leading to a rank order of B (1st), C (2nd), A (3rd), E (4th), D (5th).
Rank Order Scale
This scale is also comparative in nature. It involves asking the respondents to rank various brands/objects with regard to some criterion. For example, a respondent may be asked to rank five print ads on the basis of awareness it provides, liking of the respondent, or intention to buy.
Example: Please rank the following brands of fruit juices in order of preference from 1 to 7. Rate your most preferred brand as 1 and least preferred as 7; no two brands will receive the same rating.
- Brand A: 2
- Brand B: 6
- Brand C: 4
- Brand D: 1
- Brand E: 7
- Brand F: 5
- Brand G: 3
This technique is frequently used in marketing research. Advantages of rank-order scaling include that it is a simple concept, easy to administer, and less time-consuming to administer than other comparative scales such as paired comparison. The instructions for ranking objects are easy to comprehend. It is also said that the ranking made by the respondent is closer to his/her real purchase situation. The major disadvantage of a rank-order scale is that it produces only ordinal data. It does not mean that the first preference in the set is the most liked. It may be βleast dislikedβ in the set.
π‘ Why this matters: Rank-order scales are practical for getting a quick relative preference order but are limited to ordinal data, meaning one cannot determine the intensity of preference between ranks.
Constant-Sum Scale
In a constant-sum scale, respondents are required to allocate a fixed number of rating points (usually 100) among several objects. It is widely used to measure the relative importance of various attributes of the object.
Example: Please divide 100 points among the following characteristics of a toothpaste that reflects the relative importance of each characteristic to you.
- Taste: 8
- Fragrance: 7
- Tube: 5
- Cleanliness of teeth: 35
- Prevention of tooth decay: 25
- Price: 2
- Quality: 8
- Shining of the teeth: 10 Total: 100
The relative importance of the attributes is determined by counting the points assigned by all respondents and dividing by the number of respondents.
The main merit of the constant-sum scale is that it permits fine distinction of attributes of an object without much time. However, it may be difficult to allocate points to several categories. The main disadvantage of this scale is that respondents may allocate points that exceed or are short of the required total, say 103 or 97 instead of 100.
π Definition β Constant-Sum Scale: A comparative scale where respondents allocate a fixed number of points (typically 100) among several objects to indicate their relative importance or preference.
β Key Takeaways
The four basic rating scalesβnominal, ordinal, interval, and ratioβform a hierarchy of measurement, each with specific properties, applications, and permissible statistics. Nominal scales merely label objects, ordinal scales rank them without measuring distance, interval scales have equal intervals but no true zero, and ratio scales possess a true zero allowing for absolute comparisons. For comparative scaling, the paired comparison scale is best for a limited number of items, the rank order scale is simple and efficient for obtaining ordinal preferences, and the constant-sum scale provides fine-grained measurement of attribute importance. It is critical to choose the appropriate scale for the research objective and to understand its limitations, such as the inability to measure magnitude in ordinal scales or the potential for respondent error in constant-sum scales.
π§ Quick Revision Questions
- What is the key difference between an ordinal scale and an interval scale in terms of what they measure?
- In a paired comparison scale with 6 brands, how many total pairs would a respondent need to evaluate?
- What is the primary advantage of a ratio scale that is not available in any of the other three basic scales?
- What is a major potential problem when respondents use a constant-sum scale?
- Why might a brand that is a top choice in a paired comparison test perform poorly in an actual marketplace?
π Lecture 11 β Non-Comparative Scales
π Overview: This lecture introduces non-comparative (monadic) rating scales, where objects are evaluated independently. It covers two main types: continuous rating scales and itemized rating scales (Likert, Semantic Differential, and Staple scales). The lecture also explains critical decisions researchers must make when developing rating scales, such as the number of categories, odd/even options, and forced vs. unforced choices.
ποΈ Topics Covered
This lecture covers non-comparative scales, beginning with continuous rating scales (also called graphic scales) and itemized rating scales, specifically Likert, Semantic Differential, and Staple scales. It then explains the development of rating scales, including decisions on number of categories, odd or even numbers, balanced vs. unbalanced scales, forced vs. unforced scales, and degree of verbal description.
π Lecture Summary
Non-Comparative Scale
In non-comparative scales, objects are scaled independently of other objects in the set. For this reason, non-comparative scales are also called monadic scales. Non-comparative scales are of two kinds: continuous rating scales and itemized rating scales.
Continuous Rating Scales
A continuous rating scale requires the respondents to indicate their position by rating the object on a continuum that runs from one extreme of criterion to the other. The format of this scale varies. Such a scale is also called a graphic scale. See the following example:
How would you rate Airline A.? Version A: bad okay Good Very good Excellent Version B: Worst 0 10 20 30 40 50 60 70 80 90 100 Best
Graphic rating scales are easy to construct but answers may be unreliable and analysis complicated. However, use of computers has made the analysis easy and that has increased the use of continuous rating scales in marketing research.
Itemized Rating Scales
An itemized rating scale contains numbers or brief descriptions or both in respect of the categories of response. Respondents select the categories that best describe their rating about the object in question. Itemized rating scales are Likert, Semantic Differential, and Staple scales.
Likert Scale
This scale is named after Rensis Likert who developed this scale. The Likert scale requires a respondent to show a degree of agreement or disagreement with a variety of statements about the related object. Respondents indicate how much they agree or disagree with the statement. This scale captures the integrity of the respondents' feelings.
Example: Instructions: Listed below are some statements about Jeans Y. Please indicate your degree of agreement or disagreement by encircling the appropriate answer.
| Statement | Strongly Agree | Somewhat Agree | Neutral | Somewhat Disagree | Strongly Disagree |
|---|---|---|---|---|---|
| a. Jeans Y are easy to identify on someone | 1 | 2 | 3 | 4 | 5 |
| b. Jeans Y make me feel good | 1 | 2 | 3 | 4 | 5 |
| c. Jeans Y are good looking | 1 | 2 | 3 | 4 | 5 |
| d. Jeans Y are reasonably priced | 1 | 2 | 3 | 4 | 5 |
| e. Jeans Y will be my next pair of Jeans | 1 | 2 | 3 | 4 | 5 |
The score of each item by all the respondents in the Likert Scale are summed up and the average is drawn. Due to this reason, the Likert scale is also known as a summated scale.
π Definition β Likert Scale: A scale that requires a respondent to show a degree of agreement or disagreement with a variety of statements about the related object. It is also called a summated scale. π Example: A respondent selects "1" (Strongly Agree) for "Jeans Y are easy to identify on someone." π‘ Why this matters: Because Likert scales capture the intensity of feelings, they are widely used in attitude research.
Semantic Differential Scale
This scale has been borrowed from another area of research called "semantics". This scale contains a series of bipolar adjectives from the various characteristics of the object under study. Respondents indicate the impression of each characteristic by indicating the appropriate place on the continuum. It is usually a seven-point rating scale having two poles (adjectives), e.g., "friendly-unfriendly," "high quality-low quality," "convenient-inconvenient," or "dependable-undependable." Having seven separators between two poles is mandatory. While using the semantic differential scale, the respondent marks the blank that is closer to his rating.
Example: Please indicate your impression about Restaurant A by marking the line that best describes your opinion. Quick service ------- slow service High prices-------- low prices Good quality food ------- low quality food Limited variety of food ------- wide variety of food Poor location ------- good location
The Semantic Differential Scale is gaining popularity because of its versatility. This scale is used extensively in comparing different brands, company images, or stores. Mean is the statistic which is used in Semantic Differential Scales.
π Definition β Semantic Differential Scale: A scale containing a series of bipolar adjectives from the various characteristics of the object under study, usually a seven-point rating scale. π Example: "Quick service ------- slow service" for Restaurant A.
Staple Scales
Jan Staple developed this scale, and therefore this scale is called Staple scales. Typically, a Staple scale has ten categories of measurement ranging from -5 to +5 and usually shown vertically. Categories may be reduced if the researcher chooses to. Respondents select a category and mark the selected number to indicate their rating. The higher the positive score, the better the adjective describes the object. The Staple scale uses one pole rather than opposite poles. It is easier to construct and administer. It is equally suited to telephone interviewing.
Example: Please rate Bank A with regard to their "fast service" and "friendly environment" on the following scale.
Friendly environment Fast service +5 +5 +4 +4 +3 +3 +2 +2 +1 +1 -1 -1 -2 -2 -3 -3 -4 -4 -5 -5
π Definition β Staple Scale: A scale developed by Jan Staple, typically having ten categories ranging from -5 to +5, shown vertically, using one pole rather than opposite poles. π Example: Rating Bank A on "fast service" with +4 or "friendly environment" with +2.
Development of Rating Scales
Rating scales take different forms as you have seen above. While using a rating scale in research, the researcher has to choose among several alternatives. The researcher must make decisions regarding the scales on these aspects: a. How many categories in the scale? b. Should the number of categories be odd or even? c. Should the scale be balanced or unbalanced? d. Should the choices be forced or unforced? e. Degree of verbal description. f. The final form (physical) of the scale.
1. How Many Categories in the Scale
There is no ideal number of categories in a scale. They could be as many as 13 or as few as 3. Many categories provide for finer discrimination by the respondents, but sometimes it is very difficult for the respondents to handle many categories. They really cannot discriminate. If the respondents are quite knowledgeable, a large number of categories can be used; otherwise, a small number may be employed. In telephonic interviews, a smaller number is appropriate. If you have limited space on the paper, use a small number of categories. For marketing research, five to seven categories are commonly used in rating scales.
2. Odd or Even
If you should provide a neutral point in the rating scale, then use an odd number. But if you think there will be no neutral respondent or you want to force the respondents to give some answer, then use an even number.
3. Balanced or Unbalanced Scale
A rating scale is balanced when it has an equal number of favorable and unfavorable responses. See the following scales:
Unbalanced Scale: Balanced Scale: The taste of drink A is. The taste of drink A is. Extremely delicious Extremely delicious Very delicious Very delicious Delicious Delicious Poor Delicious to some extent Very poor Poor Extremely poor Very poor
Usually, the rating scales are balanced. Balanced scales provide more objective data.
4. Forced Scales versus Non-Forced Scales
Forced scales force the respondents to express an opinion as no provision is made for "no opinion." But if the researcher thinks that many respondents will have no opinion on the issue, the scale contains a category of "no opinion." Such a scale is called a non-forced scale.
5. Degree of Verbal Description in a Scale
The description of scale categories may be numerical, verbal, and even pictorial. The researcher has a choice to label all or sometimes only categories on the extremes are verbally labeled. However, some people think that confusion in the scale can be reduced by labeling all scale categories.
β Key Takeaways
The most critical things to remember from this lecture are the three types of itemized rating scales: Likert (summated scale measuring agreement/disagreement), Semantic Differential (bipolar adjectives on a seven-point scale, using mean as the statistic), and Staple (single-pole scale from -5 to +5, suited for telephone interviews). For scale development, researchers must decide on the number of categories (typically 5-7 for marketing research), whether to use odd (provides neutral point) or even numbers (forces a choice), balanced (equal favorable/unfavorable) or unbalanced scales, and forced (no "no opinion" option) or non-forced scales. Continuous rating scales, also called graphic scales, use a continuum from one extreme to another but may be unreliable and complicated to analyze.
π§ Quick Revision Questions
- What is the fundamental difference between comparative and non-comparative (monadic) scales?
- Name the three types of itemized rating scales discussed in this lecture.
- What is the typical number of categories in a Semantic Differential scale, and what statistic is used for analysis?
- If a researcher wants to avoid neutral responses from participants, should they use an odd or even number of categories in the scale?
- What distinguishes a forced scale from a non-forced scale?
π Lecture 12 β Physical Form of Scale
π Overview: This lecture explores the physical configurations and verbal descriptors used in rating scales, then delves into the critical concepts of measurement accuracy. It explains the components of measurement error and introduces the foundational research principles of validity and reliability, essential for ensuring that marketing research instruments produce trustworthy and actionable data.
ποΈ Topics Covered
The lecture begins by detailing the various physical forms scales can take, including horizontal/vertical layouts and face scales for children, along with common verbal descriptors for different constructs like frequency and satisfaction. It then introduces the concept of measurement error (systematic vs. random) and the equation linking observed, true, and error scores. Finally, the core concepts of validity (face/content, predictive, convergent, discriminant) and reliability (test-retest, alternative forms, split-half) are defined, explained with examples, and their interrelationship is discussed.
π Lecture Summary
Physical Form of Scale
A scale can have different physical forms. A researcher can exercise different options including horizontal, vertical, boxes, lines, or a number assigned on a continuum. Positive or negative values may be used. For children, different shapes of faces (happy or otherwise) can indicate the choice of respondents.
An example of a physical configuration is: Very powerful - Powerful - Somewhat powerful - Neither weak nor powerful - Somewhat weak - Weak - Very weak
Similarly, verbal descriptors in the scale may change with the nature of the construct being measured. Common descriptors include:
- Frequency of purchase: never, casually, sometimes, often, most often
- Attitude: Very negative, Negative, Neither negative nor positive, Positive, Very positive
- Consumer satisfaction: Highly dissatisfied, Not satisfied, Neutral, Satisfied, Quite satisfied
- Intention to purchase: Will not buy at all, Inclined not to buy, Might or might not buy, Probably will buy, Will buy definitely
Accuracy of Measurement
When we measure through scales or otherwise, a measured value is not the true value of the characteristic; rather, it is the value we observed. This difference is called measurement error. Measurement error may be caused by many potential sources:
- Personal factors (fatigue, mood, health)
- Situational factors (noise, pressure, distracters)
- Respondent factors (intelligence, education)
- Variation in interview method (telephonic, face-to-face, mailed)
- Instrument factors (ambiguity, lack of clarity)
- Data analysis factors (coding and tabulation errors)
There are two components of total measurement error: systematic error and random error.
- Systematic error causes a constant bias in measurement. For example, a stopwatch that systematically runs fast will produce the same type and amount of error each time.
- Random error is not constant. For example, using many different stopwatches to record time will produce errors that fall within a range around the true time.
Thus, observed score is comprised of true score plus systematic error plus random error: π Definition β Observed Score: Oβ = Tβ + Sβ + Rβ Where:
- Oβ = observed measurement
- Tβ = true measurement
- Sβ = systematic error
- Rβ = random error
Validity
A measure has validity if it measures what it is supposed to measure. The differences in observed scores reflect true differences among objects or individuals on the measured characteristic. Validity is the accuracy of measurementβthe extent to which measurement is free from both systematic and random error.
Face or Content Validity
Face or content validity is concerned with the degree to which the measurement βlooks likeβ it measures what it is supposed to measure. It is a judgment by the researcher. For example, if a respondent recognizes an advertisement, it can usually be accepted at face value as if they had been exposed to the ad in the past. This is the weakest method of assessing validity but can be improved by having other researchers critique the instrument.
Predictive Validity
If a measure can predict some future event, the instrument has predictive validity. For example, a measure of brand preference or buying intention is valid if it can be shown that those with strong preference or intention actually bought the brands. Predictive validity is very important for decision making.
Convergent Validity
If a researcher uses two different methods or sources of data collection for the same information and both agree, the measure has convergent validity. In a survey, 214 questionnaires were left with household heads in the morning and collected in the evening, yielding an average age of 17.2 years. Then, 40 questionnaires were picked at random, and another household member was asked ages via telephone; the average was 17.1 years. The close agreement shows convergent validity.
Discriminant Validity
In discriminant validity, questions that measure different objects should yield different results. If a researcher knows there are real differences, they should find that responses actually differ. If two questions measure concern about theft security and fire security, but there are no differences between the two constructs, there is doubt about the instrument's validity.
Reliability
Reliability is the consistency of the instrument. If the measurement of the same group is made repeatedly by the same scale, the results should be consistent. Systematic error does not affect reliability; random error does. Therefore, reliability is the extent to which an instrument is free from random error. If Rβ = 0, reliability is highest. The association between measurements obtained when the scale is administered at different times determines reliability. There are three assessment methods:
1. Test-Retest Reliability
Test-retest involves repeated measurement of the same respondents using the same scale under similar conditions. Results from both times are compared. If discrepancy is great, random error is great and reliability is low. If scores are similar, random error is small and reliability is high. Similarity is determined via correlationβa high correlation coefficient indicates high reliability.
β οΈ Problems with Test-Retest:
- It can be illogical or impossible to measure the same subjects twice (e.g., mall intercept).
- The time interval between test and retest influences reliability (longer interval = lower reliability).
- First measurement may change the second response (respondents may remember answers).
- Respondent attitude may change (e.g., thinking about high-fat milk could change health attitudes).
- Situational factors may change between measurements.
π‘ Why this matters: Test-retest reliability is best used in combination with other approaches.
2. Alternative Forms Reliability
Two equivalent alternative forms of the scale/instrument are constructed and administered to the same group at two different times. Scales are equivalent but not identical. Time interval is 2-4 weeks. Degree of similarity is determined by correlation. β οΈ Problems: Time-consuming and expensive (must develop an alternative form), and difficult to make both instruments equivalent in content.
3. Split Half Reliability
Split-half reliability is the simplest measure of internal consistency. It involves preparing a multi-item scale, dividing it into two halves, and correlating the item responses of these two halves. A high correlation shows high internal consistency and high reliability. This is a version of the alternative-forms technique used to indicate items measure the same characteristic. β οΈ Problem: Results depend on how items are split (e.g., odd/even vs. random division).
β Key Takeaways
The accuracy of any measurement is determined by its freedom from both systematic and random error, where observed scores are the sum of true scores and both error types. Validity, which ensures a measure captures the intended construct, is assessed through face, predictive, convergent, and discriminant validity. Reliability, the consistency of a measure and its freedom from random error, is evaluated using test-retest, alternative forms, and split-half methods, each with specific trade-offs regarding time, cost, and practical application.
π§ Quick Revision Questions
- What is the difference between systematic error and random error, and how does each affect validity and reliability?
- What is the formula for observed measurement, and what does each component represent?
- Explain the difference between convergent validity and discriminant validity, providing an example for each.
- What are three methods for assessing reliability, and what is a key limitation of each?
- Why is test-retest reliability considered problematic for measuring attitudes that might change over time?