📘 Lecture 45 — Overview of the Course
📖 Overview: This is the final lecture of the series, providing a comprehensive review of all topics covered in the course on Psychological Testing and Measurement. It summarizes the fundamental concepts of psychological testing, including test types, reliability, validity, norms, and item analysis, emphasizing the importance of ethical standards in test use. This lecture serves as a crucial revision for students to consolidate their understanding of the entire course material.
🗂️ Topics Covered
This lecture reviews the entire course, beginning with the definition and purpose of psychological tests as tools for quantifying behavior. It covers the categorization of tests by purpose, administration, and structure, and their major contexts of use in education, occupation, and clinical settings. The lecture then summarizes the essential characteristics of good tests—validity, reliability, and norms—and details the specific methods for assessing reliability (test-retest, alternate-form, split-half, coefficient alpha) and validity (content, criterion, and construct). It concludes with a recap of norms, item analysis (difficulty and discrimination), and the future of psychological testing, emphasizing the need for ethical standards.
📝 Lecture Summary
What is a Psychological Test?
Psychology is the scientific study of behavior and mental processes. Psychologists use carefully designed tools for data collection, and psychological tests are one of those tools. In all professional settings where psychologists work, some form of testing and assessment is used.
🔑 Definition — Test: “A measurement device or technique used to quantify behavior or aid in the understanding and prediction of behavior” (Kaplan & Saccuzzo, 2001).
Tests are a part of the assessment package and procedure.
Types of Tests
Tests can be categorized on the basis of:
- The purpose or the type of behavior/characteristics to be measured: personality, aptitude, intelligence, achievement etc.
- The administration procedure: individual versus group tests
- Speed versus ability tests
- Aptitude tests, achievement tests, or intelligence tests
- Ability versus personality tests
- Structured/objective tests versus projective tests
- Original versus translated and adapted tests
- Translated tests
Major Contexts of Current Test Use
Psychological tests are designed for use in numerous life situations. The three major contexts of test use are:
- Educational testing
- Occupational testing
- Clinical and Counseling Psychology (Anastasi & Urbina, 1997).
Test Construction
The development of a good test takes place after going through a number of stages, keeping in view the established principles of test construction.
Essential Characteristics of Psychological Tests
A good psychological test should have these qualities:
- Validity: A test should measure what it is intended to measure.
- Reliability: A test should give consistent results. It should give the same or similar results every time it is administered to the same subjects in the same conditions.
- Norm development and standardization
Reliability
According to the definition given by Anastasi & Urbina (2007), reliability refers to “the consistency of the scores obtained by the same persons when they are reexamined with the same test on different occasions, or with different sets of equivalent items, or under other variable examining conditions.”
Reliability of a measure can be measured in a number of ways, such as the stability of scales over time or the consistency between items.
a) Test-Retest Reliability:
- Test-retest reliability deals with two performances of the same test by the same persons on two different occasions.
- If reliability refers to the consistency and stability of scores over time, then it will be measured using this method.
- The test-retest coefficient is also known as the ‘coefficient of stability’.
- The test takers’ scores on the first administration of the test are correlated with their scores obtained on the second administration of the same test.
b) Alternate-Form Reliability: In this approach, the test developer develops two alternate or parallel forms of the same test. Ideally, the two forms should be independently developed and completely parallel or equivalent forms of the same measure. They should match each other in all respects, including:
- Same specifications
- Same instructions
- Same time limit
- Same content
- Same number of items
- Same item format
- Same difficulty level
c) Split-Half Reliability: Alternate form reliability is popular but has some obvious limitations. Construction of two completely parallel forms is not easy and requires a lot of time and effort; even when two alternate forms are available, the practice effect and prior experience may affect performance on the second occasion. The time gap between two administrations is another intervening variable. To overcome these problems, split-half reliability is computed.
e) Coefficient Alpha:
- Another approach, quite similar to the Kuder-Richardson technique, is to calculate ‘coefficient alpha.’ The Kuder-Richardson formula can be used only for tests in which items are scored as either zero or one.
- It is not applicable to tests where answers to items are assigned two or more scoring weights, e.g., personality inventories where a number of response options with attached weights are available (never=0, occasionally=1, often=2, and so on).
- Coefficient alpha is a general formula that caters for such tests.
Validity
“Traditionally, the validity of a test has been defined as the extent to which a test measures what it was designed to measure” (Aiken, 1994, p.95). “The extent to which a test measures the quality it purports to measure. Types of validity evidence include content validity, criterion validity, and construct validity evidence” (Kaplan & Saccuzzo, 2001, p.640).
Validity is measured in several ways, divided into three categories:
Content Validity Evidence: “The evidence that the content of a test represents the conceptual domain it is designed to cover” (Kaplan & Saccuzzo, 2001, p.635).
Construct Validity Evidence: “A process used to establish the meaning of a test through a series of studies. To evaluate evidence for construct validity, a researcher simultaneously defines some construct and develops the instrumentation to measure it. In the studies, observed correlations between the test and other measures provide evidence for the meaning of the test” (Kaplan & Saccuzzo, 2001, p.635).
Criterion Validity Evidence: “The evidence that a test score corresponds to an accurate measure of interest. The measure of interest is called the criterion” (Kaplan & Saccuzzo, 2001, p.635).
Specific Procedures for Content Validity: A number of careful decisions are taken at the time of test development when test specifications are made. Test specifications include:
- Instructional objectives, relative importance of topics/processes.
- It should clearly indicate the number of items for each topic.
- It may also provide sample material. The process of content validation should include a description of all procedures in the manual that ensure the test is appropriate and representative.
Criterion-Related Validity:
- Whenever we plan and design a test we have a certain standard in mind that we want to meet.
- We relate the score on our test with those achieved on the criterion or standard.
- One approach to assessment of validity is through comparing the test results with those on a criterion, a procedure where scores on a test being used are correlated with scores on a criterion.
Criterion Prediction Procedures: Two procedures may be adopted for this purpose:
- Predictive validity evidence
- Concurrent validity evidence
🔑 Definition — Predictive Validity Evidence: “The evidence that a test forecasts score on the criterion at some future time” (Kaplan & Saccuzzo, 2001, p.638).
🔑 Definition — Concurrent Validity Evidence: “Evidence for criterion validity in which the test and the criterion are administered at the same point in time” (Kaplan & Saccuzzo, 2001, p.635).
Anastasi & Urbina (2007) have provided an all-encompassing description of construct validity: “The construct validity of a test is the extent to which the test may be said to measure a theoretical construct or trait.”
Assumptions Underlying Construct Validity:
- Test scores will be highly correlated with tests measuring the same or a similar construct. This is convergent validity.
- Test scores will have weak/low (or at times may be negative) correlation with tests meant to measure constructs which are different from the one measured by the main test. This is discriminant validity.
Norms
The scores on any test become meaningful in the presence of norms that have been developed for that test.
🔑 Definition — Norms: “The test performance data of a particular group of test takers that are designed for use as a reference for evaluating or interpreting individual test scores.”
Norms provide standards to which the results of the test takers on different measurements can be compared. Norms are the test performance of the standardization sample, a group of people whose performance on a specific test is taken as a standard or norm for comparison. All other individuals’ performance on this specific test is compared with the scores of this standardization sample.
Item Analysis
Item analysis is “a set of methods used to evaluate test items. The most common techniques involve assessment of item difficulty and item discriminability” (Kaplan & Saccuzzo, 2001, p.637). It may involve other things as well, such as item response valence or content analysis.
🔑 Definition — Item Difficulty: “A form of item analysis used to assess how difficult items are. The most common index of difficulty is the percentage of test takers who respond with the correct choice.”
Item Discrimination: A test is supposed to discriminate between those who know and those who do not know; those who score high and those who score low. “The item-discrimination index is a measure of the difference between the proportion of high scorers answering an item correctly and the proportion of low scorers answering the item correctly; the higher the value of d, the greater the number of high scorers answering the item correctly” (Cohen & Swerdlik, 1999).
Types of Tests
A variety of psychological tests were discussed in this course:
- Intelligence tests: individual and group tests, verbal and performance tests, speed and ability tests
- Personality tests: Personality inventories and projective tests
- Achievement tests: Ordinary school tests and standardized, internationally used tests
- Tests used for occupational settings
- Tests for special populations
- Tests used in clinical and counseling settings
- Aptitude tests
- Tests of interests
- Variations of psychological tests: the Piagetian tasks.
Future of Psychological Testing
- The application of psychological tests is becoming broader every day, as the areas of application of psychology are also increasing.
- BUT one should always keep the ethical standards in mind; these standards may have been set by the APA, the government, or any other professional and/or regulatory agency.
⭐ Key Takeaways
For the exam, a student must remember that a good psychological test must possess validity (measuring what it intends to), reliability (yielding consistent results), and established norms for meaningful score interpretation. The three main types of validity are content, criterion (predictive and concurrent), and construct (which involves convergent and discriminant evidence). Reliability can be assessed through test-retest, alternate-form, split-half, and coefficient alpha methods, each addressing different sources of measurement error. Finally, item analysis using indices of difficulty and discrimination is crucial for evaluating individual test items, and ethical standards must guide all test development and use.
🧠 Quick Revision Questions
- What are the three essential characteristics that a good psychological test must possess?
- Distinguish between test-retest reliability and alternate-form reliability.
- Define construct validity and explain the two assumptions (convergent and discriminant validity) that underlie it.
- What is the purpose of item analysis, and what are the two most common indices used in this analysis?
- List the three major contexts of test use as discussed in the course.