π Lecture 20 β Creating Fixed-Choice Test Items: MCQs-IV
π Overview: This lecture continues the series on constructing effective multiple-choice questions (MCQs), providing five additional practical suggestions for improving item quality. These guidelines focus on ensuring only one correct answer, making distractors plausible, avoiding verbal clues, and equalizing answer lengths, all of which are critical for creating fair and valid assessments.
ποΈ Topics Covered
The lecture covers four main topics: ensuring only one correct or clearly best answer per item (Topic 067), making all distractors plausible to confuse the uninformed (Topic 068), avoiding verbal association between the stem and the correct answer (Topic 069), and ensuring the relative length of alternatives does not provide a clue to the answer (Topic 070). Each topic is illustrated with a poor example and a better example.
π Lecture Summary
Topic- 067: Suggestions for Constructing MCQs βV β An item should contain only one correct or clearly best answer.
An item should contain only one correct or clearly best answer. Including more than one correct answer and asking students to select all correct alternatives has two shortcomings: (a) such items are usually no more than a collection of true-false items presented in MCQ form, and (b) the number of alternatives selected as correct answers varies from one student to another, making scoring inconsistent.
π Definition β Single Correct Answer: A test item should have exactly one correct or clearly best answer among the alternatives. π Example (Poor): "Pakistan borders on: a) India b) Tajikistan c) Saudi Arabia d) China" β this has two correct answers (India and China). π Example (Better): The same options are presented as separate True/False statements: "Pakistan borders on: a) India T/F b) Tajikistan T/F c) Saudi Arabia T/F d) China T/F"
Topic- 068: Suggestions for Constructing MCQs βVI β All distractors should be plausible.
All distractors should be plausible. The purpose of a distractor is to confuse the uninformed student. To the student who has not achieved the learning outcome being tested, the distractor should be as attractive as the correct answer. If properly constructed, each distractor will be selected by some students. If a distractor is not selected by anyone, it is not contributing to the functioning of the item and should be eliminated or revised.
π Definition β Plausible Distractor: An incorrect alternative that appears attractive to a student who has not mastered the learning outcome. π Example (Poor): "Who wrote national anthem of Pakistan? a) Allama Iqbal b) Christopher Columbus c) Hafeez Jullandhuri d) Ibrar ul Haq" β "Christopher Columbus" is not plausible. π Example (Better): "Who wrote national anthem of Pakistan? a) Allama Iqbal b) Habib Jalib c) Hafeez Jullandhuri d) Munir Niazi" β all are Pakistani poets, making each distractor plausible. π‘ Why this matters: A non-functioning distractor (selected by no one) wastes a potential slot for a more effective distractor and reduces item discrimination.
Topic- 069: Suggestions for Constructing MCQs βVII β Verbal association between the stem and the correct answer should be avoided.
Verbal association between the stem and the correct answer should be avoided. Frequently, a word in the correct answer provides an alternative clue because it looks or sounds like a word in the stem. However, words similar to those in the stem might be included in the distractors to increase their plausibility. Students who depend on rote memory and verbal association will then be led away from, rather than to, the correct answer.
π Definition β Verbal Association: A clue where the correct answer contains a word that looks or sounds like a word in the stem, allowing students to guess correctly without knowing the content. π Example (Poor): "Which of the following agencies should you contact to find about a flood? a) National flood relief b) Local radio station c) Pakistan post d) Pakistan weather bureau" β "flood" in stem matches "flood" in option (a). π Example (Better): "Which of the following agencies should you contact to find about a flood? a) Disaster management office b) Radio station c) Post office d) Weather bureau" β no word matches the stem. π‘ Why this matters: Avoiding verbal association forces students to rely on actual knowledge rather than surface-level word matching.
Topic- 070: Suggestions for Constructing MCQs βVIII β The relative length of the alternatives should not provide a clue to the answer.
The relative length of the alternatives should not provide a clue to the answer. The best we can hope for is to make alternatives approximately equal in length. Because the correct answer usually needs to be qualified, it tends to be longer than the distractors unless a special effort is made.
π Definition β Equal Length Alternatives: The correct answer should not be systematically longer or shorter than the distractors, as test-takers may guess that the longest option is correct. π Example (Poor): "What is the major purpose of United Nations? a) To maintain peace among people of the world b) To establish international law c) To provide military control d) To form new governments" β option (a) is noticeably longer. π Example (Better): "What is the major purpose of United Nations? a) To maintain peace among people of the world b) To develop new system of international law c) To provide military control of new nations d) To establish democratic forms of governments" β all options are similar in length.
β Key Takeaways
For effective MCQs, always ensure a single correct or clearly best answerβavoid collections of true-false items. Make every distractor plausible and attractive to uninformed students; if a distractor is never selected, revise or delete it. Prevent verbal association clues by not repeating stem words in the correct answer; instead, put similar words in distractors. Finally, equalize the length of all alternatives so the correct answer is not systematically longer or shorter. These five rules (Topics 067-070) collectively improve item validity, reduce guessing cues, and ensure that test scores reflect true learning outcomes.
π§ Quick Revision Questions
- Why should an MCQ contain only one correct answer, and what are two shortcomings of items with multiple correct answers?
- What is a "plausible distractor," and what should you do if a distractor is never selected by any student?
- Give an example of verbal association between a stem and correct answer, and explain how to fix it.
- Why might the correct answer tend to be longer than distractors, and how can this be avoided?
- For Topic 067, the "poor" item "Pakistan borders on: a) India b) Tajikistan c) Saudi Arabia d) China" had two correct answers. What is the recommended solution for turning this into a better item?
π Lecture 21 β Creating Fixed-Choice Test Items: True/ False and Its Uses-I
π Overview: This lecture introduces True/False (alternate-response) test items as a type of fixed-choice assessment. It covers their definition, various uses for measuring simple learning outcomes, advantages and limitations, and specific guidelines for constructing high-quality items that avoid ambiguity and trivial content.
ποΈ Topics Covered
The lecture covers True/False items as declarative statements requiring students to select between two alternatives (true/false, yes/no, fact/opinion). It discusses multiple uses including measuring factual knowledge, distinguishing fact from opinion, recognizing cause-effect relationships, and simple logic. The advantages (efficiency for lower-level outcomes, broad coverage) and limitations (guessing, trivial content, limited to lower levels) are presented, followed by essential construction guidelines focusing on avoiding broad generalizations and trivial statements.
π Lecture Summary
Topic- 071: True/ False Items
Alternate-response test items consist of a declarative statement that the student is asked to mark true or false, right or wrong, correct or incorrect, or the like.
These items are used for measuring relatively simple learning outcomes by using a single declarative statement with various response methods.
π Example (True/False format): Directions: Read each of the following statements. If the statement is true, encircle T and if false, encircle F.
- A river is bigger than a stream. T F
- Founded is the past tense of found. T F
- Dozen is equivalent to 20. T F
π Example (Yes/No format): Directions: Read each statement. If the answer is yes, encircle Y; if no, encircle N.
- Is 51% of 38 more than 19? Y N
- Is 50% of 4/10 equal to 2/5? Y N
- If 60% of a number is 9, is the number smaller than 9? Y N
One of the most useful functions of True/False items is in measuring the studentsβ ability to distinguish fact from opinion.
π Example (Fact/Opinion format): Directions: Read each statement. If it is a fact, encircle F; if it is an opinion, encircle O.
- Current constitution of Pakistan is written in 1973. F O
- Pakistan progressed most under dictator rule. F O
- 18th amendment decentralized the ministry of education. F O
Topic- 072: Uses of True/ False Items
Another aspect of understanding measured by True/False items is the ability to recognize cause and effect relationship. This item type usually contains true propositions in both parts, and the student judges whether the relationship between them is true or false.
π Example (Cause-effect format): Directions: Both parts of each statement are true. Decide whether the second part explains why the first part is true. If it does, encircle Yes; if not, encircle No.
- Leaves are essentials because they shade the tree trunk. Yes No
- Some plants do not need sunlight because they get food from other plants. Yes No
True/False items can also measure simple aspects of logic, requiring students to evaluate both the original statement and its converse.
π Example (Logic format with converse): Directions: For each statement, if true encircle T; if false encircle F. Also, if the converse is true circle CT; if the converse is false circle CF. Give 2 answers for each statement.
- All trees are plants. T F CT CF
- All parasites are animals. T F CT CF
- All eight-legged animals are spiders. T F CT CF
Topic- 073: Advantages and Limitations of True-False
Alternative form questions require students to select any one of two given categories (true-false, yes-no, correct-incorrect, fact-opinion). These items are most suitable for measuring lower level learning outcomes.
Advantages of Alternative Form (True-False):
- Well suited for testing lower-level outcomes, such as:
- Identifying correctness of factual statements (e.g., "Earth is a planet").
- Definitions of terms (e.g., "Photosynthesis is the process by which leaves make food for plants").
- Statement of principles (e.g., "Earth is revolving around the sun").
- Distinguishing facts from opinion (e.g., "Islam is the official religion of Pakistan").
- Recognizing cause-and-effect relationships.
- From a teacher's perspective, useful when:
- A lot of content must be covered in a short time.
- Time available for scoring is very short.
Limitations of Alternative Form (True-False):
- Not all learning outcomes can be measured; generally limited to lower level learning outcomes.
- Ease of guessing correct answers β with only two choices, a student can expect to guess correctly on half of the items for which answers are not known.
- Tendency to take quotations from text with minor wording changes.
- Tendency to include trivial material from the text.
- Items are prone to high guessing and can only be used for measuring lower-level outcomes.
Topic- 074: Suggestions for Constructing True-False Items β I
The most important task in formulating statements is ensuring they are free from ambiguity and irrelevant clues. There is a list of things to avoid when phrasing statements.
1. Avoid broad general statements if they are to be judged true or false.
- Explanation: Most broad generalizations are false unless qualified, and the use of qualifiers provides clues to the answer.
- π Example (Poor item): The president of Pakistan is usually elected to his/her office.
- π Example (Better item): According to the constitution of Pakistan, the president of Pakistan is elected by parliament.
π‘ Why this matters: Broad statements create confusion about what criteria to use and often include clue words like "usually" that signal a false answer, compromising item validity.
Topic- 075: Suggestions for Constructing True-False Items β II
2. Avoid trivial statements.
- Explanation: To obtain statements that are clearly true or false, teachers often turn to specific statements of fact that fit the criterion but have little significance from a learning point of view. Such items test recall of unimportant details rather than meaningful learning.
- π Example (Poor item): Mamnoon Hussain is the 12th president of Pakistan.
- π Example (Poor item): India declared war on Pakistan on September 3rd, 1965.
π‘ Why this matters: Trivial items reward rote memorization of isolated facts and do not assess understanding of important concepts, principles, or relationships.
β Key Takeaways
True/False items are alternate-response questions best suited for measuring lower-level learning outcomes such as factual knowledge, definitions, principles, distinguishing fact from opinion, and recognizing cause-effect relationships. Their main advantages are efficiency in covering broad content and quick scoring, but they suffer from high guessing probability (50% chance correct by chance) and are limited to lower cognitive levels. Teachers must avoid broad generalizations that introduce qualifier clues and must avoid trivial statements that test unimportant details. Proper construction requires clear, unambiguous statements that focus on significant learning outcomes, and the item format can be varied (true/false, yes/no, fact/opinion, cause-effect, or logic with converse).
π§ Quick Revision Questions
- What are the five specific learning outcomes that True/False items are well-suited to measure?
- Why is guessing a major limitation of True/False items, and what is the probability of guessing correctly?
- What is the problem with using broad general statements in True/False items, and what alternative is recommended?
- Why should teachers avoid trivial statements when constructing True/False items?
- List three different response formats (category pairs) that can be used with alternate-response test items along with examples of each.
π Lecture 22 β Creating Fixed-Choice Test Items: True/False and Its Uses-II
π Overview: This lecture provides detailed suggestions for constructing effective true-false test items. It covers seven key principles, from avoiding negative phrasing and complex sentences to ensuring statements are factual, balanced in length, and focused on single ideas. These guidelines help educators create clear, fair, and valid assessments that accurately measure student learning.
ποΈ Topics Covered
The lecture covers seven specific topics (076β080) offering practical suggestions for constructing true-false items. Topics include avoiding negative and double negative statements, avoiding long and complex sentences, avoiding two ideas in one statement unless measuring cause-effect relationships, avoiding opinion statements not attributed to a source, and avoiding unequal length between true and false statements. Each topic includes explanations, poor examples, and better examples for clarity.
π Lecture Summary
Topic- 076: Suggestions for Constructing True-False Items βIII β Avoid the use of negative especially double negative statements.
Students often overlook negative words like "no" or "not," and double negatives create ambiguity. If a negative statement is absolutely necessary, the negative word should be underlined or written in italic to ensure students do not overlook it.
π Example (Poor item): "None of the steps in the experiment was unnecessary." (This is confusing because of the double negative.)
π Example (Better item): "All the steps in the experiment were necessary." (Clear and straightforward.)
Topic- 077: Suggestions for Constructing True-False Items βIV β Avoid long and complex sentences.
A test item should measure whether a student has achieved the intended knowledge or understanding, not their reading comprehension. Long, complex sentences introduce an extraneous factor of reading comprehension. If simplifying the sentence is impossible, consider changing to a different item format.
π Example (Poor item): "Despite the theoretical and experimental difficulties of determining the exact pH value of a solution, it is possible to determine whether a solution is acid by the red color formed on the litmus paper when it is inserted into the solution." (Too long and complex.)
π Example (Better item): "Litmus paper turns red in an acid solution." (Simple and direct.)
Topic- 078: Suggestions for Constructing True-False Items βV β Avoid including two ideas in one statement unless cause and effect relationship are being measured.
When two ideas are in one statement, students can get confused and answer based on only one idea, while the other may have a different truth value. It is better to use two simple statements. An exception is when measuring cause-and-effect relationships.
π Example (Poor item): "Pakistan could not qualify for world cup hockey because of poor resources, low talent and government support." (Should be false because "government support" is not a reason.)
π Example (Better item β simple statements):
- "Pakistan could not qualify for world cup hockey because of poor resources, low talent." (False)
- "Pakistan could not qualify for world cup hockey because of poor resources." (True)
Topic- 079: Suggestions for Constructing True-False Items βVI β Avoid using opinion that is not attributed to some source.
A statement of opinion cannot be marked true or false. It is unfair to expect students to guess how the teacher will score such items or to treat opinion statements as statements of fact.
π Example (Poor item): "All anti-state activists should be hanged." (This is an opinion, not a fact.)
π Example (Better item): "National action plan allows the justice system to give death penalty to anti-state activists if proven guilty." (This can be verified as true or false based on law.)
Topic- 080: Suggestions for Constructing True-False Items βVII β Avoid using true statements and false statements that are unequal in length.
There is a natural tendency for true statements to be longer because they must be precisely phrased to be absolutely true. This can be overcome by making false statements longer by using qualifying phrases similar to those found in true statements.
π Definition β Key Principle: To avoid giving clues to test-takers, ensure true and false statements are approximately equal in length. Lengthen false statements by adding appropriate qualifying phrases.
β Key Takeaways
- Avoid negative and especially double-negative statements; if negatives are necessary, use visual emphasis like underlining or italics.
- Keep sentences short and simple to avoid measuring reading comprehension instead of the intended knowledge.
- Avoid including two separate ideas in one statement unless testing a cause-effect relationship; use separate statements instead.
- Never use opinion statements unless attributed to a verifiable source; true-false items must measure factual knowledge only.
- Ensure true and false statements are approximately equal in length to prevent students from using length as a clue to the correct answer.
π§ Quick Revision Questions
- Why should double negatives be avoided in true-false items, and what should be done if one is necessary?
- What extraneous factor is introduced by using long and complex sentences in true-false items?
- When is it acceptable to include two ideas in one true-false statement?
- Why is it unfair to include opinion statements without attributing them to a source?
- How can test-writers overcome the natural tendency for true statements to be longer than false statements?