ENG518 — Midterm Summary (Lectures 1–22)

📘 Lecture 01 — What is Research?

📖 Overview: This lecture introduces the fundamental concept of research, defining it as a systematic inquiry for knowledge advancement. It explores various definitions from multiple scholars, outlines general characteristics and functions of research, and specifically discusses the nature and applications of research in English Language Teaching (ELT). Understanding these foundations is critical for designing rigorous academic studies.

🗂️ Topics Covered

The lecture begins with an introduction to the course and its aims, followed by defining what research is and its core components. It then presents multiple scholarly definitions of research, outlines the general characteristics of research, explains the functions of research in knowledge and practice, and concludes with the specific characteristics and interdisciplinary nature of ELT research.

📝 Lecture Summary

Topic-001: What is Research?

Research is a systematic inquiry that investigates hypotheses, suggests new interpretations of data or texts, and poses new questions for future research to explore (Jolla, 2015). It consists of asking a question nobody has asked before, doing the necessary work to find the answer, and communicating the acquired knowledge to a larger audience. Research methods vary widely depending on the academic discipline's accepted standards, the researcher's preferences, or a particular study's needs. In science and engineering, research often involves conducting experiments; in arts, humanities, and social sciences, it may include archival work, surveys, in-depth interviews, or creative projects.

🔑 Definition — Research: A systematic inquiry that investigates hypotheses, suggests new interpretations, and poses new questions for future exploration.

Topic-002: Definitions of Research

In the broadest sense, research includes any formal gathering of data, information, and facts for the advancement of knowledge. It is conducted according to the researcher's intention, purpose, and paradigm. In a scientific context, the term "research" usually refers to the entire scientific method from start to finish. Francis G. Cornell describes it as a reliable, verifiable, and exhaustive process. Clifford Woody (U of Michigan) defines research as a careful inquiry or examination in seeking facts or principles, urging for the discovery of truth through critical thinking, defining and redefining problems, formulating hypotheses, collecting, organizing, and evaluating data, making deductions, and reaching conclusions. CC Crawford states that research is a systematic and refined technique of thinking using specialized tools to obtain a more adequate solution to a problem. The Encyclopedia of Social Science defines it as the manipulation of things, concepts, or symbols to extend, correct, or verify knowledge. V. Redman and A.V.H. Mory state research is a systematized effort to gain new knowledge. Francies Rummel describes it as an endeavor to discover, develop, and verify knowledge. According to P.M. Cook, research is an honest, exhaustive, intelligent searching for facts and their meanings, producing authentic, verifiable contributions to knowledge.

🔑 Definition — Systematic Method: A complete, reliable, and verifiable process from question to conclusion.

Topic-003: General Characteristics of Research

The general characteristics of research are that it is systematic, logical, empirical, reductive, and replicable. It gathers new knowledge or data from primary or first-hand sources and emphasizes the discovery of general principles. Research involves exact, systematic, and accurate investigation using certain valid data-gathering devices. It is logical and objective, resisting the temptation to seek only data that support hypotheses and eliminating personal feelings and preferences. Research is a patient and unhurried activity that is carefully recorded and reported, and conclusions and generalizations are arrived at carefully and cautiously.

🔑 Definition — Reductive: The process of breaking down complex phenomena into simpler, manageable components for analysis. 💡 Why this matters: These characteristics ensure that research findings are trustworthy and can be verified by other scholars, which is the foundation of scientific knowledge.

Topic-004: Functions of Research

Research is the name of seeking new knowledge for continuous improvement. Through research, refinement and extension of knowledge happens, and it provides baseline data or simply a picture of how things are. Research makes decision-making easier concerning the acquisition of knowledge in a specific field. It improves learning and teaching processes, and improves various aspects of human life.

🔑 Definition — Baseline Data: Initial information collected to serve as a reference point for future comparisons or to describe the current state of a phenomenon.

Topic-005: Specific Characteristics of ELT Research

There are over 50 research journals available (TESOL website) for guidance regarding ELT research. The large volume of research in ELT provides a sound philosophy, deep insight, and imaginative approaches to problem-solving. ELT research is both theoretical and applied, and it desires to make things better by improving teaching and learning. It is not as exact as physical sciences and involves subjectivity. There is no restriction for ELT research to be conducted by specialists only; it is an interdisciplinary research that can bridge Psychology (e.g., motivation and anxiety) and Sociology (e.g., cultural assimilation and social varieties).

🔑 Definition — Interdisciplinary Research: Research that integrates concepts, theories, and methods from two or more academic disciplines to address a complex problem.

⭐ Key Takeaways

The most critical point is that research is a systematic, verifiable, and objective inquiry aimed at generating new, reliable knowledge, distinct from casual information gathering. Students must remember the multiple scholarly definitions of research and understand that its core characteristics include being systematic, logical, empirical, reductive, and replicable. The functions of research extend beyond knowledge acquisition to improving decision-making, teaching, and various aspects of human life. For ELT specifically, research is characterized as an interdisciplinary, applied, and subjective field open to non-specialists, bridging psychology and sociology.

🧠 Quick Revision Questions

1. According to Jolla (2015), what are the three core components that constitute research?
2. List five general characteristics of research as outlined in Topic-003.
3. What is the key difference between how scientists use the word "research" versus how it is used in everyday language?
4. Name three specific disciplines that ELT research can bridge, as mentioned in the lecture.
5. According to P.M. Cook, what should be the defining outcome or product of a given piece of research?

📘 Lecture 2 — Research in English Language Teaching (ELT) [Applied Linguistics]

📖 Overview: This lecture introduces the scope and importance of research in English Language Teaching (ELT) as a branch of Applied Linguistics. It covers how to identify important research questions, what ELT research investigates, and why such research is crucial for improving language education. The lecture also demythologizes common misconceptions about what research entails.

🗂️ Topics Covered

The lecture covers five main topics: identifying important questions in ELT, defining what ELT research is about, explaining why research in ELT and Applied Linguistics is important, demythologizing common misconceptions about research, and describing how to create research questions from practical problems, secondary sources, and primary research.

📝 Lecture Summary

Topic-006: Identifying Important Questions in ELT

First, identify for whom this research is intended and who will be the beneficiary of the research. The following areas are matters of concern in ELT research: understanding of language issues, students, teachers (ES/FL), admin, parents, and stakeholders. A massive number of questions and answers have been investigated in the field of English language teaching. Anything related to language and society, learning, teaching, management, decision making, and researchers can be a topic of concern in ELT research.

Topic-007: What is ELT Research About?

ELT research has vast domains of topics, with massive questions and answers. Anything related to language, practical issues, problems related to second or foreign language, and language policy are involved in the paradigm of ELT research. Applied Linguistics topics such as bilingualism, multilingualism, language education, the preservation and revival of endangered languages, assessment and evaluation, treatment of language, professional communities, and cross-cultural communication are also part of ELT research.

Topic-008: Why is Research in ELT (AL) Important?

Is language important? English language teaching has major challenges regarding the quality of learning and teaching to study. Consumers of ELT research are also emphasized while conducting research in ELT. Usually, there is not a single way of teaching; the method of teaching varies on the basis of cultures, norms, policies, countries, languages, subjects, etc. ELT research looks into all these ways of teaching. Researchers study the use of many different sets of materials to teach distinctively. The ways of conveying information are also different, leading to the questions of what to teach and how to teach. Similarly, we have different methods of assessment and evaluation and different uses of technology in classrooms. Then, researchers are also keen to know learners' and teachers' perspectives regarding all the issues in education at (L) Linguistic levels and also (T) Identity – tech incorporation.

Topic-009: Demythologizing Research

Research has some particular elements attached to it, such as searching articles and writing papers, working in labs, performing artificial experiments, and being experimented only by experts. There are many more such myths.

A true research has the following characteristics:

• Provides clear understanding of meaning
• An earnest activity
• Helps solving problems
• Carried out by skillful practitioners
• Follows a systematic research process
• Answers of 'What/why' questions
• Creates more questions
• Data is gathered, analyzed and conclusion is drawn out of it

Topic-010: Identifying Important Questions in ELT

How would we create research questions in ELT? There are some most common motivating forces like practical problems (Ferris, 1995 and Arva & Medgyes, 2000), questions in secondary sources (textbooks/theoretical papers), and primary research (sampling/design/discussions/recommendations) that lead us to locate issues and form important research questions.

Questions for research Identify a question you have in the area of teaching or learning. The question can be identified through the following sources:

• Having an experience
• Use of secondary sources (textbooks/theoretical papers)
• Reading discussion sections of related researches
• Consider: How is that important for others?

⭐ Key Takeaways

ELT research encompasses a vast range of topics from language policy to classroom technology, and is driven by practical problems, secondary sources, and prior research. It is essential to identify the beneficiaries of research (students, teachers, admin, parents, stakeholders) and to recognize that teaching methods vary across cultures and contexts. True research is a systematic, earnest activity that solves problems, answers what/why questions, and creates more questions, rather than being limited to lab experiments or expert-only work. The key motivating forces for forming research questions are practical problems, secondary sources, and reading discussion sections of primary research.

🧠 Quick Revision Questions

1. What are the six areas of concern in ELT research for identifying important questions?
2. Name four Applied Linguistics topics that are part of ELT research.
3. What is the difference between the two levels (L) and (T) that researchers investigate in education?
4. List four characteristics of true research that demythologize common misconceptions.
5. What are the three most common motivating forces for creating research questions in ELT?

📘 Lecture 3 — Classification of Research

📖 Overview: This lecture introduces the foundational classification of research, distinguishing between basic and applied levels and various types based on objectives, nature, data collection, and application. It also covers educational research, scientific versus non-scientific problem-solving methods, and the essential characteristics of an effective investigator, providing a comprehensive framework for understanding research methodology.

🗂️ Topics Covered

The lecture begins by classifying research into basic and applied levels and then explores major types based on objectives (fundamental and action), nature (exploratory and confirmatory), data collection procedures (qualitative and quantitative), and application (longitudinal, cross-sectional, conceptual, and empirical). It further details educational research, scientific ways to solve problems compared to non-scientific methods, and concludes with the key characteristics of an effective investigator.

📝 Lecture Summary

Topic-011: Classification of Research

There are two basic levels of research: basic and applied. Basic research is theoretical, contributing to subject knowledge with no immediate practical value, and is often used for broader studies. Applied research is concerned with action research, solving specific problems, and specifying a problem for practical intervention.

🔑 Definition — Basic Research: Theoretical research that contributes to general subject knowledge without immediate practical application. 🔑 Definition — Applied Research: Practical research focused on solving specific, immediate problems through action research.

Topic-012: Major Types of Research

Research is categorized into several major types based on different criteria. On the basis of research objectives, research is either fundamental or action. On the basis of nature of research, it is exploratory or confirmatory. On the basis of data collection procedures, it is qualitative or quantitative. On the basis of application of research, it is longitudinal, cross-sectional, conceptual, or empirical. Other types include educational research and historical research.

🔑 Definition — Fundamental Research: Research aimed at expanding knowledge without immediate practical application, similar to basic research. 🔑 Definition — Action Research: A type of applied research where the researcher is actively involved in solving a problem. 🔑 Definition — Exploratory Research: Research conducted to explore a problem or topic that is not well defined. 🔑 Definition — Confirmatory Research: Research designed to test hypotheses and confirm existing theories. 🔑 Definition — Qualitative Research: Research that collects non-numerical data, such as words or images, to understand concepts or experiences. 🔑 Definition — Quantitative Research: Research that collects and analyzes numerical data to test hypotheses or measure variables. 🔑 Definition — Longitudinal Research: Research that studies the same subjects over a long period of time. 🔑 Definition — Cross-sectional Research: Research that studies a population or sample at a single point in time. 🔑 Definition — Conceptual Research: Research that is theoretical and involves abstract ideas or concepts. 🔑 Definition — Empirical Research: Research based on observation and experience, using data from the real world.

Topic-013: Educational Research

In educational research, the issues of influencing educational theories and practices are emphasized. Educational problems are focused on in the library and the field. Educational philosophy is studied and investigated, and educational curriculum development is achieved. Educational research also concerns learning and teaching methods and focuses on teacher education. Key issues considered during educational research include: teacher behavior, educational administration and supervision, educational technology, research having educational implication, and educational significance.

Topic-014: Scientific Ways to Solve Problems

Non-scientific ways may solve issues, but one needs the characteristics of tenacity, intuition, authority, rationalistic approach, and experience to successfully incorporate non-scientific research in ELT. Scientific ways solve issues by following a systematic process that consists of these steps: developing the problem, forming the hypothesis, gathering the data, analyzing and interpreting data, and conclusion.

🔑 Definition — Non-scientific Ways: Problem-solving methods relying on persistence, intuition, authority, logic, or personal experience rather than systematic evidence. 🔑 Definition — Scientific Ways: A systematic, empirical method for solving problems that involves hypothesis formation, data collection, and analysis.

Topic-015: Characteristics of an Investigator

The characteristics of an effective investigator/researcher include: full understanding of functions and activities, reflective thinking, sensitive towards job, creative and imaginative, knowledge of action research, insightfulness, scientific attitude towards studying (observing problem), objectivity in thinking, democratic behavior, patience, knowledge of measuring tools, open-minded, excellence in job, and frugal.

🔑 Definition — Reflective Thinking: The ability to think critically and analytically about one's own research process and findings. 🔑 Definition — Scientific Attitude: An approach to research characterized by curiosity, skepticism, and a commitment to evidence-based inquiry. 🔑 Definition — Objectivity in Thinking: The ability to avoid personal bias and make judgments based solely on evidence.

⭐ Key Takeaways

Research is fundamentally classified into basic (theoretical) and applied (practical) levels, and further categorized by objective, nature, data collection method, and application. Educational research specifically focuses on influencing educational theory and practice, examining teacher behavior, curriculum, and teaching methods. Scientific problem-solving requires a systematic process of hypothesis formation, data gathering, and analysis, unlike non-scientific methods that rely on intuition or authority. An effective investigator must possess critical qualities such as reflective thinking, objectivity, patience, and a scientific attitude. Understanding these classifications and characteristics is essential for choosing the appropriate research approach and conducting rigorous, credible studies.

🧠 Quick Revision Questions

1. What are the two basic levels of research, and how do they differ in terms of immediate value?
2. Name the four criteria used to classify major types of research, and provide one example of a type for each criterion.
3. What are the key issues considered in educational research?
4. List the five steps involved in the scientific way of solving problems as described in this lecture.
5. Identify five characteristics of an effective investigator that are crucial for conducting objective and insightful research.

📘 Lecture 4 — Assortment of a Problem

📖 Overview: This lecture focuses on the foundational first step in research: selecting, identifying, and defining a research problem. It explains the processes of reflective and scientific thinking, outlines criteria for problem selection, and details how to properly define and state a problem to set clear directions for a research study.

🗂️ Topics Covered

The lecture covers the assortment of a problem through five key topics: the nature and process of reflective vs. scientific thinking in problem selection; identification of a problem including its sources; criteria for selecting a research problem; the process of defining a problem with its steps and precautions; and the formulation of a clear problem statement with essential criteria.

📝 Lecture Summary

Topic-016: Assortment of a Problem

The first step in research is the selection of a problem. A problem is selected through reflective thinking towards a target issue, which requires careful, ordered thinking. After selecting a problem, a keen understanding and evaluation of the research problem is required.

Reflective and scientific thinking are two key approaches. Reflective thinking involves:

• Occurrence of a problem
• Definition
• Explanation
• Elaboration
• Collection
• Conclusion

Scientific thinking involves:

• Inductive-deductive reasoning
• Cause and effect
• Assumption/hypothesis/objective/data driven

Topic-017: Identification of a Problem

This is the first and most crucial step. To identify a problem, a researcher needs to know about the following things:

• Exact nature and dimensions of the problem
• Research field
• Mastery of the area
• Basis for literary survey
• Priority field of study
• Definitions/analogies
• Pinpoint specific aspects

Sources of a problem — A researcher can find a research problem through various resources like:

• Personal experience
• Studying available literature
• Innovation (technological changes)
• Discussions in professional circles
• Secondary sources; publications/reports

Topic-018: Criteria for the Selection of a Problem

Before selecting a specific research problem, it is crucial to follow a criterion. Points to be kept in mind for selection of a problem are:

• Novelty (avoid unnecessary duplication)
• Importance for the field
• Interest and intellectual curiosity
• Training and personal qualification
• Availability of data
• Special equipment and working conditions
• Approachability of the sample
• Sponsorship and admin cooperation
• Cost and return
• Time factor

Topic-019: Defining a Problem

This is the process of reaching to the core of a problem.

Need to define a research problem — While defining a research problem, keep in mind that a research problem:

• Sets the directions for the research
• Reveals the methodology
• Controls subjectivity
• Specifies variables
• Makes research work practicable

Precautions to be taken in defining a research problem:

• Specific words
• Brief but comprehensive
• Assumptions recognized
• Practical importance
• Certain rationale

Steps involved in making a research problem:

1. Define the problem
2. Set the conceptual framework
3. Delimit the elements
4. Specify the elements
5. Key points

Topic-020: Statement of a Problem

A problem statement is the description of an issue currently existing which needs to be addressed. It provides the context for the research study and generates the questions which the research aims to answer. This involves delimiting the task and isolating specific problems.

Three criteria for a good problem statement:

• Concerned with variables involved
• Clear and unambiguous
• Amenable to empirical testing

💡 Why this matters: A well-stated problem ensures the research is focused, testable, and scientifically valid, preventing vague or unanswerable research questions.

⭐ Key Takeaways

Selecting a research problem is the foundational step of any study, requiring both reflective and scientific thinking. A researcher must identify the problem using multiple sources like personal experience and literature, and then evaluate it against key criteria such as novelty, importance, data availability, and time/cost factors. Defining a problem carefully controls subjectivity and specifies variables, while a proper problem statement must be clear, concerned with variables, and testable through empirical methods. The entire process ensures the research is practical, focused, and valuable to the field.

🧠 Quick Revision Questions

1. What are the six sequential stages involved in reflective thinking for problem selection?
2. Name at least four sources a researcher can use to find a research problem.
3. List five criteria that must be considered when selecting a research problem.
4. What are three precautions a researcher should take when defining a research problem?
5. What are the three essential criteria for a good problem statement?

📘 Lecture 5 — Foundation of a Hypothesis

📖 Overview: This lecture establishes the theoretical foundation for formulating a hypothesis in research. It distinguishes a hypothesis from assumptions and postulates, explains its functions and importance, categorizes different kinds of hypotheses, and outlines the characteristics of a good hypothesis. Understanding these concepts is critical for designing testable and effective research studies.

🗂️ Topics Covered

The lecture covers the foundation of a hypothesis, beginning with formulation as the second step in research. It then distinguishes between assumption, postulate, and hypothesis, detailing their characteristics. The functions and importance of a hypothesis are explained, followed by a classification of hypothesis types including null, directional, declarative, and question hypotheses. Finally, the characteristics of a good hypothesis are presented.

📝 Lecture Summary

Foundation of a Hypothesis

Formulation of a hypothesis is the second step in research, coming after the development of a research problem. A tentative solution to the problem is devised and formulated at the beginning. This tentative solution is a presumptive statement of a proposition — essentially a brilliant guess about the solution that is temporarily accepted as true.

Assumption, Postulate and Hypothesis

There are specific terms in research literature that are important to know for a researcher: assumptions, postulates, and hypotheses.

🔑 Definition — Assumption: A realistic expectation which is something that we believe to be true, but no adequate evidence exists to support this belief. An assumption is an act of faith which does not have empirical evidence to support it. Assumptions provide a basis to develop theories and research instruments, influencing the development and implementation of the research process.

🔑 Definition — Postulate: A basic belief upon which the scientific research method is based. Postulates are not proven but are accepted at face value for starting work and discovering other things.

🔑 Definition — Hypothesis: A tentative statement about the relationship between two or more variables. It is a specific, testable prediction about what you expect to happen in a study.

Characteristics of Assumptions:

• Taking things for granted
• Restrictive conditions
• Based on logical insight
• Truthfulness is observed

Characteristics of Postulates:

• Working beliefs of most scientific activity
• Not proven – simply accepted at face value
• Used for starting working
• Allow for discovery of other things

Characteristics of Hypothesis:

• Differs from both assumption and postulates
• Presumptive statement of a proposition
• Suggested solution
• Subject to verification
• Basis for research study to be proved or otherwise
• Hypothesis testing is an activity

💡 Why this matters: Distinguishing these three terms prevents conceptual confusion in research writing and ensures that you correctly label the logical foundations of your study.

Functions and Importance of a Hypothesis

Functions of a hypothesis:

• Provides a temporary solution to the research problem
• Offers a basis for study
• May lead to formulating more hypotheses
• Serves both a preliminary and final role
• Can be objectively tested
• Delimits the field and sensitizes the researcher to relevant data
• Acts as a means for collecting data

Importance of a hypothesis:

• Acts as the eyes of the investigator, directing observation and inquiry
• Ensures focused research rather than aimless data collection
• Provides clear and specific goals for the study
• Serves as a linking together of theory and investigation
• Functions as a guiding light throughout the research process
• Gives direction to research
• Offers greater clarity in understanding the research problem
• Has potential for stimulating further research

Kinds of a Hypothesis

Hypotheses vary in form and function and are basically of two primary types:

🔑 Definition — Relational Hypothesis: A hypothesis that proposes a relationship between variables without necessarily implying that one causes the other.

🔑 Definition — Causal Hypothesis: A hypothesis that proposes a cause-and-effect relationship between variables.

Further types include:

🔑 Definition — Null (ND) / Zero Hypothesis: An assertion that NO relation exists between the variables. It is a statistical hypothesis that is testable within a probability framework. It is used more in education and psychology research. The null hypothesis is tested for significance, leading to either acceptance or rejection.

🔑 Definition — Directional Hypothesis: Indicates the expected direction of the relationship between variables. It states a specific direction for the outcome (e.g., "Group A will score higher than Group B").

🔑 Definition — Declarative Hypothesis: States the anticipated relationship between variables in a declarative form. Existing evidence is examined to formulate this type of hypothesis.

🔑 Definition — Question Hypothesis: Used when there is no general consensus in the literature. It represents the simplest level of empirical evidence. A question may or may not qualify as a hypothesis depending on its testability and specificity.

Other names for hypothesis types include: simple, complex, alternative, statistical, and logical hypotheses.

Characteristics of a Good Hypothesis

A good hypothesis should be in agreement with the observed facts and contain the following characteristics:

• No conflict with universal laws — it must not contradict established scientific principles
• Simplest possible form — parsimony is valued (Occam's razor principle)
• Permits deductive reasoning — allows logical conclusions to be drawn and tested
• Clear verbalization — expressed in unambiguous, precise language
• Effective use of tools/techniques — can be investigated with available methods
• Provides controls for verification — can be empirically tested with proper controls
• Ensures an approachable sample — the required data and subjects are accessible
• Indicates clear roles for variables — specifies independent and dependent variables distinctly

Additionally, a good hypothesis must keep a clear distinction among:

• Theory
• Law
• Facts
• Assumption
• Postulate

💡 Why this matters: A hypothesis that fails any of these criteria will produce ambiguous results or be untestable, wasting research resources and potentially leading to invalid conclusions.

⭐ Key Takeaways

The foundation of a hypothesis begins with formulating a tentative solution to a research problem. A hypothesis is distinct from assumptions (accepted without evidence) and postulates (working beliefs of science) — it is a testable, presumptive statement about variable relationships. Hypotheses serve critical functions: focusing research, providing direction, delimiting the field, and enabling objective testing. Different types of hypotheses exist for different research purposes, including null (asserting no relationship), directional (predicting specific outcomes), declarative (stating anticipated relationships), and question forms. A good hypothesis must be simple, consistent with universal laws, clearly verbalized, testable with available tools, and must clearly distinguish variables from related concepts like theory, law, facts, assumptions, and postulates.

🧠 Quick Revision Questions

1. What is the difference between an assumption, a postulate, and a hypothesis in research methodology?
2. List four key functions that a hypothesis serves in the research process.
3. What does a null hypothesis (zero hypothesis) assert, and how is it typically tested?
4. Describe three characteristics that a good hypothesis must possess to be considered scientifically valid.
5. How does a directional hypothesis differ from a causal hypothesis?

📘 Lecture 6 — Research Variables

📖 Overview: This lecture defines research variables and their role in making hypotheses testable. It covers the major types of variables—independent, dependent, moderator, control, and intervening—and explains how they interact within a research study. Understanding variables is essential for designing clear, valid experiments.

🗂️ Topics Covered

The lecture begins by defining variables and their characteristics, then introduces the concept of operational definitions in hypotheses. It proceeds to explain each type of variable with examples, including how moderator, control, and intervening variables function. Finally, it demonstrates how multiple variables interact in a research hypothesis using a concrete example.

📝 Lecture Summary

Topic-026: Variables in a Hypothesis

A variable is anything that has a quantity or quality that varies. A hypothesis is made testable by operational definitions of variables and terms. Characteristics of variables include that they take different values and can increase or decrease over time. The types of variables are independent, dependent, moderator, control, and intervening.

Variables in a hypothesis require operational definitions of the terms involved. Any misleading thing present in the hypothesis is defined, enabling the researcher to limit the meaning according to the area discussed. Specific definitions are mentioned to ensure clarity of the statement, and the researcher must stick to those definitions.

🔑 Definition — Variable: Anything that has a quantity or quality that varies.

Topic-027: Types of Variables

Independent Variable (IV): Provides a stimulus or gives an input. It is measured, manipulated, or selected by the researcher. It is the cause of change and is always interested in affecting other variables.

Dependent Variable (DV): Responsible for the response or output. It is observed to measure the effect of the independent variable.

Moderator Variable (MD): A special type of independent variable, also called a secondary independent variable. It modifies the relation between the independent and dependent variables.

Control Variable (CV): A control variable in scientific experimentation is an experimental element which is constant and unchanged throughout the course of the investigation. It is neutralized, and factors are controlled by the researcher.

Intervening Variable (IntV): An intervening variable is a hypothetical variable used to explain causal links between other variables. Intervening variables cannot be observed in an experiment (that's why they are hypothetical). Not all variables can be seen, measured, or manipulated.

🔑 Definition — Independent Variable (IV): Provides a stimulus or input; the cause of change; measured/manipulated/selected by the researcher. 🔑 Definition — Dependent Variable (DV): Responsible for the response/output; observed to measure the effect of the IV. 🔑 Definition — Moderator Variable (MD): A special type of independent variable that modifies the relation between IV and DV. 🔑 Definition — Control Variable (CV): An experimental element kept constant and unchanged throughout the investigation; neutralized by the researcher. 🔑 Definition — Intervening Variable (IntV): A hypothetical variable used to explain causal links between other variables; cannot be observed directly.

📌 Example: Among students of the same age and intelligence, skill performance is directly related to the number of practice traits particularly among boys but less directly among girls.

• IV: Number of practice traits
• DV: Skill performance
• MD: Sex
• CV: Age, intelligence
• IntV: Learning

💡 Why this matters: This example shows how all variable types work together in a single hypothesis. The intervening variable (learning) explains why practice improves performance, even though it cannot be directly measured.

Topic-028: Research Variables Combined

Variables interact among themselves. Independent, moderator, and control variables interact with each other. It is in the researcher's control that at least one independent variable and one dependent variable in a hypothesis do interact. Intervening, extraneous, and contaminating variables may also interact. Extraneous variables are all variables which are not the independent variable, but could affect the results of the experiment.

📌 Example (repeated from Topic-027): Among students of the same age and intelligence, skill performance is directly related to the number of practice traits particularly among boys but less directly among girls.

• IV: Number of practice traits
• DV: Skill performance
• MD: Sex
• CV: Age, intelligence
• IntV: Learning (not stated in the hypothesis, but inferred)

🔑 Definition — Extraneous Variable: All variables which are not the independent variable, but could affect the results of the experiment.

⭐ Key Takeaways

A variable is anything that varies, and hypotheses become testable through operational definitions of all terms and variables. The five main types are independent (cause), dependent (effect), moderator (modifies the relationship), control (kept constant), and intervening (hypothetical causal link). In any experiment, at least one independent variable and one dependent variable must be present and interacting. Extraneous variables are uncontrolled factors that could influence results, and they must be minimized. Always identify IV, DV, MD, CV, and IntV in a research hypothesis to ensure clarity and validity.

🧠 Quick Revision Questions

1. What is a variable, and why are operational definitions important in a hypothesis?
2. Distinguish between an independent variable and a dependent variable.
3. How does a moderator variable differ from a control variable?
4. Why is an intervening variable called "hypothetical"?
5. In the example "Among students of the same age and intelligence, skill performance is directly related to the number of practice traits particularly among boys but less directly among girls," identify the IV, DV, MD, CV, and IntV.

📘 Lecture 7 — How to Locate Research?

📖 Overview: This lecture focuses on the practical process of finding existing research studies to generate and answer research questions. It explains the different types of sources (preliminary and secondary), introduces key databases like ERIC, and outlines systematic search techniques using keywords and delimiters for effective academic inquiry.

🗂️ Topics Covered

The lecture covers how to locate research by generating questions from existing studies, using search techniques with keywords and delimiters on databases like ERIC. It explains the main goal of primary research and the shift to digital resources like PCs and the internet. Finally, it details preliminary sources (expert-prepared materials and databases) and secondary sources (literature reviews, position papers, books, and bibliographies).

📝 Lecture Summary

Topic-029: (HTLR) How to Locate Research? Generating Research Questions and answers

To create research questions and find potential answers, a researcher must look for published primary research. There are various sources to find this research. Preliminary sources include the Educational Index, MLA/APA, CIJE, SSI (Social Sciences Index), and ERIC/RIE. Secondary sources include literature reviews, position papers, books, and tables of references or bibliographies.

Topic-030: Searching Answers for Your Questions

When searching for answers to research questions, specific techniques are required. First, you must delimit the area of your search and use an advanced search function. A delimiter (a character marking the beginning or end of a unit of data) is useful for this. Using specific keywords is essential.

A key resource is ERIC (Education Resources Information Center), an online library of education research sponsored by the Institute of Education Sciences (IES) of the U.S. Department of Education. You can visit it at www.eric.ed.gov. To use it, you mention your area of interest and search for it. Before starting, you must know your keywords. For example, for the question “What is the relationship between anxiety and language learning?” the keywords would be “Anxiety and language learning.” Other useful resources include Jstor and Google Scholar.

Topic-031: HTLR: Where to Look and What to Look For?

The main goal of primary research (new research carried out to answer specific issues or questions via questionnaires, surveys, or interviews) is to test proposed answers to research questions. The shift to PCs and the Internet has revolutionized this process, allowing you to perform hours or days of work in minutes. Instead of going to libraries, writing notes, or carrying books, you can use gigantic indexes and look for materials by simply clicking buttons. The sources to look for remain the same: preliminary sources, secondary sources, and tables of references and bibliographies.

Topic-032: HTLR: Preliminary Sources

Preliminary sources are expert-prepared sources that help us find research. These include publications that lead to primary research and electronic databases such as storehouses, CD-ROMs, the university library, and HEC (Higher Education Commission) or other digital resources. Keywords are also a helpful preliminary source. Other modes include the Educational Index, MLA/APA, ERIC, CIJE, and SSI (Social Sciences Index).

Topic-033: HTLR: Secondary Sources

Secondary sources are references and summaries where primary research is accessed through the eyes of someone other than the actual researchers. Valuable places to find references to primary research include literature reviews (where existing literature is summarized, viewed, and reviewed), position papers (similar to literature reviews but where concepts are expressed), books, and tables of references and bibliographies.

⭐ Key Takeaways

To locate research effectively, you must distinguish between preliminary sources (like ERIC, CIJE, and expert databases) and secondary sources (like literature reviews and position papers) that cite primary work. The core technique involves using specific keywords and delimiters for advanced searches on platforms like ERIC (eric.ed.gov). Remember that primary research is new research designed to answer specific questions, while secondary sources offer a review done by others. The digital age, with PCs and the internet, has made locating these sources fast and efficient, replacing much of the manual library work of the past.

🧠 Quick Revision Questions

1. What is the difference between a preliminary source (e.g., Educational Index) and a secondary source (e.g., a literature review)?
2. What is the main goal of conducting primary research?
3. What is a "delimiter" in the context of searching for research?
4. For the research question "How does motivation affect student achievement?", what would be the most logical set of keywords to use in ERIC?
5. List three electronic databases or digital resources mentioned as places to search for preliminary sources.

📘 Lecture 8 — Locating Primary Research

📖 Overview: This lecture introduces students to the methods and tools for locating primary research in academic databases. It distinguishes primary research from position papers, explains the importance of refereed and blind-reviewed sources, and provides practical strategies for using databases like ERIC, JSTOR, and Google Scholar to find credible research studies.

🗂️ Topics Covered

This lecture covers exploring databases for primary research, examples of popular databases like ERIC and JSTOR, the distinction between position papers and primary research, how to use tables of references and bibliographies to locate studies, and the key differences between primary and secondary sources including refereed and blind review processes.

📝 Lecture Summary

Topic-034: (LPR) Locating Primary Research: Exploring Databases

Experts prepare sources to find research. Preliminary sources that experts prepare are found in bound copies in libraries. Furthermore, databases like educational index, MLA/APA, ERIC/RIE, CIJE/LLBA/EI, and SSI (Social Science Index) are also available to find research.

Databases organize topics by a set of keywords and have a focus of studies. They are all available on CDs/Databases, Academic Search Premier, PsychInfo, Sociological Abstracts, JSTOR (1800s), and www.eric.ed.gov.

💡 Why this matters: Knowing which databases to search and how they are organized by keywords is essential for efficient and effective literature searching in academic research.

Topic-035: LPR II: Examples

Use the following options for locating primary research: • ERIC • www.eric.ed.gov • JSTOR • Google Scholar

These are key digital repositories and search engines that provide access to millions of academic articles, conference papers, and research reports.

Topic-036: LPR: Position Papers vs Primary Research

A position paper is a written report outlining someone's attitude or intentions regarding a particular matter. It is similar to a literature review. Writers argue their positions on various issues. Primary research is not included. It is used for proposing answers to research questions. It helps in generating hypothesis.

Key warnings about position papers: • Avoid the trap of circular reasoning • Generating hypothesis is okay • Supporting and justifying is NOT • Position papers cannot be a substitute for a literature review. They contain: • Witnessing of your own eyes • Focused summaries

🔑 Definition — Position Paper: A written report outlining someone's attitude or intentions regarding a particular matter, similar to a literature review, that does not include primary research.

📌 Example: A student writes a paper arguing that online learning is more effective than traditional learning, using only summaries of other researchers' opinions and existing literature, without conducting any original experiments or surveys. This is a position paper.

Topic-037: LPR: Tables of References and Bibliographies

To locate primary research, one should follow the following studies: • Profitable places to find research studies • Benchmark studies • Studies with sparking interest of the area • Particular issues • Directional change regarding the area • Use Seminal studies • Use Tactics: • Identify the frequency of citation • Important in the history of the area • Area of investigation • Go to Google Scholar

💡 Why this matters: Using tables of references and bibliographies from known studies is a powerful "snowball" method to find additional primary research in your area of interest.

Topic-038: LPR: Difference Between Primary & Secondary Sources

Primary vs secondary sources • Primary research is the only way to find answers • Not all primary research is equally important • TWO criteria: • Refereed (pass a matter to (a higher body) for a decision) • Blind review (This means that the reviewers of the paper won't get to know the identity of the author(s), and the author(s) won't get to know the identity of the reviewer.)

Distinguishing primary research from position papers Primary research and position papers are different in the following aspects: • From title • Terms and techniques • Must be known to active researchers

🔑 Definition — Refereed: A process where a research paper is passed to a higher body (experts) for a decision on publication. 🔑 Definition — Blind Review: A review process where the reviewers of the paper do not know the identity of the author(s), and the author(s) do not know the identity of the reviewer.

⭐ Key Takeaways

Students must remember that primary research is the only way to find definitive answers in research, and it must be distinguished from position papers which lack original data. The most credible primary research comes from refereed and blind-reviewed sources found in databases like ERIC, JSTOR, and Google Scholar. Using bibliographies and seminal studies is a key tactic for locating additional research. Knowing the difference between primary and secondary sources, and being able to identify position papers by their title and lack of original research techniques, is critical for academic rigor.

🧠 Quick Revision Questions

1. What are the two criteria that determine if primary research is equally important?
2. Name at least three databases mentioned for locating primary research.
3. What is a position paper, and why can it not substitute for a literature review?
4. Explain the difference between a refereed review and a blind review.
5. What tactics should a researcher use to locate primary research from tables of references and bibliographies?

Here is the summary for Lecture 9, formatted exactly as requested.

📘 Lecture 9 — Exploring Primary Research in ELT

📖 Overview: This lecture provides a practical guide for students on how to locate and access research articles in the field of English Language Teaching (ELT). It outlines various sources for obtaining these articles, identifies key professional organizations and their associated journals, and provides specific examples of relevant journals at both international and national levels. This foundational knowledge is crucial for conducting a literature review and engaging with ongoing scholarly conversations in ELT research.

🗂️ Topics Covered

This lecture covers three main topics: first, the various sources from which ELT research articles can be obtained, including libraries and digital databases; second, a list of major professional organizations and journals related to Applied Linguistics and ELT research; and third, a more detailed exploration of specific international and Pakistani journals, along with a pointer to the course textbook's appendix for further resources.

📝 Lecture Summary

Topic-039: Obtaining ELT Related Research Articles

Research articles can be obtained from a variety of sources. The most traditional source is a university library, which may also offer interlibrary loans to access materials not held in their own collection. For a fee, dedicated databases can be used to search for articles. In the Pakistani context, the HEC digital library is a key resource. Globally, JSTOR and Google Scholar are invaluable, as are TESOL specific resources.

Topic-040: Journals Related to ELT Research

The field of Applied Linguistics (ELT) has a wealth of research journals. Many of these are associated with major professional organizations. Key journals and organizations to consult include:

• TESOL (Teachers of English to Speakers of Other Languages)
• AAAL (American Association for Applied Linguistics)
• IATEFL (International Association of Teachers of English as a Foreign Language)
• AERA (American Educational Research Association)
• ESP Journal (English for Specific Purposes)

Topic-041: Exploring ELT Research Journals

This topic provides a categorized list of specific journals to explore. It recommends looking into journals from major organizations like TESOL, AAAL, IATEFL, and AERA, as well as specific journals like the ESP journal. It also highlights the importance of consulting Pakistani journals for local research. Finally, students are directed to Appendix C of your course book for a more comprehensive, specific list of journals.

⭐ Key Takeaways

A student must remember the primary channels for obtaining ELT research, including physical libraries, digital databases like the HEC library and Google Scholar, and specific society resources. The major professional organizations—TESOL, AAAL, IATEFL, and AERA—are the cornerstones of the field and their journals are essential reading. A student should be able to name these key journals and understand the importance of consulting both international publications and local Pakistani journals for a well-rounded perspective. Finally, always check the course textbook's appendix for a curated list of resources.

🧠 Quick Revision Questions

1. Name three different sources from which you can obtain ELT-related research articles.
2. What is the significance of the HEC digital library for a Pakistani student?
3. List the four major professional organizations mentioned in the lecture that are associated with ELT research journals.
4. Besides international journals, what other category of journals is specifically recommended for exploration in this lecture?
5. Where in your course book can you find a more specific list of journals to consult?

📘 Lecture 10 — Understanding Where Data Come From?

📖 Overview: This lecture introduces the fundamental concepts of sampling in research methodology, explaining where data originates and how researchers select participants or objects for study. It covers key terminology, two major sampling paradigms, and the ethical considerations involved in sampling human participants, forming the foundation for designing rigorous research studies.

🗂️ Topics Covered

The lecture covers sampling terminology including population and sample definitions with examples. It then presents two sampling paradigms: the information-rich paradigm used in qualitative research and the representative sampling paradigm used for generalization. Finally, it addresses ethics in sampling of human participants, including protection of rights and informed consent procedures.

📝 Lecture Summary

Topic-042: Sampling Terminology

Population is the entire mass of observations (the universe). All members of a group or objects are generalized as population. It is impossible to access all the population. Sample is the data drawn to test hypothesis. A sample can be a subject, informant, or participant, or one or more cases like case or inanimate objects (corpora or newspapers).

Examples of samples include Su (2001) using 122 Chinese/English native speakers, and Borg (1998) using one teacher. The sample is where data come from — it is the data source to answer research questions. A sample contains the characteristics of a specific group, and various attributes and features are studied through the sample.

🔑 Definition — Population: The entire mass of observations or all members of a group that is the focus of study. 🔑 Definition — Sample: Data drawn from a population to test a hypothesis; the actual data source to answer research questions.

Topic-043: Sampling Paradigms

There are two sampling paradigms, depending upon the nature and purpose of the study:

1. Information rich paradigm
2. Representative sampling paradigm

The information rich paradigm aims to uncover information and requires in-depth analysis. It is mainly used in qualitative research and focuses on views and feelings with a holistic small sample size. It is less generalizable.

The representative sampling paradigm serves as a replica of a larger group. It aims to be generalizable and uses large samples such as questionnaires.

🔑 Definition — Information rich paradigm: A sampling approach that prioritizes depth of information from small samples, common in qualitative research, and less focused on generalization. 🔑 Definition — Representative sampling paradigm: A sampling approach that aims to replicate a larger population through large samples, enabling generalization of findings.

💡 Why this matters: Choosing between these paradigms determines whether your study can generalize findings or instead provides deep, contextual understanding — a fundamental decision in research design.

Topic-044: The Information-Rich Paradigm

Qualitative sampling involves:

• Maximizing information
• Small sample size ranging from 1 to large
• Emphasis on the quality of information
• NOT generalizing

The example given is Borg (1998) who took one teacher as a sample.

Two guidelines are provided:

1. Very good example of small sample: Borg (1998) took one teacher
2. Examples of large samples: Guardado (2002) studied six families, specifically Spanish children in Canada

📌 Example: Borg (1998) used a sample of one teacher to conduct in-depth qualitative research. Guardado (2002) used a larger qualitative sample of six families of Spanish children in Canada, still within the information-rich paradigm.

Topic-045: Representative Sampling Paradigm

The sample serves as a true representative of the population. The goal of sampling is to generalize the findings, as access to the whole population is impossible.

For example, "EFL learners of English as students of English medium universities" — in this case, it is impossible to take the whole population to study larger populations. The rule is: the more sample size, the better it would be. The sample should highlight the important features, and the attributes of the population must be found in the selected samples.

📐 Rule: The larger the sample size, the better the representativeness → Larger samples more accurately reflect population attributes.

📌 Example: Studying "EFL learners of English as students of English medium universities" requires selecting a sample that represents the attributes of this entire population.

Topic-046: Ethics in Sampling of Human Participants

Several ethical issues relate to human participants. Protection of the rights and privacy of human beings is a matter of serious concern in research. The US Commission in 1974 and the Belmont Report 1979 devised ethics for taking humans as samples. Some organizations work for the regulation of human participants' protection.

Researchers should know the rights and exemptions before sampling. These issues can be avoided through:

• Reading and discussing ethical guidelines
• Taking informed consent with study staff
• Answering any questions from participants
• Ensuring voluntary participation
• Collecting all sorts of information
• Providing information about the study's procedures, risks, and benefits

There can be exemptions in some cases, such as educational/public behaviour or public benefit/service.

🔑 Definition — Informed consent: The process of obtaining voluntary agreement from participants after they have been fully informed about study procedures, risks, and benefits. 🔑 Definition — Belmont Report (1979): A foundational document that established ethical principles for research involving human participants.

⭐ Key Takeaways

Sampling is the fundamental process of selecting data sources from a larger population because accessing the entire population is impossible. The two main paradigms — information-rich for qualitative depth and representative for generalization — determine sample size, research approach, and the nature of findings. In qualitative research, small samples maximize information quality without aiming for generalization, as demonstrated by Borg (1998) studying one teacher. In quantitative research, larger samples better represent population attributes for generalization. Ethical sampling of human participants requires informed consent, voluntary participation, and protection of rights as established by the Belmont Report (1979), with some exemptions for educational or public behaviour.

🧠 Quick Revision Questions

1. What is the difference between a population and a sample in research?
2. Name the two sampling paradigms and explain when each is appropriate.
3. Why is the information-rich paradigm less concerned with generalization than the representative sampling paradigm?
4. What is the "rule of thumb" for sample size in the representative sampling paradigm?
5. What are the key ethical requirements when sampling human participants, according to the Belmont Report?

📘 Lecture 11 — Research Planning and Sampling

📖 Overview: This lecture introduces the crucial steps in research planning and the fundamental concept of sampling in social studies research. It explains why sampling is an indispensable technique, details the two main categories of sampling techniques (probability and non-probability), and outlines the components of sampling designs and the characteristics of a good sample.

🗂️ Topics Covered

The lecture begins by establishing research planning and sampling as a crucial step in social studies research, covering components like devising objectives and selecting strategies. It then defines sampling as a portion of a larger population, explains its importance for research feasibility and accuracy, and details two major categories of sampling techniques: probability sampling (including simple random, stratified, systematic, cluster, and multi-stage) and non-probability sampling (including convenience, purposive, quota, and referral/snowball). Finally, it covers the elements of a sampling design (sampling method and estimator) and lists the key characteristics of a good sample.

📝 Lecture Summary

Topic-047: Research Planning and Sampling

Research planning and sampling are crucial steps in social studies research. The sampling procedure consists of making choices while answering research questions. Researchers must select mandatory design components. Firstly, to select a population, secondly, go for true sampling and then choose suitable techniques and procedures. Design components are based on class of enquiry and the model to be utilized.

The components and steps involved in research planning include:

• Devise research objectives and sampling
• Research strategy + tools and techniques
• Analyzing data
• Reporting the results
• Selecting qualitative & quantitative procedures
• Selecting a valid and reliable research design

Topic-048: Meaning and Definition of Sampling

Sampling is an indispensable technique in research. It refers to a portion of a larger population. Research is impossible without sampling, as selecting a total population is impossible to be selected due to limitations of time, energy, and money. A research design is based on sampling, which is both economical and accurate. Sampling is fundamental to all statistical methods. A sample should contain maximum information about the population, and the generalizability of research data is drawn from it.

Topic-049: Sampling Techniques

There are a lot of sampling techniques which are grouped into two categories:

Probability Sampling This sampling technique uses randomization to make sure that every element of the population gets an equal chance to be part of the selected sample. It is alternatively known as random sampling. Types include:

• Simple Random Sampling
• Stratified sampling
• Systematic sampling
• Cluster Sampling
• Multi stage Sampling

Non-Probability Sampling This technique does not rely on randomization. It is more reliant on the researcher’s ability to select elements for a sample. The outcome of sampling might be biased and makes it difficult for all elements of the population to be part of the sample equally. This type of sampling is also known as non-random sampling. Types include:

• Convenience Sampling
• Purposive Sampling
• Quota Sampling
• Referral / Snowball Sampling

💡 Why this matters: The choice between probability and non-probability sampling determines the objectivity of the sample and the ability to generalize findings to the larger population.

Topic-050: Sampling Designs

A sample design is made up of two elements.

Sampling method refers to the rules and procedures by which some elements of the population are included in the sample. Some common sampling methods are simple random sampling, stratified sampling, and cluster sampling.

Estimator: The estimation process for calculating sample statistics is called the estimator. Different sampling methods may use different estimators. For example, the formula for computing a mean score with a simple random sample is different from the formula for computing a mean score with a stratified sample. Similarly, the formula for the standard error may vary from one sampling method to the next.

The "best" sample design depends on survey objectives and on survey resources. For example, a researcher might select the most economical design that provides a desired level of precision. Or, if the budget is limited, a researcher might choose the design that provides the greatest precision without going over budget.

🔑 Definition — Sample Design: A sample design is made up of two elements: the sampling method (rules for selecting elements) and the estimator (process for calculating sample statistics). 📐 Formula: Estimator → The formula for calculating a mean or standard error varies depending on the sampling method used (e.g., simple random vs. stratified).

Topic-051: Characteristics of Good Sampling

A sample should be a true representative of the population.

Characteristics of a good sample:

• Free from bias
• Objective
• Maintains accuracy
• Comprehensive in nature
• Economical
• Approachable
• Appropriate sample size
• Makes work more feasible
• Practicability
• Having maximum possible features and attributes of the population
• Free from errors

⭐ Key Takeaways

Research planning and sampling are foundational to valid social studies research. Sampling is an indispensable technique because it is impossible to study an entire population due to limitations of time, energy, and money; a good sample must be a true representative of the population, free from bias, objective, accurate, and economical. The two main categories of sampling techniques are probability sampling (using randomization for equal chance of selection) and non-probability sampling (reliant on researcher’s judgment, which may introduce bias). A sample design consists of two elements: the sampling method (rules for selection) and the estimator (formula for calculating statistics), and the best design balances precision with available resources.

🧠 Quick Revision Questions

1. What are the two main categories of sampling techniques, and what is the primary difference between them?
2. List the four types of non-probability sampling mentioned in the lecture.
3. What are the two elements that make up a sample design?
4. Why is sampling considered an indispensable technique in research?
5. State five key characteristics that define a good sample.

📘 Lecture 12 — Avoiding Errors in Sampling

📖 Overview: This lecture addresses the critical issue of sampling errors in research and how to avoid them. It emphasizes that a sample is not always a perfect representation of the population and discusses two main types of errors: random and systematic. The lecture also explores the importance of sample size, and deeply examines internal and external validity as key concepts for ensuring the credibility and generalizability of research findings.

🗂️ Topics Covered

The lecture covers the fundamental errors that can occur in sampling, including random and systematic errors. It details the importance of sample size as a crucial factor in avoiding these errors. The concept of internal validity is explored, focusing on how well an experiment avoids confounding variables and ensures that observed effects are due to the independent variable. Finally, external validity is discussed, which concerns the generalizability of research findings to other settings and populations.

📝 Lecture Summary

Topic-052: Avoiding Errors in Sampling

A sample is not necessarily always a true representative of the population. There can be errors that occur in sampling, which are classified into two primary types: random error and systematic error.

Errors in research can also be categorized in four ways: random-constant and sampling-measurement. These errors can be avoided through a solid sampling design, using a large sample, and making multiple contacts with the sample.

🔑 Definition — Random Error: An error that occurs due to chance and is unpredictable, often leading to less precision in estimates. 🔑 Definition — Systematic Error: An error that is not due to chance and consistently affects measurements in a particular direction, leading to bias. 💡 Why this matters: Understanding the difference between random and systematic errors is vital because each requires a different strategy to minimize. Random error is reduced by increasing sample size, while systematic error requires careful research design and instrument calibration.

Topic-053: Avoiding Errors: Size of Sample

The sample size is a crucial problem in avoiding sampling errors. By carefully choosing the sample size, a researcher can avoid many sampling errors. The required sample size depends upon the total number of subjects/participants present and the desired precision of the research. Before selecting the sample size, we estimate the population parameter, but there is no single rule for this. The researcher should try to reach as large a sample as possible.

For example, ( n = 30 ) in each group of the population is a common guideline. The larger the sample, the smaller the standard error would be. A sample should ideally cover 10 to 20% of the population. In descriptive research, larger size samples are typically taken. In pure experimental studies, different groups of equal sizes are preferred as sample units.

📐 Concept: Standard Error → The standard deviation of a sampling distribution; it decreases as the sample size increases. 📌 Example: A researcher wants to study the average height of students in a university with 10,000 students. To minimize error, they should aim for a sample size of 10-20% (1,000 to 2,000 students). Using a sample of only 30 students would lead to a large standard error and less reliable results.

Topic-054: Internal Validity

Internal validity refers to how well an experiment is done, especially whether it avoids confounding (more than one possible independent variable acting at the same time). The less chance for confounding in a study, the higher its internal validity.

Therefore, internal validity refers to how well a piece of research allows you to choose among alternate explanations of something. A research study with high internal validity lets you choose one explanation over another with a lot of confidence, because it avoids many possible confounds. There are two types of validity. Validity determines how sound your research design is. Internal validity is critically important. It is concerned with the effects observed in the dependent variables being only due to independent variables. It also defines the causal relationship between independent variables and dependent variables.

Extraneous variables affect internal validity because if they are not controlled, the study has lower internal validity. It is very difficult or impossible to replicate extraneous variables, and the reliability of findings cannot be checked.

🔑 Definition — Internal Validity: The extent to which a study's results can be attributed to the manipulation of the independent variable, not to other confounding factors. 🔑 Definition — Confounding: A situation where the effects of two or more variables are mixed, making it impossible to determine which variable caused the observed effect.

Credibility of sampling in Qualitative research methods faces several challenges:

• Hard to avoid selection biasness
• Maturation is required
• Sample size matters
• Careful instrumentation
• Confounding
• Testing method
• History
• Attrition (loss of participants over time)

📌 Example: A study tests a new teaching method. If the group using the new method also has a better teacher (an extraneous variable), it is impossible to know if improved test scores are due to the method or the teacher. This is a threat to internal validity.

Topic-055: External Validity

External validity refers to how well data and theories from one setting apply to another. This question is usually asked about laboratory research: Does it apply in the everyday "real" world outside the lab?

External validity determines the degree of generalizability. Can the findings of the study be generalized to the whole population? External and internal validity are not limited to each other. It depends upon the transferability to a situation.

Threats to external validity include:

• Participant characteristics should be studied keenly. Is the sample a true representative? Does the sample have all the characteristics of the population?
• Factors relating to the setting of the study. Does it relate to the real world?
• The timing of the study. Is it realistic?

🔑 Definition — External Validity: The extent to which the results of a study can be generalized to or across other populations, settings, and times. 💡 Why this matters: A study with high internal validity but low external validity may produce accurate findings that are only applicable to a very specific, artificial situation, limiting its real-world usefulness.

📌 Example: A psychological experiment on memory is conducted with university students in a quiet, controlled lab (high internal validity). However, the results may not apply to elderly people or to real-world environments with distractions (low external validity).

⭐ Key Takeaways

A student must remember that a sample can be unrepresentative due to random or systematic errors, which can be mitigated by a large and well-designed sample. The concept of validity is split into two critical types: internal validity, which concerns the accuracy of causal claims within a study and is threatened by confounding extraneous variables; and external validity, which concerns the generalizability of findings to other populations, settings, and times. It is essential to understand that a study can have high internal validity but low external validity, and vice versa. Finally, factors like participant characteristics, study setting, and timing are major threats to external validity that must be carefully considered.

🧠 Quick Revision Questions

1. What are the two main types of errors that can occur in sampling?
2. Why is sample size a crucial factor in avoiding sampling errors, and what is a general guideline for its selection?
3. Define internal validity and explain how extraneous variables threaten it.
4. Define external validity and list at least three key threats to it.
5. Can a study have high internal validity but low external validity? Provide a brief example to explain your answer.

📘 Lecture 13 — Classifying Research Designs

📖 Overview: This lecture introduces the foundational framework for classifying research designs in academic studies. It explains that research design is the overall blueprint of a study and presents three key continua — basic-applied, qualitative-quantitative, and exploratory-confirmatory — that help researchers systematically categorize and select the most appropriate design for their research questions.

🗂️ Topics Covered

The lecture covers the concept of research design as an overall plan, the three continua for classifying research designs (basic-applied, qualitative-quantitative, and exploratory-confirmatory), detailed explanations of basic versus applied research with a comparison chart, the qualitative-quantitative continuum including their data collection methods and characteristics, and the exploratory-confirmatory continuum defining their purposes. The step-wise study framework from problems to research designs is also presented.

📝 Lecture Summary

Topic-058: Classifying Research Designs

Research design is the blue print of your study, and there can be more than one possibility in selecting research design for the study. The step wise study framework is given below: Problems (Variables) → Research Questions → Population → Sample → Research Designs

You have to choose a specific type of research design that can be selected from the following categories: • Basic-Applied • Qualitative – Quantitative • Exploratory – Confirmatory

Characteristics of a Research Design • Research design must be the plan that can guide the strongest plan for your research. • Research design should be the most efficient structure to provide data for your study. • Research designs must be the most useful plan to answer research questions. • A poor research design is a house full of problems. • Mapping strategy through critical planning can help for a systematic research process.

Topic-059: CRD: Three Continua

The three continua for classifying research designs are: • Basic – Applied • Qualitative – Quantitative • Exploratory – Confirmatory

These continua represent different dimensions along which research can be positioned. They provide a systematic framework for categorizing any given study.

Topic-060: CRD: Three Continua

Classifying research designs: the basic-applied continuum

Basic Research — otherwise called pure or fundamental research — is one that focuses on advancing scientific knowledge for the complete understanding of a topic or certain natural phenomenon, primarily in natural sciences. In a nutshell, when knowledge is acquired for the sake of knowledge it is called basic research.

Basic Research is completely theoretical, focusing on basic principles and testing theories. It tends to understand the basic law. Basic Research deals with generalization and formulation of theory about human behaviour. It is aligned towards collecting information that has universal applicability. Therefore, basic research helps in adding new knowledge to the already existing knowledge.

Characteristics of Basic Research: o Highly theoretical o Hypothetical o No immediate value o Broader studies

Applied Research can be defined as research that encompasses real life application of the natural science. It is directed towards providing a solution to specific practical problems and develop innovative technology. In finer terms, it is the research that can be applied to real-life situations. It studies a particular set of circumstances, so as to relate the results to its corresponding circumstances.

Applied research includes research that focuses on certain conclusions experiencing a business problem. Moreover, research that is aligned towards ascertaining social, economic or political trends are also termed as applied research.

Characteristics of Applied Research: o Action research o Solving problems o Very practical

Comparison Chart:

💡 Why this matters: Understanding where your research falls on the basic-applied continuum defines the purpose, scope, and expected outcomes of your study. Basic research builds foundational knowledge, while applied research solves immediate real-world problems.

Topic-061: CRD: The Qualitative - Quantitative Continuum

The qualitative - quantitative continuum is the center of focus (last 30 years). They were two schools of thoughts initially depending upon different data collection procedures. Quantitative and qualitative are two ends of a continuum having separate designs and methodologies.

Quantitative data collection

Quantitative research is a form of research that relies on the methods of natural sciences, which produces numerical data and hard facts. It aims at establishing cause and effect relationship between two variables by using mathematical, computational and statistical methods. The research is also known as empirical research as it can be accurately and precisely measured.

Characteristics of Quantitative Research: • Psychology • Statistics (generalize) • To larger populations • Large scale researches

Qualitative data collection

Qualitative research is used to gain an in-depth understanding of human behaviour, experience, attitudes, intentions, and motivations, on the basis of observation and interpretation, to find out the way people think and feel. It is a form of research in which the researcher gives more weight to the views of the participants. Case study, grounded theory, ethnography, historical and phenomenology are the types of qualitative research.

Characteristics of Qualitative Research: • Anthropological – Sociological research • Verbal description • Uncover information-rich • Natural setting • Concentrated contact • Patterns-comparisons-contrast

💡 Why this matters: The choice between qualitative and quantitative approaches determines your entire data collection strategy — whether you seek numerical patterns from large samples or deep understanding from small, information-rich cases.

Topic-062: CRD: The Exploratory - Confirmatory Continuum

Confirmatory Research: Confirmatory research is where researchers have a pretty good idea of what's going on. That is, the researcher has a theory (or several theories), and the objective is to find out if the theory is supported by the facts. • Confirming something • Whether the study is going to find evidence • Supporting a hypothesis • To confirm

Exploratory Research: An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to or rely upon to predict an outcome. The focus is on gaining insights and familiarity for later investigation or undertaken when research problems are in a preliminary stage of investigation. Exploratory designs are often used to establish an understanding of how best to proceed in studying an issue or what methodology would effectively apply to gathering information about the issue.

Characteristics of Exploratory Research: • Exploring a phenomenon • Prior to developing a Ho (null hypothesis) • Doesn’t test a hypothesis • What is happening?

💡 Why this matters: This continuum determines whether you are testing an existing theory (confirmatory) or generating initial understanding when little is known (exploratory). The exploratory stage often precedes confirmatory testing.

⭐ Key Takeaways

The single most important idea is that research designs are classified along three independent continua: basic-applied, qualitative-quantitative, and exploratory-confirmatory, and every study can be positioned on each continuum. Basic research builds theoretical knowledge with no immediate application, while applied research solves practical problems; quantitative research relies on numerical data and statistics for generalization, while qualitative research provides deep verbal descriptions in natural settings; confirmatory research tests existing hypotheses, while exploratory research investigates new phenomena with no prior theory. The step-wise framework — Problems → Research Questions → Population → Sample → Research Designs — must guide systematic planning, and a poor design leads to a flawed study. Understanding these classifications allows researchers to select the most appropriate and efficient design to answer their specific research questions.

🧠 Quick Revision Questions

1. What are the three continua used to classify research designs according to this lecture?
2. How does basic research differ from applied research in terms of nature, utility, and goal?
3. List two characteristics of quantitative research and two characteristics of qualitative research.
4. What is the key difference between exploratory research and confirmatory research?
5. According to the step-wise study framework presented, what must come immediately before "Research Designs" in the planning process?

📘 Lecture 14 — Questions and Research Designs

📖 Overview: This lecture explains how research questions determine research design. It distinguishes between “What questions” (descriptive, exploratory) and “Why questions” (causal, explanatory), showing how each type of question leads to different methodological choices in quantitative, qualitative, basic, and applied research.

🗂️ Topics Covered

The lecture covers the basic relationship between questions and research designs, detailed characteristics of “What questions” and examples of studies that use them, then moves to “Why questions” that explore causation, with examples of causal-comparative, experimental, and quasi-experimental designs applied in language research.

📝 Lecture Summary

Topic-063: Questions and Research Designs

Research questions are the basic determining points for research designs. You must understand your research designs in the light of your research questions. The process involves gathering, analyzing, and concluding data.

Two generic questions are asked in the process:

• What?
• Why?

Examples of “What questions” include:

• What phenomena are important?
• What simple relationships exist between phenomena?

Examples of “Why questions” include:

• Why do some people learn languages better than others?
• Why do certain variables relate with one another?

💡 Why this matters: The type of question you ask determines whether your research will be exploratory or confirmatory, and whether you will use quantitative or qualitative methods.

Topic-064: Q&D: The WHAT Questions

Characteristics of “What questions”:

• Explain phenomenon
• Describe nature
• Description of philosophy
• State function(s)
• For example:
  • What is important about it?
  • What relationship exists?
  • What is phonological memory?
  • What is influence of bilingualism?

Key features include: information not known previously, no hypothesis to confirm, and studies mainly based on what questions are primarily exploratory. They could be both quantitative and qualitative or basic and applied.

Topic-065: Q&D: The WHAT Questions (Examples)

Example 1: What phenomena are important?

• About personal experiences of ethnic and language of pre-service teachers
• Can be studied through: Qualitative, Exploratory, Applied research designs

Example 2: What simple relationships exist between phenomena?

• The relationship among previous knowledge, topic interest and L2 reading (Carrel & Wise, 1998)
• Can be studied through: Quantitative, Exploratory, Applied research designs

Example 3: Influence of bilingualism on learning a third language (Sanz, 2000)

• Can be studied through: Quantitative, Exploratory, Basic research designs

Topic-066: Q&D: The WHY Questions

After WHAT (Phenomenon + Relation) between variables, ‘Why questions’ are asked. Why questions show causation. They involve discovering why independent variable affects dependent variable (at least one).

Used in:

• Causal studies
• Relationships and correlations
• Causal comparative studies
• Experimental and quasi experimental designs
• Exploratory – Confirmatory
• Could be both quantitative and qualitative
• Basic and applied

Topic-067: Q&D: The WHY Questions (Examples)

Why Qs and their examples:

• Why some variables influence others?
• Why simple relationships exist between phenomena?

For example: Kobayashi (2002) studied:

• Method Effects on Reading Comprehension Test Performance: Text Organization and Response Format
• Shows causation (one or more relationships)

Other terms used: impact, influence, improve, change, role of:

• Qualitative - quantitative
• Exploratory - confirmatory
• Applied - basic

Causal Qualitative Studies example: (Wesche & Paribakht, 2000) asked Why a particular enhanced reading method works better than reading only for learning vocab? — This is a qualitative-exploratory-applied study.

Causal-Comparative Designs examine effect, impact, influence.

Experimental and Quasi-Experimental Designs are used to establish causation.

⭐ Key Takeaways

The single most critical thing to remember is that research questions drive research design: “What questions” are exploratory, describe phenomena, and do not require hypotheses, while “Why questions” seek causation and require causal, comparative, experimental, or quasi-experimental designs. “What questions” can be answered by both quantitative and qualitative exploratory studies (basic or applied), whereas “Why questions” explicitly test cause-effect relationships using independent and dependent variables. The examples provided (Carrel & Wise, 1998; Sanz, 2000; Kobayashi, 2002; Wesche & Paribakht, 2000) illustrate how each question type maps onto specific methodological choices.

🧠 Quick Revision Questions

1. What are the two generic types of research questions, and what does each type aim to discover?
2. What is the main methodological difference between studies based on “What questions” versus “Why questions” regarding hypothesis formation?
3. Name two research designs suitable for answering “What questions” and two suitable for answering “Why questions.”
4. According to the lecture, what terms (besides causation) are used to describe the effect studied in “Why questions”?
5. Why would a study on the influence of bilingualism on learning a third language (Sanz, 2000) be classified as exploratory and basic, rather than confirmatory and applied?

📘 Lecture 15 — Q&RD: Extraneous Factors to Avoid

📖 Overview: This lecture explains how extraneous variables threaten the internal and external validity of research. It covers the specific types of extraneous factors—history, maturation, control group contamination, testing effects, and researcher/participant biases—and provides strategies for avoiding them. Understanding these threats is critical for designing rigorous, credible research.

🗂️ Topics Covered

The lecture begins by defining internal and external validity and the role of extraneous and confounding variables. It then examines history and maturation as temporal threats, followed by four types of control group contamination: rivalry, diffusion, compensatory equalization, and demoralization. Five ways testing can spoil results are detailed, including instrumentation, pretest/posttest effects, and time of measurement effects. Finally, the lecture covers the Pygmalion effect, Hawthorne effect, treatment novelty, accumulative treatment effects, treatment fidelity, and treatment strength-time interaction.

📝 Lecture Summary

Topic-068: Q&RD: Extraneous Factors to Avoid — Internal and External Validity

Any variable you are not intentionally studying in your dissertation is an extraneous variable that could threaten the internal validity of your results. Researchers try to control these extraneous variables so they do not become confounding variables. When an extraneous variable changes systematically along with the variables you are studying, this is called a confounding variable.

Extraneous factors are a research minefield. They may contaminate your data, affecting internal as well as external validity, as internal and external validity are not mutually exclusive. If the results (in the dependent variable, DV) are not only due to the independent variable (IV), then the results are not generalizable.

🔑 Definition — Extraneous variable: Any variable that you are not intentionally studying that could threaten the internal validity of your results.

🔑 Definition — Confounding variable: An extraneous variable that changes systematically along with the variables you are studying.

🔑 Definition — Internal validity: The extent to which the results in the DV are solely due to the IV. If results are contaminated, internal validity is weakened.

🔑 Definition — External validity: The extent to which results can be generalized beyond the study; it is compromised when extraneous factors contaminate the data.

Topic-069: EFTA: History & Maturation

The factor of history includes events taking place at different points in time. Longitudinal studies are more vulnerable and may be affected by such factors. Studies based on natural changes (in society) and physical changes (in personality) may influence the results of the study. External events occurring between pre-test and post-test may influence the results.

Maturation is an internal extraneous factor involving natural processes. It includes changes in physical, emotional, and cognitive structures. For example, in a study of L2 acquisition by young learners using different teaching methods, the IV is the teaching method, and the DV is improvement in L2. However, natural cognitive maturation in young learners could also cause improvement, confounding the effect of the teaching method.

🔑 Definition — History (extraneous factor): External events occurring at different points in time, especially between pre-test and post-test, that can influence study results.

🔑 Definition — Maturation (extraneous factor): Natural, internal processes of physical, emotional, or cognitive change that occur over time and can influence study results independently of the IV.

📌 Example: In a study testing a new teaching method (IV) on L2 improvement (DV) in young learners, the natural cognitive maturation of the children over the study period is a maturational threat. Any observed improvement could be due to maturation, not the teaching method.

Topic-070: EFTA: Control Group Contamination

Control group contamination occurs when the control group influences the results because of the treatment given to the experimental group. The control group may influence results in four ways:

1. Control group rivalry (John Henry effect): The control group tries to outdo the target (experimental) group. This happens when a group is explicitly labeled as the control group, or if group identities are not kept secret.

2. Experimental treatment diffusion (compromise): Treatment conditions leak when groups are in close proximity and there is a possibility for participants to discuss the kind of treatment they are receiving.

3. Compensatory equalization of treatments: Extra materials or special treatment are given to the control group, distorting the difference between the control group and the experimental group.

4. Demoralization (boycott) of the control group: The control group, feeling resentful about the special treatment of the experimental group, lowers their performance, acting as a potential contaminator.

🔑 Definition — John Henry effect: A form of control group rivalry where the control group tries to outperform the experimental group because they know they are the control group.

🔑 Definition — Experimental treatment diffusion: When treatment conditions spread from the experimental group to the control group because the groups are in close proximity and communication is possible.

🔑 Definition — Compensatory equalization: When researchers provide extra materials or special treatment to the control group to compensate for the experimental treatment, distorting the intended experimental-control group difference.

🔑 Definition — Demoralization (boycott): When the control group becomes resentful and deliberately lowers their performance because the experimental group receives special treatment.

💡 Why this matters: These four types of contamination can completely invalidate a comparison between experimental and control groups. To avoid them, keep group identities secret and physically separate groups.

Topic-071: EFTA: Testing

Five ways the results may be spoiled by testing procedures:

1. Instrumentation: Different instruments are used for pre-test and post-test. For example, using MCQs for the pre-test and an essay for the post-test to measure English proficiency. The solution is parallel testing—using equivalent forms of the same test.

2. Measurement–treatment interaction: The same instrument is used for all types of variables, but it may not be appropriate for all. For example, teaching grammar in an EFL class and then testing it only with MCQs (not essay type) focuses on recognition, not production.

3. The pretest effect: Taking a pretest makes participants aware of the material, giving them an advantage over another group that did not take the pretest.

4. The posttest effect (the click of comprehension effect): Administering a posttest can itself influence results, especially during an oral interview where participants suddenly understand the material because of the test format.

5. Time of measurement effect: Testing immediately after treatment gives a misleading idea about long-term effects. To control this, test participants after a week or so without warning.

🔑 Definition — Instrumentation threat: When different measurement instruments are used at different times (e.g., pre-test vs. post-test), causing differences in results that are due to the instruments, not the treatment.

🔑 Definition — Pretest effect: The awareness of material gained from taking a pretest, which can influence post-test performance independently of the treatment.

🔑 Definition — Posttest effect (click of comprehension effect): The act of taking a posttest itself causes participants to understand or perform differently, especially in oral interviews.

🔑 Definition — Time of measurement effect: Testing immediately after treatment may show temporary effects; results may not be lasting. Delayed testing without warning provides a better measure of long-term effects.

Topic-072: EFTA: Avoiding Various Factors

Several ways in which measuring the dependent variable (DV) can distort study results:

• Pygmalion effect: The researcher's effect—the researcher's perception of a participant's behavior influences how they treat the participant, which in turn affects the participant's actual behavior.

• Hawthorne effect: Participants behave differently simply because they are aware that they are being observed or participating in a study.

• Treatment intervention: The novelty and disruption of a new treatment can cause temporary effects that are not representative of long-term results.

• Accumulative treatment effect: When several treatments are given, the order of treatments can affect results. Counterbalanced designs are used to control for this.

• Treatment fidelity: Whether the treatment was administered in the correct manner, as intended.

• Treatment strength–time interaction: Whether sufficient time was given for a noticeable effect to occur. For example, a teaching methodology needs enough time before any effect can be observed.

🔑 Definition — Pygmalion effect: A researcher's expectation or perception of a participant's behavior influences the researcher's actions, which then causes the participant to behave in accordance with that expectation.

🔑 Definition — Hawthorne effect: Participants alter their behavior because they know they are being studied, not because of the treatment itself.

🔑 Definition — Treatment fidelity: The extent to which the treatment was delivered consistently and correctly as planned by the researcher.

📌 Example: If a study aims to test a new teaching methodology, the treatment might show no effect if it was not given enough time (e.g., only one class period). This is a treatment strength–time interaction—the treatment needs sufficient duration for a noticeable effect to occur.

⭐ Key Takeaways

Extraneous variables are any unstudied variables that threaten internal validity; when they change systematically with the IV, they become confounding variables. History (external events over time) and maturation (natural internal changes) are two key temporal threats, especially in longitudinal studies. Control group contamination can occur through rivalry (John Henry effect), treatment diffusion, compensatory equalization, or demoralization, so group identities must be kept secret. Testing can spoil results through instrumentation mismatches, pretest or posttest effects, and inappropriate timing of measurement. Finally, researcher and participant biases—Pygmalion effect, Hawthorne effect, treatment novelty, accumulative effects, and treatment fidelity or strength-time issues—must be controlled to ensure valid, generalizable findings.

🧠 Quick Revision Questions

1. What is the difference between an extraneous variable and a confounding variable?
2. Explain how maturation could threaten the internal validity of a study on L2 acquisition in young learners.
3. List and briefly describe the four types of control group contamination.
4. What is the instrumentation threat and how can it be controlled in a pre-test/post-test design?
5. How does the Pygmalion effect differ from the Hawthorne effect in terms of who is influenced?

📘 Lecture 16 — Data Collection Procedures

📖 Overview: This lecture introduces the two main categories of data collection procedures in applied linguistics research: observational and instrumental. It explains who or what is used to gather data and how different procedures provide answers to research questions.

🗂️ Topics Covered

The lecture covers data collection procedures, focusing on observational procedures (including self-observation, outside observers, and interviewers) and instrumental procedures (including questionnaires, tests, and surveys). Examples of each type are provided, such as protocol analysis, ethnographic studies, Likert scales, and standardized tests like TOEFL and IELTS.

📝 Lecture Summary

Topic-073: Data Collection Procedures

Once research designs are decided, researchers need to select the data collection procedure. There are many ways for data collection, and the value of a data collection procedure is how well it provides answers to the research questions. Commonly used data collection procedures in applied linguistics fall into two main categories: observational and instrumental.

🔑 Definition — Observational DCP: Data collection procedures that rely on human observation, including participant observation, non-participant observation, sheets, and judges/raters.

🔑 Definition — Instrumental DCP: Data collection procedures that rely on instruments or tools, including questionnaires (closed-open) and tests.

Topic-074: DCP: Observational Procedures

These are data gathering procedures based on observation with many possibilities. The observer can be self (introspection/retrospection) or others (participants – full, non-, or partial). Other roles include interviewers and judges-raters.

Topic-075: DCP: Observational Procedures-Examples

Data gathering procedures based on observation can occur through visual observation using human observation for data gathering. The self as observer uses participants as observers of their own behavior, including protocol analysis (their own internal cognitive states – introspection/retrospection). Outside observers are others who are observing, including full participants, partial participants, and non-participants. This is used in ethnographic studies and conversational analysis. The interviewer conducts observation under highly structured conditions with paper-and-pencil data recording that can be structured, semi-structured, or unstructured.

📌 Example: In an ethnographic study, a researcher might live within a community as a full participant observer, recording language use patterns in daily interactions.

Topic-076: DCP: Instrumental Procedures

These are data gathering procedures based on instrumentation with many possibilities. They include questionnaires (closed-form or open-form) and tests (discrete items or constructed response).

🔑 Definition — Closed-form questionnaire: A questionnaire with predetermined response options (e.g., yes/no, agree/disagree). 🔑 Definition — Open-form questionnaire: A questionnaire allowing free, unstructured responses from participants.

Topic-077: DCP: Instrumental Procedures-Examples

Data gathering procedures based on instrumentation use a range of devices – questionnaires and tests. They can reach maximum participants and are very economical ways to collect data. Surveys may be open or closed. Standardized tests include TOEFL (ETS) and IELTS (UoC). Questionnaires may use Yes-No Questions, Agree-disagree Questions, or the Likert scale (a scale measuring agreement levels). Examples of such tests include the First Certificate in English (FCE), Comprehensive English Language Test (CELT), and McArthur Communicative Development Inventory (CDI). Tests can be norm-referenced (comparing to a norm group) or criterion-referenced (measuring against a fixed standard).

📐 Formula: Likert scale → A psychometric scale where respondents specify their level of agreement or disagreement with a statement (e.g., Strongly Agree – Agree – Neutral – Disagree – Strongly Disagree). 💡 Why this matters: Choosing between norm-referenced and criterion-referenced tests affects how you interpret results and what conclusions you can draw about your participants' abilities.

📌 Example: A researcher studying learner attitudes might use a questionnaire with Likert-scale items like "I enjoy speaking English in class" (Strongly Agree to Strongly Disagree), then administer a standardized test like TOEFL to measure proficiency.

⭐ Key Takeaways

Students must remember that data collection procedures are divided into observational (based on human observation, including self, participants, interviewers, and judges) and instrumental (based on tools like questionnaires and tests). Observational procedures vary by the observer's role (full, partial, non-participant) and structure (structured to unstructured). Instrumental procedures include both closed-form and open-form questionnaires, as well as tests with discrete items or constructed responses. The choice of procedure directly impacts how well it answers research questions. Standardized tests like TOEFL and IELTS, alongside instruments like Likert scales, are common in applied linguistics research.

🧠 Quick Revision Questions

1. What are the two main categories of data collection procedures in applied linguistics?
2. What is the difference between full participant observation and non-participant observation?
3. What is protocol analysis and who uses it in observational data collection?
4. What is the difference between closed-form and open-form questionnaires?
5. What is the distinction between norm-referenced and criterion-referenced tests?

📘 Lecture 17 — Qualities of Good Data-Gathering Procedures

📖 Overview: This lecture examines the two essential qualities of any data-gathering procedure: reliability and validity. Understanding these qualities allows researchers to critically evaluate the strength of research findings and discern weak from strong studies. The lecture provides detailed explanations, examples, and practical applications of different types of reliability and validity.

🗂️ Topics Covered

The lecture covers the concept of reliability in data collection, including the reliability coefficient and various types such as interrater, intrarater, test-retest, alternate-form, and internal consistency methods. It discusses factors that affect reliability like subjectivity, test length, and item quality, before moving to the Standard Error of Measurement. The second half addresses validity as a global construct, examining trait accuracy and utility, along with criterion-related, content coverage, and face appearance facets.

📝 Lecture Summary

Reliability

Reliability concerns the consistency of data results. A reliable method gives the same results regardless of who takes the measurement or when it is administered. The most common indicator for reporting reliability is the correlation coefficient, a number that quantifies the degree to which two variables relate to one another. When used for reliability, these are called reliability coefficients, which range between 0.00 and +1.00. A coefficient of 0.00 means no reliability (inconsistent results), while 1.00 indicates perfect reliability (identical results every time). Seldom do coefficients occur at these extremes. The rule of thumb is the higher the better, but what constitutes "adequate" depends on the measurement procedure: observation techniques using judges are satisfactory with coefficients from 0.80 upward, achievement and aptitude tests should have reliabilities in the 0.90s, and interest inventories or attitude scales tend to be lower. Generally, reliabilities falling below 0.60 are considered low regardless of procedure type (Nitko, 2001).

🔑 Definition — Reliability coefficient: A number between 0.00 and +1.00 that represents the consistency of an observation or measurement procedure.

🔑 Definition — Correlation coefficient: A number that quantifies the degree to which two variables relate to one another.

Types of Reliability Coefficients

Different types of reliability coefficients reveal different kinds of consistency, and different measurement procedures require different kinds of consistency.

Interrater or interobserver reliability is required whenever different observers are used to observe or rate participants' behavior. Researchers determine reliability by computing a correlation coefficient or calculating a percentage of agreement. The Bejarano et al. (1997) study used two independent raters and reported interrater reliabilities of 0.98, 0.86, and 0.96, revealing high agreement among raters.

Intrarater reliability addresses whether the same observer/rater gives consistent results if given the opportunity to observe/rate participants on more than one occasion. High agreement is expected within the same person if the attribute is stable and the observer understood the task. Goh's (2002) study on listening comprehension techniques computed an interrater reliability coefficient of 0.76 and an intrarater reliability coefficient of 0.88, showing that she agreed with herself more than with her colleague.

Test–retest reliability measures the stability of the same instrument over time. The instrument is given at least twice, and a correlation coefficient is computed on the scores. This only works if the trait being measured remains stable between administrations. Camiciottoli (2001) used a 22-item questionnaire, gave 20 participants the same questionnaire 6 weeks later, and found a reliability coefficient of 0.89, considered fairly high.

Alternate-form reliability is used when a test has several different forms. Different forms are given to the same individuals with several days or more between administrations, and results are correlated. This tests both stability over time and whether items in different forms represent the same general attribute.

Researchers cannot assume that borrowing items from commercially produced standardized tests inherits the same reliability. Test items often behave differently when put into other configurations, so subtests from larger instruments should be reevaluated for reliability before use.

Internal Consistency Reliability

The last three methods of estimating reliability concern internal consistency — whether all items in an instrument measure the same general attribute. This is important because item responses are normally added to make a total score; if items measure different traits, a total score would not make sense.

Split-half (odd/even) reliability divides the test items in half. Responses on each half are summed and compared. The odd/even method (comparing odd-numbered items with even-numbered items) is preferred because it is not influenced by qualitative changes in items like difficulty or fatigue that often occur in different sections.

Factors Affecting Reliability

Several factors affect reliability. Subjectivity — the more a procedure is vulnerable to perceptual bias, lack of awareness, or fatigue, the lower the reliability. Test length — instruments with fewer items automatically produce smaller reliability coefficients due to mathematical limitations. The Spearman-Brown prophecy formula projects what the reliability estimate would be if the test had more items. Garcia and Asencion (2001) used this procedure, reporting an interrater reliability of 0.98 for a text reconstruction test and a split-half reliability with Spearman-Brown adjustment of 0.73 for a 10-item listening test. Conversely, instruments that are too long cause fatigue, producing inconsistent responses and lowering reliability. Item quality — ambiguous test items, items with more than one correct answer, or trick items produce inconsistent results and lower reliability. Scarcella and Zimmerman (1998) dropped 10 items from their Test of Academic Lexicon because they lowered the Cronbach alpha coefficient.

🔑 Definition — Standard Error of Measurement (SEM): An estimate of how much error there is in a measurement procedure, calculated from the reliability coefficient. Error is any variation in instrument results due to factors other than what is being measured.

📐 Formula: Perfect reliability (r = 1.00) means no error — all differences between scores are true differences. No reliability (r = 0.00) means no difference between scores can be interpreted as true difference on the trait being measured.

📌 Example: If a language proficiency test has no reliability, even though scores differ across individuals, the researcher cannot conclude that someone who scored higher has higher proficiency — all differences would be attributed to error from unknown sources.

Validity

Validity refers to the ability of an instrument or observational procedure to accurately capture data needed to answer a research question. Since the early 1990s, previous notions of validity have been subsumed under construct validity (Bachman, 1990; Messick, 1989), with different types now represented as different facets under this global title.

Trait accuracy addresses how accurately the procedure measures the trait (construct) under investigation. Accuracy depends on the definition of the construct being measured. If language proficiency is defined as grammar and vocabulary knowledge plus reading and listening comprehension, the approach must measure all these components. If other researchers define it as oral and writing proficiency, they must assess speaking and writing ability. The degree to which a procedure is valid for trait accuracy is determined by how well the procedure corresponds to the definition of the trait. These definitions should appear in the introduction or methodology section. Gardner et al. (1997) defined language anxiety as "communication apprehension, test anxiety, and fear of negative evaluation" based on the Foreign Language Classroom Anxiety Scale.

Utility is concerned with whether measurement/observational procedures are used for the right purpose. If a procedure is not used for what it was originally intended, the researcher must provide additional evidence of validity for their specific purpose.

🔑 Definition — Construct validity: The overarching concept that subsumes all types of validity, referring to the degree to which a procedure measures the theoretical construct it claims to measure.

Facets of Validity

Three additional facets qualify the main facets of trait accuracy and utility.

Criterion related means the procedure is validated by being compared to some external criterion. It is divided into two types: capacity to succeed (a person having the aptitude to succeed in some endeavor) and current characteristics. Capacity to succeed is typically used for prediction purposes. Predictive utility is determined by correlating measurements from the procedures with measurements on the criterion being predicted. The Modern Language Aptitude Test (Carroll & Sapon, 1959) was developed to predict whether people have aptitude for learning languages. Steinman and Smith (2001) presented evidence that this test is valid for making predictions and has become used as an external criterion for validating other tests.

⭐ Key Takeaways

The two most important qualities of any data-gathering procedure are reliability and validity, and the confidence we place in research findings is directly proportional to how reliable and valid these procedures are. Reliability concerns consistency and is measured using correlation coefficients ranging from 0.00 to +1.00, with different types (interrater, intrarater, test-retest, alternate-form, and internal consistency) revealing different kinds of consistency for different measurement situations. Validity has been reconceptualized as a global concept of construct validity, with trait accuracy and utility as main facets, and criterion-related, content coverage, and face appearance as qualifying facets. Researchers must always report reliability and validity evidence for their instruments, and cannot assume that borrowing items from validated tests inherits the same psychometric properties. The Standard Error of Measurement demonstrates that without reliability, no differences between participant scores can be interpreted as true differences on the trait being measured.

🧠 Quick Revision Questions

1. What is the difference between interrater reliability and intrarater reliability, and what does each measure?
2. Why can a researcher not assume that borrowing items from a commercially produced standardized test will result in the same reliability?
3. What is the Spearman-Brown prophecy formula, and when is it typically used?
4. According to the modern conceptualization of validity, what are the two main facets of construct validity, and how do they differ?
5. Why would using TOEFL results to measure the effects of a 2-week training program be considered invalid?

📘 Lecture 18 — Understanding Research Results

📖 Overview: This lecture explains how research results—both verbal (qualitative) and numerical (quantitative)—are presented, analyzed, and evaluated. It emphasizes that numerical data are not inherently more scientific than verbal data and details the processes for ensuring credibility in qualitative research findings. Understanding these evaluation strategies is crucial for critically consuming research.

🗂️ Topics Covered

The lecture begins by addressing the false assumption that numerical data is more scientific than verbal data. It outlines the common procedure of data selection and reduction in all research reports. The main body distinguishes between the presentation and analysis of verbal data versus numerical data, delving into the qualitative "data analysis spiral" and potential biases like holistic fallacy, elite bias, and going native. It then presents a comprehensive list of verification procedures, which are categorized into tactics for evaluating data quality and for evaluating explanations and conclusions. Key strategies for data quality include representativeness, prolonged engagement, clarifying researcher bias, researcher effects, and weighting the evidence. For evaluating explanations, the lecture covers spurious relationships, if-then tests, rival explanations, replicating findings, informant feedback, rich/thick description, and external audits.

📝 Lecture Summary

Presentation and Analysis of Verbal Data

In qualitative research, the presentation and analysis of data are deeply intertwined. Unlike numerical data, where summaries are presented before analysis, the analysis of verbal data begins at the start of data collection and continues throughout the study. The researcher becomes the primary "measurement device," engaging in what is called the data analysis spiral—a process of engaging, reflecting, noting, organizing, coding, reducing data, and looking for themes. However, published reports rarely describe this process clearly, which raises credibility concerns.

Several analytical biases can threaten the validity of conclusions. These include the holistic fallacy, where a researcher sees patterns that are not actually present; elite bias, which occurs when too much weight is given to articulate or well-informed informants, making the data unrepresentative; and going native, where the researcher becomes so close to the respondents that they adopt their perceptions.

🔑 Definition — Data analysis spiral: A model of qualitative data analysis where the researcher repeatedly engages with the data, reflects, makes notes, organizes, codes, reduces, looks for relationships and themes, checks credibility, and eventually draws conclusions.

Evaluating the Quality of Data

Verbal data cannot be taken at face value. Several strategies help ensure the data is dependable for analysis.

1. Representativeness: This concerns the internal validity or credibility of the data, not just the sample. The veracity of information can be influenced by the choice of respondents or events. A researcher must provide evidence that generalizations are based on appropriate events and an adequate number of observations, so conclusions are not based on the "luck of the draw."

2. Prolonged Engagement and Persistent Observation: The researcher must spend enough time interacting with respondents or events to gather accurate data. While necessary, too much time on site can lead to other researcher effects.

3. Clarifying Researcher Bias: Every researcher has biases. Since analysis begins during data collection, disclosing these biases helps the consumer understand why data is gathered and interpreted in a certain way, and how the researcher arrived at their conclusions.

4. Researcher Effects: There is a reciprocal relationship between the researcher and the participants/events. The researcher's presence can influence the behavior being observed. Consumers should look for evidence that the researcher tried to control for or was at least aware of their own effect on the data and the analysis process. Strategies to avoid this include spending more time on site to become unnoticed and using unobtrusive methods.

5. Weighting the Evidence: Some data are stronger than others. To evaluate data strength, consider: the proximity of informants to the data (closer is stronger), the extent of firsthand observation of actual behavior, and the effort made to check for biases during data gathering.

Evaluating Explanations and Conclusion

This section outlines tactics to evaluate the validity of the researcher's conclusions.

• Spurious Relationships: Not all relationships are direct. A relationship between two variables may be caused by a third, unmeasured variable. A researcher must provide a convincing argument that no other variables are producing an observed relationship.
• If-Then Tests: This is the "workhorse of qualitative data analysis." It involves forming a conditional hypothesis (if X is true, then Y should occur) and testing it on a new sample to see if the predicted consequence appears.
• Rival Explanations: A powerful way to add weight to a conclusion is to formulate and test a plausible competing explanation. The explanation that best fits the data is the most plausible. However, consumers should beware of straw man arguments—explanations that are not truly plausible and are easy to refute. The last "standing" explanation may not be the best one.
• Replicating Findings: When the same findings occur across different samples and conditions, confidence in the conclusions increases. A single study from one setting has limited practical use.
• Informant Feedback (Member Checks): Getting feedback from the informants on the study's conclusions can check the plausibility of the researcher's patterns. However, care is needed as respondents may agree to please the researcher or may not understand a technical report.
• Rich, Thick Description: A detailed description of the participants, context, and data-gathering/analysis stages. This allows the reader to decide if the conclusions are transferable to other similar situations.
• External Audits: Hiring an outside expert to evaluate the study is a powerful, though seldom used, method to increase credibility.

🔑 Definition — Spurious relationship: A misleading relationship between two variables that appears to be directly related but is actually caused by a third, related variable.

💡 Why this matters: These evaluation tactics are essential for the consumer of qualitative research. Without them, it is impossible to trust the conclusions drawn from verbal data, which are inherently subjective.

⭐ Key Takeaways

The critical point from this lecture is that verbal and numerical data are not different in scientific value; numbers are just a transformation of verbal constructs. The core of qualitative research analysis is a spiral process where the researcher is the main instrument, and this process must be clearly reported for the findings to be credible. To achieve credibility, a researcher must actively evaluate the quality of their data using tactics like ensuring representativeness, clarifying their own bias, and managing researcher effects. Finally, the validity of explanations must be rigorously tested using if-then tests, rival explanations, and replication, with results presented through rich, thick descriptions to allow readers to assess transferability.

🧠 Quick Revision Questions

1. What is the difference between the "holistic fallacy" and "elite bias" as threats to validity in qualitative analysis?
2. Explain the "data analysis spiral" as described by Creswell.
3. What is a "spurious relationship," and what must a researcher do to rule one out in their conclusions?
4. Why is "clarifying researcher bias" important for evaluating the quality of data?
5. What is the purpose of rich, thick description in a qualitative research report, and how does it help the consumer?

📘 Lecture 19 — Understanding Research Results-II

📖 Overview: This lecture explains how researchers analyze numerical data, covering both descriptive and inferential statistics. It focuses on understanding the shape of data distributions, measures of average and variation, and the crucial concepts of null hypothesis testing and statistical significance. The goal is to equip research consumers to evaluate whether appropriate statistical procedures were used and whether results were interpreted correctly.

🗂️ Topics Covered

The lecture begins with an introduction to presentation and analysis of numerical data, explaining why numerical data is preferred and the distinction between population, sample, statistics, and parameters. It then covers understanding statistics of data, focusing on shape of distribution, measures of average (mean, median, mode), and measures of variation (standard deviation, semi-interquartile range, range). The lecture continues with understanding inferential statistical procedures, explaining the null hypothesis and its role in research. Finally, it covers statistical procedures, dividing them into nonparametric and parametric categories, and explores relationships between variables using chi-square and Spearman rank correlation.

📝 Lecture Summary

Presentation and Analysis of Numerical Data

Researchers often convert ideas into numerical data because it is easier to work with than verbal data. Statistical procedures quickly identify patterns and relationships in large data sets and can estimate whether findings exceed random chance. The purpose is to introduce common procedures for analyzing numerical data and the basic concepts underpinning them. Understanding formulas is not necessary for research consumers; instead, knowing whether an appropriate procedure was used and whether results were interpreted correctly is essential. Population is the entire number of people to which the researcher wants to generalize conclusions. Sample is a subgroup of that total number. Statistics are quantities gathered on a sample, estimates of what would be found using the whole population. Parameters are quantities gathered directly from the entire population, representing true values that exactly describe the population. Because researchers almost always deal with samples, they use statistics and must make inferences about the population, with statistics helping to understand the chance of making a mistake when inferring from sample to population. Statistics divide into two main categories: descriptive statistics describe a set of data, and inferential statistics use those descriptions to determine whether researchers can generalize findings to a target population.

🔑 Definition — Population: The entire number of people to which the researcher wants to generalize conclusions. 🔑 Definition — Sample: A subgroup of the total population used for data collection. 🔑 Definition — Statistics: Quantities (or numbers) gathered on a sample that estimate what would be found if the whole population were used. 🔑 Definition — Parameters: Quantities gathered directly from the entire population; the true values that exactly describe the population. 🔑 Definition — Descriptive Statistics: Statistics that describe a set of data. 🔑 Definition — Inferential Statistics: Statistics that produce answers and determine whether researchers can generalize findings to a target population.

Understanding Statistics of Data

There are three basic concerns when using descriptive statistics to describe numerical data: the shape of the distribution, measures of average, and measures of variation. The first concern is whether the data are symmetrically distributed and approximate a normal curve. If a distribution is severely skewed (lopsided), rectangular (no curve), or multimodal (more than one cluster of data), certain statistics should not be used. Based on shape, the second concern is which statistic to use to describe average. There are three: mean, median, and mode. The mean is computed by adding all scores and dividing by the total number of scores. The median is the middle point dividing the number of people in half. The mode is the most frequent score. The mean is the most common estimate of average, but if the distribution does not approximate normal, other indicators are more accurate. The third concern, also affected by shape, is what statistic to use to indicate variance (how much data vary). There are three measures of variation: standard deviation, semi-interquartile range, and range. The standard deviation (SD) is the average deviation of scores from the mean. The semi-interquartile range estimates where the middle 50% of scores are located. The range is the distance from lowest to highest scores. The standard deviation is most commonly used but is only appropriate if the distribution does not vary too much from normalcy.

🔑 Definition — Normal curve: A symmetrical distribution of data that is bell-shaped. 🔑 Definition — Mean: Computed by adding up all scores and dividing by the total number of scores. 🔑 Definition — Median: The middle point in the distribution of data that divides the number of people in half. 🔑 Definition — Mode: The most frequent score in a data set. 🔑 Definition — Standard deviation (SD): The average deviation of scores from the mean. 🔑 Definition — Semi-interquartile range: A measure of variation that estimates where the middle 50% of scores are located in the data distribution. 🔑 Definition — Range: The distance from the lowest to the highest scores in the distribution.

Understanding Inferential Statistical Procedures

Researchers attempt to infer their findings to a population based on a sample, and inferential statistics play a crucial role in this process. The main goal is to describe common inferential statistical procedures, explain why they are used, and provide examples. However, understanding null hypothesis and statistical significance is more important than remembering procedure names.

The Null Hypothesis

Statistical significance directly relates to testing the null hypothesis. Inferential statistics answer two types of questions: are there relationships between variables, or are there differences between groups? The null hypothesis states that there is either no relationship or no difference between groups. Regardless of whether a research hypothesis exists, the null hypothesis is always tested. In exploratory studies with no stated hypotheses, behind every relationship studied is a null hypothesis stating no relationship. For every study exploring differences between groups, a null hypothesis states no real difference. Few published studies explicitly state null hypotheses today, but they are always lurking. An example from Tsang (1996) stated five null hypotheses, one being "There is no significant main effect for nature of program...as a factor in writing performance." The phrase "no significant main effect" means no differences between different programs regarding effect on writing performance. Stating hypotheses in null form is more accurate because inferential statistics test the null hypothesis, not positively stated hypotheses.

🔑 Definition — Null hypothesis: The hypothesis that states there is either no relationship between variables or no difference between groups. 🔑 Definition — Statistical significance: A concept directly related to testing the null hypothesis, indicating whether results are greater than random chance.

Statistical Procedures

Inferential statistics divide into two general categories: nonparametric and parametric. Nonparametric statistics are used for analyzing data in the form of frequencies, ranked data, and data that do not approximate a normal distribution. Parametric statistics are used for data that do not stray too far from a normal distribution and typically involve means and standard deviations. Scores on tests and surveys usually fit these criteria. The objectives are to find relationships between variables or differences between groups, with both nonparametric and parametric procedures available under each objective.

🔑 Definition — Nonparametric statistics: Statistical procedures used for frequencies, ranked data, and data not approximating a normal distribution. 🔑 Definition — Parametric statistics: Statistical procedures used for data approximating a normal distribution, typically involving means and standard deviations.

Relationships between Variables: Nonparametric Procedures

Two frequently seen procedures are chi-square and Spearman rank correlation. The Pearson chi-square (pronounced Ky-square, symbol χ²) is the procedure of preference when dealing with data in the form of frequencies or percentages. In its simplest form, chi-square compares observed frequency (or percentages) of different levels of a variable with what would be expected if no relationship existed (the null hypothesis). For example, if a researcher asks whether there is a relationship between gender and success in learning English as a foreign language, they would compare a random sample of males and females on success rate. The null hypothesis would be: no relationship between gender and success rate, therefore no difference between males and females who pass or fail. If true, expected frequency should be 20/20 for each sex. In fictional data, the researcher found 27 females versus 17 males passed, and 13 females versus 23 males failed. Although frequencies appear to differ, the researcher must do a chi-square analysis rather than rely on "eyeball" analysis to determine if they differ from what would be expected under the null hypothesis.

🔑 Definition — Pearson chi-square (χ²): A nonparametric statistical procedure that compares observed frequency of different levels of a variable with expected frequency under the null hypothesis. 📌 Example: Researcher asks "Is there a relationship between gender and success in learning English?" Null hypothesis: no relationship, so expected frequency is 20/20 for each sex. Observed: 27 females and 17 males passed; 13 females and 23 males failed. The chi-square analysis determines if the observed frequencies differ significantly from expected frequencies under the null hypothesis.

⭐ Key Takeaways

Students must remember that statistics divide into descriptive (describing data) and inferential (generalizing findings) categories. The three concerns when describing numerical data are shape of distribution (normal vs. skewed/rectangular/multimodal), measures of average (mean, median, mode), and measures of variation (standard deviation, semi-interquartile range, range). The null hypothesis always states no relationship or no difference and is what inferential statistics actually test, even when not explicitly stated. Parametric statistics require normal distributions while nonparametric statistics handle frequencies, ranks, and non-normal data. Finally, chi-square is the appropriate procedure for analyzing relationships between variables when data are in frequency form.

🧠 Quick Revision Questions

1. What is the difference between statistics and parameters?
2. Name the three measures of average and briefly define each.
3. Under what conditions should the mean and standard deviation NOT be used to describe data?
4. What does the null hypothesis always state about relationships or differences?
5. When would a researcher use chi-square rather than a parametric procedure?

📘 Lecture 20 — WRITING DISCUSSION OF YOUR RESEARCH

📖 Overview: This lecture focuses on how to write the discussion and conclusion sections of a research paper. It explains the main goals, key ingredients, and effective writing approaches, emphasizing how to interpret findings, relate them to previous studies, and defend their significance.

🗂️ Topics Covered

The lecture covers the main goals and writing approaches for the discussion section, the essential ingredients for writing effective discussion and conclusion sections, how to summarize key points by checking logical consistency, and the difference between discussion and conclusion with strategies for making the discussion very effective.

📝 Lecture Summary

Topic-092: Discussion: main goals and writing approaches

The format of the discussion and conclusion sections varies across different journals and disciplines. This is often the most read part of a research study because it interprets findings and suggests practical applications. The writer must check for valid interpretation of the results. A good discussion should provide an overview of the study, an overview of the findings, and the relation of the findings with previous studies.

Topic-093: Discussion: what is important?

The key ingredients for the discussion and conclusion include: an overview of the study; an overview of the findings; the relation of findings to previous studies; attention to limitations; possible applications of the research; and future possibilities for research.

Topic-094: Discussion and conclusion: summarizing your key points

When summarizing key points, you must ask several critical questions. Do the findings logically answer the research questions or support the research hypothesis? Does the nature of the study remain consistent from beginning to end? Are the findings generalized to the correct population or situations? Are the conclusions consistent with the type of research design used? Are the findings and conclusions related to theory or previous research? Are any limitations of the study made clear? Is there consistency between the findings and the applications?

Topic-095: Discussion and conclusion: defending your answers and their significance

It is crucial to understand the difference between a discussion and a conclusion before drafting your manuscript. The discussion is the hardest part; you must think. After presenting results, you answer the research questions—what do they mean? The discussion should interpret results, compare with previous research, discuss limitations, address unexpected results, and explain how they add value. The conclusion should restate your hypothesis, state important findings, highlight limitations, explain overall significance, and state future directions.

To make your discussion very effective, you should put the most important findings front and center, contextualize the meaning, most effectively demonstrate your ability, and highlight the importance of your study. Do not be apologetic—repress your doubts and convey confidence.

⭐ Key Takeaways

The discussion section interprets results and relates them to previous studies, while the conclusion summarizes the overall significance and future directions. Both sections require an overview of the study, findings, limitations, applications, and future possibilities. A writer must check for logical consistency between findings, research questions, design, and theory. The most important findings should be presented first, and the writer must convey confidence without being apologetic. Understanding the difference between discussion and conclusion is essential for drafting an effective manuscript.

🧠 Quick Revision Questions

1. What are the five essential ingredients for an effective discussion and conclusion?
2. What is the primary difference between the functions of a discussion section and a conclusion section?
3. What key questions should you ask to ensure your findings logically answer your research questions?
4. What strategy should a writer use to make their discussion more effective regarding their findings and confidence?
5. Why is it important to highlight limitations in both the discussion and conclusion sections?

📘 Lecture 21 — Writing Your Conclusion

📖 Overview: This lecture focuses on how researchers structure their Discussion/Conclusion sections in academic papers. It outlines the essential components that should appear in this final section and provides a critical framework for evaluating whether these conclusions are logically sound and appropriately applied. Understanding this material is crucial for both writing effective research papers and critically evaluating published studies.

🗂️ Topics Covered

The lecture first explains the varying formats researchers use for their concluding sections and details the six essential components that should be included: overview of the study, overview of findings, relation of findings, attention to limitations, possible applications, and future possibilities. It then presents seven critical questions that consumers of research should ask when evaluating Discussion/Conclusion sections, including concerns about logical reasoning, design consistency, generalization, causation, theoretical grounding, limitations, and practical significance.

📝 Lecture Summary

Writing Your Conclusion

Researchers vary in the format they use to wrap up their studies. Some only have a Discussion section, whereas others have both Discussion and Conclusion sections. You might also see additional subheadings, such as Summary and/or Implications. Some attach their Discussion section to their Results section, labeled something like "Results and Discussion" followed by a final Conclusion. Regardless of the format, they usually include the following components:

Overview of the study: The purpose of the study is restated, the questions under investigation are summarized, and any propounded hypotheses are reiterated.

Overview of the findings: The researcher should show how the findings address the research question and/or support or fail to support any hypothesis being proposed.

Relation of findings: The researcher should relate the findings of his or her study to previous research findings and theoretical thinking.

Attention to limitations: The researcher should evaluate his or her own study and point out any weaknesses and/or limitations regarding the study.

Possible applications: The researcher should suggest in his or her conclusions how the results can be applied to practical situations.

Future possibilities: The researcher should suggest topics for future research.

💡 Why this matters: These six components form a checklist for both writing and evaluating the concluding section of any research paper. Missing any component weakens the study's contribution.

Questions Every Consumer Should Ask

When evaluating the Discussion/Conclusion section of a study, there is a set of seven questions that the consumer should address:

1. Do the findings logically answer the research questions or support the research hypothesis? The consumer must be wary because many researchers have biases and would love to find answers to their questions or support their hypothesis from the results. Because this final section gives researchers the right to conjecture about what the findings mean, it is easy to unintentionally suggest things that the results do not support.

🔑 Definition — Biases: Personal preferences or preconceptions that may influence a researcher's interpretation of their findings without conscious intent.

2. Does the nature of the study remain consistent from beginning to end? Some studies begin as exploratory studies but end up as confirmatory ones. In such cases, the introduction section has research questions with no specific hypothesis stated, but in the Discussion section, the researcher claims "our hypothesis is confirmed." Another variation is that researchers generate a hypothesis in the Discussion section—which is their right—but then suggest their results support it. This is circular reasoning: we cannot use the same data to support a hypothesis from which it has been formulated. A new study must be conducted to test this hypothesis.

🔑 Definition — Circular reasoning: A logical fallacy where the conclusion of an argument is used as a premise to support itself; in research, this occurs when the same data is used both to generate and confirm a hypothesis.

3. Are the findings generalized to the correct population or situations? Most studies cannot be generalized to a broadly defined population because most samples are not randomly selected, nor are they typically large enough to adequately represent a target population. Results of such studies are suggestive at most and need to be followed up with replications. If the same findings are repeated using different samples from the target population, we can have more assurance we are on the right track. A well-written Discussion section will warn readers of this problem.

🔑 Definition — Replications: Repeating a study with different samples from the same target population to verify whether the original findings hold true.

4. Are the conclusions consistent with the type of research design used? The main concern is whether causation is being inferred from research designs that are not geared to demonstrate this effect. Non-experimental designs such as descriptive or correlational ones cannot be used to directly show causation. Yet, especially in the latter case, some researchers have slipped into concluding that their findings indicate that one variable influences another. When researchers apply their findings, they are often tempted to recommend manipulating one variable to cause changes in another—a logical error unless the research design warrants this application.

🔑 Definition — Logical error: An error in reasoning where conclusions are drawn that are not supported by the research design, such as inferring causation from correlational data.

5. Are the findings and conclusions related to theory or previous research? A well-written Discussion/Conclusion section should attempt to tie the findings and interpretations to any current theoretical thinking or previous research. This might be done by showing how the findings support what has gone before or by providing evidence to refute some theory or challenge previous research.

6. Are any limitations of the study made clear? There are very few, if any, perfect studies in the literature. Regardless of how good a study is, a conscientious researcher will mention what the limitations are to caution the reader from being overly confident about the results.

7. Is there consistency between the findings and the applications? Small correlations, such as r = 0.30, are often interpreted as important findings because they are statistically significant, or a difference of 5 points between a treatment and a control group is given importance for the same reason. Yet the consumer needs to consider whether these findings are large enough to justify the cost in time, human resources, and finance. The consumer needs to be on alert when a researcher advocates costly changes based solely on statistical significance.

📐 Formula: Statistical significance ≠ Practical significance → A finding can be statistically significant but too small in magnitude to justify the cost of implementation.

📌 Example: A researcher finds a correlation of r = 0.30 between a new teaching method and student achievement, and this is statistically significant. However, implementing this method costs $50,000 per school. Despite statistical significance, the practical benefit may be too small to warrant such expense.

⭐ Key Takeaways

The Discussion/Conclusion section must include six essential components: overview of the study, overview of findings, relation to previous research, limitations, applications, and future possibilities. When evaluating this section, consumers must ask seven critical questions to avoid being misled by biased interpretations, circular reasoning, or overgeneralization. A crucial distinction exists between statistical significance and practical significance—not all statistically significant results justify costly implementation. The research design must match the conclusions drawn; non-experimental designs cannot support causal claims. Finally, well-written conclusions will acknowledge limitations and connect findings to existing theory or research, rather than claiming support for hypotheses generated from the same data.

🧠 Quick Revision Questions

1. What are the six essential components that should appear in a Discussion/Conclusion section?
2. What is circular reasoning in the context of research conclusions, and why is it problematic?
3. Why can't correlational studies be used to draw causal conclusions?
4. Explain the difference between statistical significance and practical significance using the example from the lecture.
5. How can a consumer determine whether a study's findings are appropriately generalized to a target population?

📘 Lecture 22 — Constructing a Research Literature Review

📖 Overview: This lecture explains why conducting a literature review is essential before undertaking any research study. It demonstrates how an integrated review can reveal patterns, resolve conflicting findings, and guide researchers toward feasible methodologies. The lecture also provides practical strategies for searching databases, narrowing keywords, and accessing primary research articles.

🗂️ Topics Covered

The lecture begins by explaining the central purpose of a literature review: integrating multiple studies to reveal answers, identify conflicting results, and assess external validity. It then discusses how reviews help researchers understand common methodologies and feasibility constraints. The second half focuses on the practical search process: using preliminary sources like ERIC, selecting effective keywords, setting time limits, and obtaining full articles through libraries or electronic databases.

📝 Lecture Summary

WHY DO A REVIEW OF RESEARCH?

The main benefit of a literature review is to provide a mosaic of what is happening concerning a given topic. No single research study exhausts all knowledge on a topic. By integrating recent research articles, you can discover whether plausible answers to your questions exist. When weaving studies together, you may find answers for practical use or discover conflicting results between studies. On careful scrutiny, you might realize that differences in samples, procedures, or materials produced the differing results. You must then decide which study best corresponds to your particular research question. The closer the correspondence, the more applicable the findings might be.

If the same results are replicated over a variety of studies, you can have more confidence that you are on the right track. Here, external validity comes into play: regardless of sample, procedures, materials, or tests, if the same findings keep appearing, you have a workable answer. Without a well-done literature review, you cannot have this assurance.

Occasionally, you may discover little recent research on a particular question. This serves as a warning: perhaps your research question is stated so that your search accessed only a few studies, or you may need to go further back in time. Alternatively, your question may be so novel that little research is available. 💡 Why this matters: Recognizing gaps in research helps you avoid pursuing questions that cannot be answered with current methods.

🔑 Definition — External Validity: The extent to which research findings can be generalized across different populations, settings, procedures, and measurements. 📐 Concept: Replicated findings → greater confidence → stronger external validity. 📌 Example: A student investigating the critical period hypothesis found insufficient research from the past 5 years. The variables used to test this hypothesis are beyond current capabilities to manipulate or measure, so strong conclusions about children's language learning abilities compared with adults cannot yet be made.

Second, doing a research review is important if you plan to do a study yourself. Such a review provides an overview of different methodologies, instruments for collecting data, and ways to analyze data commonly used in a given area. This knowledge helps you decide whether your proposed study is feasible given your time, material, and financial constraints. Many fledgling researchers could have saved themselves needless angst if they had realized the study required more time and resources than were available before launching into it.

SEARCHING FOR STUDIES

The first place to begin is searching for studies using preliminary sources. These are used to find documents that report research studies or theoretical positions. Most university libraries in the United States and Europe, as well as some public libraries, have computerized search capabilities. With the Internet available in most countries, you should be able to obtain a list of research studies pertinent to your questions from your home computer. Your search is as good as the keywords (or descriptors) you use. You might try different combinations of these words to obtain sufficient results, or use a thesaurus from the preliminary source to identify related keywords.

Your goal is to access firsthand research studies (i.e., primary studies) that relate to your questions. How many studies you include depends on the nature of your question(s). For an exhaustive literature review, you cover as many studies as possible. However, most people place limitations, such as time constraints and/or only journal articles, to confine their search to studies with certain characteristics.

📌 Example: Figure A.1 illustrates a search using the ERIC database on the Internet. With time limits (1990–2002) and location limits (research journals only), starting with the broad keyword ESL yielded 1,200 documents. Narrowing to ESL and writing resulted in 336 references. Narrowing further to ESL and writing strategies captured 37 articles. Adding learning strategies reduced the search to 11 articles.

🔑 Definition — Preliminary Sources: Databases or indexes used to locate documents that report original research studies or theoretical positions (e.g., ERIC, library catalogs).

As for how far back to search, the recommendation is to begin by looking at the last 5 years of research. This usually provides enough current research to address your question(s). Start with the most recent research and work backward in time. This keeps you abreast of the most recent issues and findings, saving time by avoiding outdated issues with which people in the discipline are no longer concerned.

OBTAINING ARTICLES

Once you have identified the studies you want to include, you face the challenge of getting the actual articles. Ideally, you will be near a good library that carries the journals. If the library does not subscribe, it may have a library loan agreement with other libraries. Some journals such as Language Learning and Modern Language Journal have electronic versions to which your library might have access, allowing you to download full articles. If all else fails, you can order journal articles through databases such as ERIC, and they will mail them either through your library or to your address. Ordering the microfiche version is recommended to keep costs down and save trees—though you will need a microfiche reader, which should be available at your library.

⭐ Key Takeaways

A literature review integrates multiple studies into a meaningful mosaic, revealing practical answers, resolving conflicting findings through careful scrutiny of differences in samples and procedures, and strengthening confidence when results are replicated across varied contexts—this demonstrates external validity. Literature reviews also help researchers understand common methodologies and assess feasibility before starting their own study. When searching, use preliminary sources like ERIC with well-chosen keywords, narrowing from broad to specific terms, and focusing on the most recent 5 years of research working backward. If little research exists, consider that your question may need refining or that current methods cannot adequately test the variables. Finally, access full articles through library subscriptions, electronic journals, interlibrary loans, or database ordering services like ERIC microfiche.

🧠 Quick Revision Questions

1. What are the two main reasons the lecture gives for doing a literature review?
2. How should a researcher resolve conflicting results found when reviewing studies on the same topic?
3. What role does external validity play in interpreting replicated findings across multiple studies?
4. Describe the step-by-step process demonstrated in the ERIC database example, starting from the broad keyword "ESL" down to the final 11 articles.
5. What is the recommended time frame for a literature search, and why should you start with the most recent studies?