📊Experimental Design Unit 1 – Introduction to Experimental Design

Key Concepts and Terminology

• Experimental design involves planning and conducting experiments to test hypotheses and draw conclusions
• Independent variable (IV) manipulated by the researcher to observe its effect on the dependent variable (DV)
• Dependent variable (DV) measured or observed to determine the effect of the independent variable
• Control group does not receive the treatment or intervention, serves as a baseline for comparison
• Experimental group receives the treatment or intervention being tested
• Confounding variables extraneous factors that can influence the dependent variable and affect the validity of the results
  • Researchers must identify and control for confounding variables to ensure the observed effects are due to the independent variable
• Randomization assigning participants to groups randomly to minimize bias and ensure that any differences between groups are due to chance
• Blinding concealing information about group assignment from participants, researchers, or both to reduce bias

Principles of Experimental Design

• Randomization assigns participants to groups randomly to minimize bias and ensure that any differences between groups are due to chance
• Replication repeating the experiment multiple times or with different participants to increase the reliability and generalizability of the results
• Blocking divides participants into homogeneous subgroups based on a known confounding variable to reduce its impact on the results
• Balancing ensures that the groups are as similar as possible in terms of participant characteristics and other relevant factors
• Control identifies and manages extraneous variables that could affect the dependent variable to isolate the effect of the independent variable
• Manipulation varies the levels of the independent variable systematically to observe its effect on the dependent variable
• Measurement uses reliable and valid tools to assess the dependent variable accurately and consistently
• Generalization considers the external validity of the results and the extent to which they can be applied to other populations or settings

Types of Experimental Designs

• Between-subjects design compares different groups of participants, each exposed to a different level of the independent variable
• Within-subjects design exposes the same group of participants to all levels of the independent variable, with each participant serving as their own control
• Factorial design manipulates two or more independent variables simultaneously to examine their individual and combined effects on the dependent variable
• Repeated measures design measures the dependent variable multiple times for each participant, allowing for the analysis of changes over time or under different conditions
• Quasi-experimental design lacks random assignment of participants to groups, but still manipulates the independent variable to observe its effect on the dependent variable
  • Quasi-experimental designs are often used when random assignment is not feasible or ethical
• Pretest-posttest design measures the dependent variable before and after the manipulation of the independent variable to assess the effect of the intervention
• Solomon four-group design combines elements of between-subjects and pretest-posttest designs to control for the potential effects of pretesting on the results

Variables and Their Roles

• Independent variable (IV) manipulated by the researcher to observe its effect on the dependent variable
  • The IV is the presumed cause or predictor variable in the experiment
• Dependent variable (DV) measured or observed to determine the effect of the independent variable
  • The DV is the presumed effect or outcome variable in the experiment
• Confounding variables extraneous factors that can influence the dependent variable and affect the validity of the results
• Moderating variables affect the strength or direction of the relationship between the independent and dependent variables
  • Moderating variables can help explain individual differences in response to the independent variable
• Mediating variables explain the mechanism or process through which the independent variable affects the dependent variable
  • Mediating variables are intermediate variables that transmit the effect of the independent variable to the dependent variable
• Control variables held constant or manipulated systematically to isolate the effect of the independent variable on the dependent variable
• Extraneous variables unintended factors that can influence the dependent variable and threaten the internal validity of the experiment
  • Researchers must identify and control for extraneous variables to minimize their impact on the results

Sampling Techniques

• Simple random sampling selects participants from the population at random, giving each individual an equal chance of being chosen
• Stratified random sampling divides the population into homogeneous subgroups (strata) and then randomly selects participants from each stratum
  • Stratified random sampling ensures that the sample is representative of the population in terms of the stratifying variable (age, gender)
• Cluster sampling divides the population into naturally occurring groups (clusters) and then randomly selects entire clusters for the sample
  • Cluster sampling is often used when a complete list of individuals in the population is not available or when it is more convenient to sample groups rather than individuals
• Systematic sampling selects participants from the population at regular intervals (every 10th person on a list) after a random starting point
• Convenience sampling selects participants who are easily accessible or willing to participate in the study
  • Convenience sampling is often used in pilot studies or when resources are limited, but it can limit the generalizability of the results
• Purposive sampling selects participants based on specific characteristics or criteria relevant to the research question
  • Purposive sampling is often used in qualitative research or when the researcher wants to focus on a particular subgroup of the population
• Quota sampling selects participants based on predetermined quotas for specific characteristics (50% male, 50% female) to ensure that the sample is representative of the population

Data Collection Methods

• Surveys collect data through questionnaires or interviews, either in person, by phone, or online
  • Surveys are often used to gather information about attitudes, opinions, behaviors, or characteristics of a population
• Observations involve systematically watching and recording behavior or events in a natural or controlled setting
  • Observations can be structured (using a predetermined coding scheme) or unstructured (allowing for more flexibility and exploration)
• Experiments manipulate one or more independent variables and measure the effect on the dependent variable while controlling for other factors
  • Experiments are often used to establish cause-and-effect relationships between variables
• Interviews involve asking participants open-ended or closed-ended questions to gather in-depth information about their experiences, opinions, or knowledge
  • Interviews can be structured (following a predetermined script), semi-structured (allowing for some flexibility), or unstructured (allowing for more exploration and probing)
• Focus groups bring together a small group of participants to discuss a specific topic or issue, guided by a moderator
  • Focus groups are often used to gather qualitative data and explore group dynamics or collective opinions
• Archival research involves analyzing existing data or records (historical documents, public records) to answer research questions
  • Archival research is often used when collecting new data is not feasible or when the research question involves past events or trends
• Case studies involve in-depth analysis of a single individual, group, event, or phenomenon to provide a detailed understanding of the subject
  • Case studies are often used in fields such as psychology, sociology, or education to explore complex or rare phenomena

Statistical Analysis Basics

• Descriptive statistics summarize and describe the main features of a dataset, such as measures of central tendency (mean, median, mode) and measures of variability (range, standard deviation)
  • Descriptive statistics provide a concise and informative summary of the data, but do not allow for generalizations beyond the sample
• Inferential statistics use sample data to make inferences or predictions about the larger population from which the sample was drawn
  • Inferential statistics involve hypothesis testing and estimation of population parameters based on sample statistics
• Hypothesis testing involves formulating a null hypothesis ($H_0$) and an alternative hypothesis ($H_1$) and using statistical tests to determine whether to reject or fail to reject the null hypothesis based on the sample data
  • The null hypothesis typically states that there is no significant difference or relationship between variables, while the alternative hypothesis states that there is a significant difference or relationship
• $p$-value represents the probability of obtaining the observed results (or more extreme results) if the null hypothesis is true
  • A small $p$-value (typically $p < 0.05$) indicates strong evidence against the null hypothesis, leading to its rejection in favor of the alternative hypothesis
• Effect size measures the magnitude or strength of the relationship between variables or the difference between groups
  • Effect size is important for interpreting the practical significance of the results, beyond just statistical significance
• Confidence intervals provide a range of plausible values for a population parameter (mean, proportion) based on the sample data and a specified level of confidence (95%)
  • Confidence intervals indicate the precision of the estimate and the uncertainty associated with generalizing from the sample to the population
• Statistical power refers to the probability of detecting a true effect or difference if it exists in the population
  • Statistical power depends on factors such as sample size, effect size, and significance level ($\alpha$), and it is important for designing studies that have a high chance of detecting meaningful effects

Ethical Considerations in Experiments

• Informed consent ensures that participants are fully informed about the purpose, procedures, risks, and benefits of the study and that they voluntarily agree to participate
  • Informed consent is a fundamental principle of research ethics and is required for most studies involving human participants
• Confidentiality protects participants' privacy by ensuring that their personal information and data are kept secure and not disclosed to unauthorized individuals
  • Researchers must take appropriate measures to safeguard participants' confidentiality, such as using codes instead of names and storing data in secure locations
• Anonymity goes beyond confidentiality by ensuring that participants' identities cannot be linked to their data, even by the researchers
  • Anonymity is often used in sensitive or controversial research topics to protect participants from potential harm or repercussions
• Debriefing involves providing participants with information about the true purpose and nature of the study after their participation is complete
  • Debriefing is important for studies that involve deception or withholding information from participants, as it allows them to understand the reasons for the deception and to ask questions or express concerns
• Risk minimization involves designing studies in a way that minimizes potential harm or discomfort to participants, both physical and psychological
  • Researchers must carefully consider the risks and benefits of their study and take steps to minimize risks, such as providing appropriate support services or allowing participants to withdraw from the study at any time
• Vulnerable populations (children, prisoners, individuals with mental illness) require special considerations and protections in research due to their increased risk of exploitation or coercion
  • Researchers must obtain appropriate permissions (parental consent) and take extra precautions to ensure that vulnerable participants are not unduly influenced or harmed by their participation
• Scientific integrity involves conducting research in an honest, objective, and transparent manner, free from bias, fraud, or misconduct
  • Researchers must adhere to high standards of scientific integrity, such as properly citing sources, accurately reporting results, and disclosing conflicts of interest
• Institutional review boards (IRBs) are committees that review and approve research proposals to ensure that they meet ethical standards and protect the rights and welfare of human participants
  • IRBs play a crucial role in overseeing research ethics and ensuring that studies are conducted in a responsible and ethical manner