12.3 Effect size interpretation and practical significance

Effect size interpretation and practical significance are crucial aspects of experimental design. They help researchers understand the magnitude and real-world impact of their findings beyond statistical significance.

Standardized effect size measures like Cohen's d and eta squared quantify the strength of relationships between variables. Practical significance measures, such as clinical significance and number needed to treat, assess the real-world relevance of research outcomes.

Standardized Effect Size Measures

Measures of Standardized Mean Differences

Top images from around the web for Measures of Standardized Mean Differences

• 

  Frontiers | The Meaningfulness of Effect Sizes in Psychological Research: Differences Between ... View original

  Is this image relevant?

• 

  Standardized mean differences cause funnel plot distortion in publication bias assessments | eLife View original

  Is this image relevant?

• 

  Frontiers | The Meaningfulness of Effect Sizes in Psychological Research: Differences Between ... View original

  Is this image relevant?

• 

  Standardized mean differences cause funnel plot distortion in publication bias assessments | eLife View original

  Is this image relevant?

1 of 2

Top images from around the web for Measures of Standardized Mean Differences

• 

  Frontiers | The Meaningfulness of Effect Sizes in Psychological Research: Differences Between ... View original

  Is this image relevant?

• 

  Standardized mean differences cause funnel plot distortion in publication bias assessments | eLife View original

  Is this image relevant?

• 

  Frontiers | The Meaningfulness of Effect Sizes in Psychological Research: Differences Between ... View original

  Is this image relevant?

• 

  Standardized mean differences cause funnel plot distortion in publication bias assessments | eLife View original

  Is this image relevant?

1 of 2

• Cohen's d expresses the difference between two means in standard deviation units
  • Calculated as the difference between two means divided by the pooled standard deviation
  • Commonly used benchmarks: 0.2 (small effect), 0.5 (medium effect), 0.8 (large effect)
• Eta squared ($\eta^2$) represents the proportion of variance in the dependent variable explained by the independent variable
  • Ranges from 0 to 1, with higher values indicating a stronger effect
  • Calculated as the ratio of the between-groups sum of squares to the total sum of squares

Measures of Association

• Pearson's r is a correlation coefficient that measures the strength and direction of a linear relationship between two continuous variables
  • Ranges from -1 (perfect negative correlation) to +1 (perfect positive correlation), with 0 indicating no correlation
  • Squared value ($r^2$) represents the proportion of variance in one variable explained by the other variable
• Odds ratio (OR) compares the odds of an event occurring in one group to the odds of it occurring in another group
  • An OR of 1 indicates no difference between groups, while values greater than 1 suggest higher odds in the first group compared to the second group
  • Commonly used in case-control studies and logistic regression analyses

Measures of Risk

• Relative risk (RR) compares the risk of an event in an exposed group to the risk in an unexposed group
  • An RR of 1 indicates no difference in risk between groups, while values greater than 1 suggest a higher risk in the exposed group
  • Often used in cohort studies and clinical trials to assess the impact of a risk factor or treatment on an outcome

Practical Significance Measures

Clinical Significance

• Number needed to treat (NNT) represents the average number of patients that need to be treated for one additional patient to benefit compared to a control
  • Lower NNT values indicate a more effective treatment
  • Calculated as the reciprocal of the absolute risk reduction (1/ARR)
• Clinical significance refers to the practical or real-world impact of a treatment effect on patient outcomes
  • Considers factors such as the magnitude of the effect, the severity of the condition, and the risks and costs associated with the treatment
  • Determined by clinicians and experts in the field based on their experience and judgment

Practical vs. Statistical Significance

• Practical significance assesses whether the observed effect is large enough to be meaningful or important in a real-world context
  • Focuses on the magnitude and relevance of the effect rather than just its statistical significance
  • A statistically significant result may not always be practically significant if the effect size is small or the outcome is not clinically relevant
• Statistical significance indicates the likelihood that the observed effect is due to chance alone
  • Determined by the p-value, which represents the probability of obtaining the observed results if the null hypothesis is true
  • A statistically significant result (p < 0.05) suggests that the observed effect is unlikely to be due to chance, but does not necessarily imply practical significance