📊Probability and Statistics Unit 9 – Confidence Intervals & Hypothesis Testing

Key Concepts

• Confidence intervals estimate a population parameter based on sample data
• Hypothesis testing evaluates claims or assumptions about a population parameter
• Null hypothesis ($H_0$) represents the default or assumed claim about a population parameter
• Alternative hypothesis ($H_a$) represents the claim that contradicts the null hypothesis
• Type I error (false positive) occurs when rejecting a true null hypothesis
• Type II error (false negative) occurs when failing to reject a false null hypothesis
• Significance level ($\alpha$) is the probability of making a Type I error
  • Commonly used significance levels include 0.01, 0.05, and 0.10
• Power of a test is the probability of correctly rejecting a false null hypothesis

Confidence Intervals Explained

• A confidence interval is a range of values that likely contains the true population parameter
• Confidence level (e.g., 95%) represents the probability that the interval contains the true parameter
• Factors affecting the width of a confidence interval include sample size, variability, and confidence level
  • Larger sample sizes lead to narrower intervals
  • Higher variability in the data leads to wider intervals
  • Higher confidence levels lead to wider intervals
• Formula for a confidence interval: $\text{point estimate} \pm \text{margin of error}$
• Margin of error is calculated using the standard error and a critical value from the appropriate distribution (e.g., z, t)
• Interpreting a confidence interval involves understanding the range of plausible values for the population parameter
• Confidence intervals can be one-sided (upper or lower bound) or two-sided (both bounds)

Types of Hypothesis Tests

• One-sample tests compare a sample statistic to a hypothesized population parameter (e.g., one-sample t-test, z-test)
• Two-sample tests compare statistics from two independent samples (e.g., two-sample t-test, z-test)
• Paired tests compare two related samples or repeated measures (e.g., paired t-test)
• ANOVA (Analysis of Variance) tests compare means across three or more groups or factors
• Chi-square tests assess the relationship between categorical variables (e.g., goodness-of-fit, independence)
• Non-parametric tests are used when assumptions of parametric tests are violated (e.g., Mann-Whitney U, Wilcoxon signed-rank)
  • These tests often rely on ranks or medians rather than means

Steps in Hypothesis Testing

1. State the null and alternative hypotheses
2. Choose the appropriate test statistic and distribution
3. Determine the significance level ($\alpha$)
4. Calculate the test statistic using sample data
5. Find the p-value associated with the test statistic
   • P-value represents the probability of observing a test statistic as extreme as the one calculated, assuming the null hypothesis is true
6. Compare the p-value to the significance level
   • If p-value $\leq \alpha$, reject the null hypothesis
   • If p-value $> \alpha$, fail to reject the null hypothesis
7. Interpret the results in the context of the problem

Interpreting Results

• Rejecting the null hypothesis suggests that there is sufficient evidence to support the alternative hypothesis
• Failing to reject the null hypothesis does not prove it is true, but rather that there is insufficient evidence to support the alternative
• Confidence intervals provide a range of plausible values for the population parameter
  • If the hypothesized value falls outside the interval, it suggests rejecting the null hypothesis
• Effect size measures the magnitude of the difference or relationship between variables (e.g., Cohen's d, correlation coefficient)
• Statistical significance does not always imply practical significance
  • Large sample sizes can lead to statistically significant results even for small effects
• Interpreting results should consider the context, limitations, and potential implications of the study

Common Mistakes to Avoid

• Misinterpreting p-values as the probability that the null hypothesis is true
  • P-values represent the probability of observing the data, assuming the null hypothesis is true
• Confusing statistical significance with practical significance
• Failing to check assumptions of the chosen test (e.g., normality, homogeneity of variance)
• Interpreting non-significant results as evidence of no effect or relationship
• Multiple testing without adjusting the significance level (e.g., Bonferroni correction)
• Overgeneralizing results beyond the scope of the study or population
• Confounding variables that may influence the relationship between the variables of interest
• Misinterpreting confidence intervals as containing the true parameter with certainty

Real-World Applications

• Quality control in manufacturing processes (e.g., testing product defect rates)
• Clinical trials for new medications or treatments (e.g., comparing efficacy to a placebo)
• Market research and consumer preferences (e.g., testing product appeal)
• Psychological research (e.g., comparing treatment outcomes, assessing group differences)
• Environmental studies (e.g., testing pollution levels, species abundance)
• Political polls and surveys (e.g., estimating support for candidates or policies)
  • Confidence intervals help quantify the margin of error in poll results
• A/B testing in web design and online marketing (e.g., comparing click-through rates)

Practice Problems

1. A researcher wants to estimate the average height of students at a university. They take a random sample of 100 students and find a mean height of 68 inches with a standard deviation of 4 inches. Construct a 95% confidence interval for the population mean height.

2. A company claims that their new battery has an average life of more than 500 hours. A random sample of 50 batteries has a mean life of 490 hours with a standard deviation of 60 hours. Test the company's claim at the 0.05 significance level.

3. A psychologist believes that a new therapy can reduce anxiety levels. They measure anxiety scores before and after the therapy for 30 patients. The mean difference (before - after) is 10 points with a standard deviation of 15 points. Test the effectiveness of the therapy at the 0.01 significance level.

4. A manufacturer wants to compare the strength of two alloys. Random samples of 50 units from each alloy are tested, resulting in means of 200 MPa and 210 MPa, with standard deviations of 20 MPa and 25 MPa, respectively. Test for a significant difference in strength at the 0.05 level.

5. A biologist wants to compare the average weights of three different species of fish. They collect random samples of 30 fish from each species and record their weights. Use ANOVA to test for significant differences among the species at the 0.05 level.