📊Intro to Business Analytics Unit 4 – Statistical Inference: CI & Hypothesis Tests

Key Concepts

• Statistical inference draws conclusions about a population based on a sample of data
• Confidence intervals estimate a range of plausible values for an unknown population parameter
• Hypothesis testing evaluates claims or assumptions about a population using sample data
• Type I error (false positive) occurs when rejecting a true null hypothesis
• Type II error (false negative) occurs when failing to reject a false null hypothesis
• Statistical significance indicates the likelihood of observing results as extreme as the sample data, assuming the null hypothesis is true
• P-value represents the probability of observing a test statistic as extreme as the one calculated, given that the null hypothesis is true
• Statistical power is the probability of correctly rejecting a false null hypothesis

Types of Statistical Inference

• Estimation involves using sample data to estimate unknown population parameters
  • Point estimation provides a single value estimate of a population parameter (sample mean)
  • Interval estimation provides a range of plausible values for a population parameter (confidence interval)
• Hypothesis testing involves using sample data to test claims or assumptions about a population
  • Null hypothesis ($H_0$) represents the default or status quo assumption
  • Alternative hypothesis ($H_a$ or $H_1$) represents the claim or assumption being tested
• Bayesian inference updates prior beliefs about a population parameter based on observed data to obtain a posterior distribution
• Decision theory involves making optimal decisions under uncertainty based on the consequences of different actions

Confidence Intervals Explained

• A confidence interval is a range of values that is likely to contain the true population parameter with a specified level of confidence
• The confidence level (e.g., 95%) represents the proportion of intervals that would contain the true population parameter if the sampling process were repeated many times
• The margin of error determines the width of the confidence interval and depends on the sample size, variability, and desired confidence level
• Larger sample sizes and lower variability lead to narrower confidence intervals and more precise estimates
• The central limit theorem justifies the use of normal distribution-based confidence intervals for means when the sample size is sufficiently large or the population is normally distributed
• Confidence intervals for proportions use the normal approximation when the sample size is large and the population proportion is not close to 0 or 1
• Interpreting a confidence interval involves understanding that the interval provides a range of plausible values for the population parameter, not a probability statement about the parameter itself

Hypothesis Testing Basics

• The null hypothesis ($H_0$) represents the default or status quo assumption, while the alternative hypothesis ($H_a$ or $H_1$) represents the claim or assumption being tested
• The significance level ($\alpha$) is the probability of rejecting the null hypothesis when it is true (Type I error)
• The test statistic is a value calculated from the sample data that measures the difference between the observed data and what is expected under the null hypothesis
• The p-value is the probability of observing a test statistic as extreme as the one calculated, assuming the null hypothesis is true
• Rejecting the null hypothesis occurs when the p-value is less than the chosen significance level ($\alpha$)
• Failing to reject the null hypothesis occurs when the p-value is greater than the chosen significance level ($\alpha$)
• The decision rule specifies the conditions under which the null hypothesis is rejected based on the test statistic and critical value(s)
• One-tailed tests are used when the alternative hypothesis specifies a direction (greater than or less than), while two-tailed tests are used when the alternative hypothesis does not specify a direction (not equal to)

Common Statistical Tests

• Z-test compares a sample mean to a known population mean when the population standard deviation is known and the sample size is large or the population is normally distributed
• T-test compares a sample mean to a known population mean when the population standard deviation is unknown and the sample size is small
• Paired t-test compares the means of two related samples or repeated measures on the same individuals
• Chi-square goodness-of-fit test determines whether an observed frequency distribution differs from a theoretical distribution
• Chi-square test of independence assesses whether two categorical variables are associated or independent
• Analysis of Variance (ANOVA) compares the means of three or more groups to determine if they are significantly different
• Regression analysis examines the relationship between a dependent variable and one or more independent variables

Interpreting Results

• A statistically significant result indicates that the observed data are unlikely to have occurred by chance alone, assuming the null hypothesis is true
• The p-value represents the strength of evidence against the null hypothesis; smaller p-values provide stronger evidence
• Failing to reject the null hypothesis does not prove that the null hypothesis is true; it only suggests that there is insufficient evidence to reject it
• Practical significance considers the magnitude and importance of the observed effect, beyond statistical significance
• Confidence intervals provide a range of plausible values for the population parameter and can be used to assess the precision and practical significance of the results
• Effect size measures the magnitude of the difference between groups or the strength of the relationship between variables
• Statistical power is the probability of detecting a true effect when it exists; higher power increases the likelihood of correctly rejecting a false null hypothesis

Real-World Applications

• A/B testing in marketing compares the effectiveness of two versions of a website or advertisement to determine which performs better
• Quality control in manufacturing uses hypothesis testing to ensure that products meet specified standards and to identify sources of variation
• Clinical trials in medicine use statistical inference to evaluate the safety and efficacy of new treatments compared to existing treatments or placebos
• Polling and surveys use confidence intervals to estimate population characteristics (voter preferences) based on a sample of respondents
• Econometrics applies statistical methods to economic data to test economic theories and predict future trends
• Environmental studies use statistical inference to assess the impact of human activities on natural systems and to evaluate the effectiveness of conservation efforts

Common Pitfalls and Mistakes

• Misinterpreting the p-value as the probability that the null hypothesis is true or false, rather than the probability of observing the data given that the null hypothesis is true
• Confusing statistical significance with practical significance and overemphasizing small but statistically significant differences
• Failing to consider the assumptions underlying statistical tests and using inappropriate tests for the data and research question
• Overgeneralizing results from a sample to a population without considering the representativeness of the sample and potential sources of bias
• Multiple testing and the increased likelihood of Type I errors when conducting many tests on the same data without adjusting the significance level
• Confusing correlation with causation and inferring a causal relationship between variables based on an observed association
• Inadequate sample size and the risk of Type II errors (false negatives) when the sample is too small to detect a true effect
• Misinterpreting confidence intervals as probability statements about the population parameter rather than a range of plausible values based on the sample data