🎣Statistical Inference Unit 1 – Statistical Inference: Foundations & Probability

Key Concepts and Definitions

• Statistical inference draws conclusions about a population based on a sample of data
• Probability quantifies the likelihood of an event occurring and forms the foundation for statistical inference
• Random variables assign numerical values to outcomes of a random process (discrete or continuous)
• Probability distributions describe the probabilities of different outcomes for a random variable
  • Discrete distributions include binomial, Poisson, and hypergeometric
  • Continuous distributions include normal, uniform, and exponential
• Sampling is the process of selecting a subset of individuals from a population to estimate characteristics of the whole population
• Sample statistics (mean, variance, proportion) are used to estimate population parameters
• The central limit theorem states that the sampling distribution of the mean approaches a normal distribution as the sample size increases, regardless of the shape of the population distribution

Probability Fundamentals

• Probability is a measure of the likelihood that an event will occur, expressed as a number between 0 and 1
• The probability of an event A is denoted as P(A)
• The complement of an event A, denoted as A', is the event "not A" and P(A') = 1 - P(A)
• Two events are mutually exclusive if they cannot occur at the same time, and their probabilities add up to 1
• Independent events do not influence each other, and the probability of both events occurring is the product of their individual probabilities
• Conditional probability is the probability of an event occurring given that another event has already occurred, denoted as P(A|B)
• Bayes' theorem describes the probability of an event based on prior knowledge of conditions related to the event: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

Random Variables and Distributions

• A random variable is a variable whose value is determined by the outcome of a random event
• Discrete random variables have countable outcomes (number of defective items in a batch)
• Continuous random variables have an infinite number of possible outcomes within a range (height of students in a class)
• The probability mass function (PMF) gives the probability of each value for a discrete random variable
• The probability density function (PDF) describes the likelihood of a continuous random variable falling within a particular range of values
• The cumulative distribution function (CDF) gives the probability that a random variable is less than or equal to a specific value
• The expected value (mean) of a random variable is the sum of each possible outcome multiplied by its probability
• The variance and standard deviation measure the dispersion of a random variable around its expected value

Sampling and Sample Statistics

• Sampling is the process of selecting a subset of individuals from a population to estimate characteristics of the entire population
• Simple random sampling ensures each member of the population has an equal chance of being selected
• Stratified sampling divides the population into subgroups (strata) and then randomly samples from each subgroup
• Cluster sampling divides the population into clusters, randomly selects clusters, and then samples all individuals within selected clusters
• Systematic sampling selects individuals at regular intervals from the sampling frame
• Sample statistics are used to estimate population parameters
  • Sample mean ($\bar{x}$) estimates population mean ($\mu$)
  • Sample variance ($s^2$) and standard deviation ($s$) estimate population variance ($\sigma^2$) and standard deviation ($\sigma$)
  • Sample proportion ($\hat{p}$) estimates population proportion ($p$)

Estimation Techniques

• Point estimation provides a single value as an estimate of a population parameter (sample mean)
• Interval estimation gives a range of values that is likely to contain the population parameter with a certain level of confidence
• Confidence intervals are commonly used for interval estimation
  • A 95% confidence interval means that if the sampling process is repeated many times, 95% of the intervals will contain the true population parameter
• The margin of error is the maximum expected difference between the true population parameter and the sample estimate
• The width of the confidence interval depends on the sample size, variability in the data, and the desired confidence level
• Increasing the sample size or decreasing the desired confidence level will result in a narrower confidence interval
• The t-distribution is used for constructing confidence intervals when the sample size is small or the population standard deviation is unknown

Hypothesis Testing Basics

• Hypothesis testing is a statistical method to determine whether there is enough evidence to support a claim about a population parameter
• The null hypothesis ($H_0$) is the default claim that there is no significant effect or difference
• The alternative hypothesis ($H_a$ or $H_1$) is the claim that there is a significant effect or difference
• The significance level ($\alpha$) is the probability of rejecting the null hypothesis when it is true (Type I error)
• The p-value is the probability of observing a test statistic as extreme as the one calculated, assuming the null hypothesis is true
• If the p-value is less than the significance level, we reject the null hypothesis in favor of the alternative hypothesis
• Type I error (false positive) occurs when the null hypothesis is rejected when it is actually true
• Type II error (false negative) occurs when the null hypothesis is not rejected when it is actually false

Common Statistical Tests

• Z-test compares a sample mean to a known population mean when the population standard deviation is known and the sample size is large
• T-test compares a sample mean to a known population mean when the population standard deviation is unknown or the sample size is small
• Paired t-test compares the means of two related samples or repeated measures on the same individuals
• Chi-square test for goodness of fit determines whether an observed frequency distribution differs from a theoretical distribution
• Chi-square test for independence assesses whether two categorical variables are associated or independent
• Analysis of Variance (ANOVA) tests for differences among three or more group means by comparing variances
• Correlation analysis measures the strength and direction of the linear relationship between two continuous variables
• Regression analysis models the relationship between a dependent variable and one or more independent variables

Practical Applications and Examples

• A pharmaceutical company tests a new drug to determine if it effectively lowers blood pressure compared to a placebo
  • Null hypothesis: The drug has no effect on blood pressure
  • Alternative hypothesis: The drug significantly lowers blood pressure
• A market researcher wants to estimate the proportion of consumers who prefer a new product over an existing one
  • A 95% confidence interval is constructed using the sample proportion to estimate the population proportion
• A psychologist investigates whether there is a significant difference in test anxiety levels between male and female students
  • An independent samples t-test is used to compare the mean anxiety scores of the two groups
• An ecologist studies the relationship between the size of a habitat and the number of species found in that habitat
  • Correlation analysis is used to measure the strength and direction of the relationship between habitat size and species count
• A quality control manager wants to determine if the defect rate of a manufacturing process has increased
  • A chi-square test for goodness of fit compares the observed defect frequencies to the expected frequencies based on historical data