📈Theoretical Statistics Unit 11 – Sampling theory

Key Concepts and Definitions

• Population refers to the entire group of individuals, objects, or events of interest in a study
• Sample is a subset of the population selected for analysis and inference about the population
• Sampling frame is a list or database that represents the entire population from which a sample is drawn
• Sampling unit is the basic unit of the population that is sampled, such as individuals, households, or organizations
• Parameter is a numerical characteristic of a population, such as the mean or proportion
  • Parameters are usually unknown and estimated using sample statistics
• Statistic is a numerical characteristic of a sample, such as the sample mean or sample proportion, used to estimate population parameters
• Representativeness is the degree to which a sample accurately reflects the characteristics of the population
• Randomization involves selecting sample units from the population using a chance mechanism to ensure representativeness and minimize bias

Types of Sampling Methods

• Probability sampling uses random selection methods to ensure each unit in the population has a known, non-zero probability of being selected
  • Allows for statistical inference and generalization to the population
  • Examples include simple random sampling, stratified sampling, and cluster sampling
• Non-probability sampling does not use random selection methods and relies on the researcher's judgment or convenience
  • Does not allow for statistical inference or generalization to the population
  • Examples include convenience sampling, snowball sampling, and quota sampling
• Mixed methods sampling combines probability and non-probability sampling techniques to balance representativeness and feasibility
• Adaptive sampling adjusts the sampling design based on information gathered during the sampling process to focus on areas of interest or rare populations
• Network sampling selects individuals based on their social connections or relationships within a network structure
• Spatial sampling selects units based on their geographic location or spatial distribution

Probability Sampling Techniques

• Simple random sampling (SRS) selects a fixed number of units from the population, with each unit having an equal probability of being chosen
  • Requires a complete sampling frame and can be inefficient for large or geographically dispersed populations
• Systematic sampling selects units at regular intervals from a randomly ordered sampling frame
  • More efficient than SRS but can introduce bias if the ordering is related to the variable of interest
• Stratified sampling divides the population into homogeneous subgroups (strata) based on a relevant characteristic and selects a random sample from each stratum
  • Ensures representation of important subgroups and can increase precision compared to SRS
• Cluster sampling divides the population into naturally occurring groups (clusters), randomly selects a subset of clusters, and samples all units within the selected clusters
  • Useful when a complete sampling frame is not available or when the population is geographically dispersed
  • Requires larger sample sizes than SRS to achieve the same level of precision
• Multistage sampling combines different probability sampling techniques in multiple stages, such as stratifying and then clustering
• Probability proportional to size (PPS) sampling selects units with probabilities proportional to a measure of their size, ensuring that larger units have a higher chance of being selected

Non-Probability Sampling Techniques

• Convenience sampling selects units that are easily accessible or readily available to the researcher
  • Prone to selection bias and not representative of the population
• Purposive sampling selects units based on the researcher's judgment and specific criteria relevant to the study objectives
  • Useful for exploratory research or when focusing on a particular subgroup
• Snowball sampling recruits initial participants who then refer other participants from their social networks
  • Useful for hard-to-reach or hidden populations but can result in a biased sample
• Quota sampling sets quotas for different subgroups in the population and selects units until the quotas are filled
  • Ensures representation of important subgroups but does not use random selection within the quotas
• Self-selection sampling allows individuals to voluntarily participate in the study, often in response to an open invitation
  • Prone to self-selection bias and attracting participants with strong opinions or interest in the topic
• Expert sampling selects individuals with specialized knowledge or expertise in the subject matter
  • Provides valuable insights but may not be representative of the broader population

Sample Size Determination

• Sample size is the number of units selected from the population for the study
• Adequate sample size is crucial for obtaining precise estimates, detecting meaningful effects, and ensuring the representativeness of the sample
• Factors influencing sample size include the desired level of precision, confidence level, variability of the population, and expected response rate
• Larger sample sizes generally lead to more precise estimates and greater statistical power but also increase the cost and time required for the study
• Sample size calculation formulas depend on the type of variable (continuous or categorical), the sampling method, and the study design
  • For example, the formula for a simple random sample with a continuous variable is: $n = \frac{z^2 \sigma^2}{E^2}$, where $n$ is the sample size, $z$ is the critical value for the desired confidence level, $\sigma$ is the population standard deviation, and $E$ is the margin of error
• Finite population correction factor adjusts the sample size formula when the population is relatively small compared to the sample size
• Stratified and clustered sampling designs require additional considerations for sample size allocation across strata or clusters

Sampling Distributions

• Sampling distribution is the probability distribution of a sample statistic over all possible samples of a given size from a population
  • Describes the variability and expected value of the sample statistic
• Central Limit Theorem states that the sampling distribution of the mean approaches a normal distribution as the sample size increases, regardless of the shape of the population distribution
  • Allows for the use of normal-based inferential methods when the sample size is sufficiently large (typically $n \geq 30$)
• Standard error is the standard deviation of the sampling distribution and measures the variability of the sample statistic
  • Smaller standard errors indicate more precise estimates and are influenced by the sample size and population variability
• Sampling distribution of the proportion follows a normal distribution for large sample sizes and when the population proportion is not close to 0 or 1
• Finite population correction factor adjusts the standard error formula when the population is relatively small compared to the sample size
• Bootstrap sampling involves resampling with replacement from the original sample to create multiple bootstrap samples and estimate the sampling distribution empirically

Estimation Theory

• Point estimation provides a single value as an estimate of a population parameter based on a sample statistic
  • Examples include the sample mean, sample proportion, and sample regression coefficients
• Interval estimation provides a range of plausible values for a population parameter with a specified level of confidence
  • Confidence interval is a common form of interval estimation and is constructed using the point estimate and its standard error
• Properties of good estimators include unbiasedness (the expected value of the estimator equals the true parameter value), efficiency (minimum variance among unbiased estimators), and consistency (the estimator converges to the true parameter value as the sample size increases)
• Maximum likelihood estimation (MLE) is a method for estimating parameters by maximizing the likelihood function, which quantifies the probability of observing the sample data given the parameter values
• Method of moments estimation (MME) equates sample moments (e.g., mean, variance) to their population counterparts and solves for the parameter estimates
• Bayesian estimation incorporates prior information about the parameters and updates the estimates based on the observed data using Bayes' theorem

Bias and Error in Sampling

• Sampling bias occurs when the sample systematically differs from the population due to the sampling method or non-response
  • Examples include selection bias, non-response bias, and voluntary response bias
• Non-sampling error arises from sources other than the sampling process, such as measurement error, processing error, and coverage error
• Selection bias occurs when the sampling method favors certain units over others, resulting in a non-representative sample
  • Can be addressed through proper randomization and ensuring that all units have a known, non-zero probability of being selected
• Non-response bias occurs when the characteristics of respondents differ systematically from those of non-respondents
  • Can be mitigated through follow-up efforts, incentives, and weighting adjustments based on known characteristics of non-respondents
• Measurement error arises from inaccurate or inconsistent data collection methods, such as poorly designed questionnaires or interviewer bias
  • Can be reduced through careful questionnaire design, standardized data collection procedures, and interviewer training
• Processing error occurs during the data entry, coding, or cleaning stages and can introduce inaccuracies in the final dataset
  • Can be minimized through data validation checks, double entry, and thorough documentation of data processing steps
• Coverage error occurs when the sampling frame does not accurately represent the target population, leading to undercoverage or overcoverage
  • Can be addressed by using multiple sampling frames, post-stratification weighting, or adjusting the estimates based on known population characteristics

Applications in Real-World Statistics

• Market research uses sampling to gather information about consumer preferences, brand awareness, and product satisfaction
  • Helps businesses make informed decisions about product development, pricing, and marketing strategies
• Public opinion polls employ sampling techniques to gauge public sentiment on various social, political, and economic issues
  • Influences policy decisions, election campaigns, and media coverage
• Quality control in manufacturing relies on sampling to monitor the quality of products and identify defective items
  • Helps ensure that products meet specified standards and reduces the need for expensive 100% inspection
• Epidemiological studies use sampling to estimate the prevalence and incidence of diseases in a population and identify risk factors
  • Informs public health interventions, resource allocation, and disease prevention strategies
• Environmental monitoring employs sampling to assess the quality of air, water, and soil and detect the presence of pollutants
  • Guides environmental policy, regulation, and remediation efforts
• Educational research uses sampling to evaluate the effectiveness of teaching methods, assess student performance, and identify achievement gaps
  • Informs educational policy, curriculum development, and resource allocation decisions
• Sampling is crucial in auditing and financial analysis to detect fraud, errors, and irregularities in financial statements
  • Helps ensure the integrity and reliability of financial reporting and protects stakeholders' interests