13.3 Limit theorems for discrete distributions

Limit theorems are the backbone of understanding how random variables behave as sample sizes grow. They show us that even with unpredictable individual outcomes, patterns emerge when we look at the big picture.

These theorems help us make sense of real-world data. From predicting election outcomes to estimating financial risks, they give us tools to work with uncertainty and make informed decisions based on large-scale trends.

Limit Theorems

Fundamental Limit Laws

Top images from around the web for Fundamental Limit Laws

• 

  probability theory - Strong Law of Large Numbers (Klenke's proof) - Mathematics Stack Exchange View original

  Is this image relevant?

• 

  Clarifying a Step in Central Limit Theorem Derivation - Mathematics Stack Exchange View original

  Is this image relevant?

• 

  probability theory - Strong Law of Large Numbers (Klenke's proof) - Mathematics Stack Exchange View original

  Is this image relevant?

• 

  Clarifying a Step in Central Limit Theorem Derivation - Mathematics Stack Exchange View original

  Is this image relevant?

1 of 2

Top images from around the web for Fundamental Limit Laws

• 

  probability theory - Strong Law of Large Numbers (Klenke's proof) - Mathematics Stack Exchange View original

  Is this image relevant?

• 

  Clarifying a Step in Central Limit Theorem Derivation - Mathematics Stack Exchange View original

  Is this image relevant?

• 

  probability theory - Strong Law of Large Numbers (Klenke's proof) - Mathematics Stack Exchange View original

  Is this image relevant?

• 

  Clarifying a Step in Central Limit Theorem Derivation - Mathematics Stack Exchange View original

  Is this image relevant?

1 of 2

• Law of large numbers describes behavior of sample averages as sample size increases
  • Weak law states sample mean converges in probability to expected value
  • Strong law states sample mean converges almost surely to expected value
  • Applies to independent, identically distributed random variables
• Central limit theorem establishes convergence of standardized sums to normal distribution
  • For large samples, distribution of sample mean approximates normal distribution
  • Applies even when underlying distribution is not normal
  • Requires finite mean and variance
• Local limit theorem provides more precise approximation for probabilities of specific values
  • Refines central limit theorem for discrete distributions
  • Approximates probability mass function rather than cumulative distribution function
  • Useful for estimating probabilities of rare events

Advanced Limit Concepts

• Large deviation theory studies probabilities of rare events in limit distributions
  • Focuses on tail probabilities that decay exponentially
  • Cramer's theorem provides exponential bounds for sums of independent random variables
  • Applications in risk analysis, queueing theory, and statistical physics
• Berry-Esseen theorem quantifies rate of convergence in central limit theorem
  • Provides upper bound on difference between cumulative distribution functions
  • Depends on third absolute moment of random variables
  • Useful for assessing accuracy of normal approximation for finite samples

Convergence Concepts

Types of Convergence

• Convergence in distribution (weak convergence) occurs when cumulative distribution functions converge
  • Denoted by $X_n \xrightarrow{d} X$ as $n \to \infty$
  • Equivalent to convergence of characteristic functions
  • Does not imply convergence of moments or other properties
• Convergence in probability measures likelihood of small differences between random variables
  • Denoted by $X_n \xrightarrow{P} X$ as $n \to \infty$
  • For any $\epsilon > 0$, $P(|X_n - X| > \epsilon) \to 0$ as $n \to \infty$
  • Stronger than convergence in distribution
• Almost sure convergence (strong convergence) requires convergence with probability 1
  • Denoted by $X_n \xrightarrow{a.s.} X$ as $n \to \infty$
  • Implies $P(\lim_{n \to \infty} X_n = X) = 1$
  • Strongest form of convergence among these three types

Relationships and Applications

• Hierarchy of convergence types: almost sure $\Rightarrow$ in probability $\Rightarrow$ in distribution
• Slutsky's theorem combines convergence results for sums and products of random variables
  • If $X_n \xrightarrow{d} X$ and $Y_n \xrightarrow{P} c$, then $X_n + Y_n \xrightarrow{d} X + c$ and $X_n Y_n \xrightarrow{d} cX$
  • Useful for deriving limit distributions of transformed random variables
• Continuous mapping theorem extends convergence to continuous functions of random variables
  • If $X_n \xrightarrow{d} X$ and $g$ is continuous, then $g(X_n) \xrightarrow{d} g(X)$
  • Applies to various types of convergence (in distribution, probability, almost sure)

Approximations

Poisson Approximation Techniques

• Poisson approximation estimates probabilities for rare events in large samples
  • Applies to sum of many independent, rare events
  • Approximates binomial distribution when $n$ is large and $p$ is small
  • Probability mass function given by $P(X = k) = \frac{e^{-\lambda}\lambda^k}{k!}$, where $\lambda = np$
• Law of small numbers justifies Poisson approximation for certain limit processes
  • As $n \to \infty$ and $p \to 0$ with $np \to \lambda$, binomial distribution converges to Poisson
  • Useful in modeling rare events (radioactive decay, website traffic spikes)
• Le Cam's theorem provides bounds on the accuracy of Poisson approximation
  • Total variation distance between binomial and Poisson distributions bounded by $2(1-e^{-\lambda})p$
  • Helps assess when Poisson approximation is appropriate

Other Discrete Approximations

• Normal approximation to binomial distribution improves for large $n$
  • Uses continuity correction for better accuracy
  • Applies when $np$ and $n(1-p)$ are both greater than 5
• Geometric approximation for negative binomial distribution
  • Useful when number of successes $r$ is large
  • Approximates waiting time until $r$th success
• Stirling's approximation for factorials in large discrete distributions
  • $n! \approx \sqrt{2\pi n} (\frac{n}{e})^n$
  • Improves accuracy of calculations involving large factorials (binomial coefficients)