📊Advanced Quantitative Methods Unit 2 – Probability Theory & Distributions

Key Concepts and Definitions

• Probability the likelihood of an event occurring, expressed as a number between 0 and 1
• Random variable a variable whose value is determined by the outcome of a random event
  • Discrete random variables have a countable number of possible values (number of heads in 10 coin flips)
  • Continuous random variables can take on any value within a specified range (height of a randomly selected person)
• Probability distribution a function that describes the likelihood of different outcomes for a random variable
• Expected value the average value of a random variable over a large number of trials, calculated by multiplying each possible value by its probability and summing the results
• Variance a measure of how much the values of a random variable deviate from the expected value, calculated by taking the average of the squared differences between each value and the mean
• Standard deviation the square root of the variance, used to measure the spread of a distribution
• Central Limit Theorem states that the sum or average of a large number of independent random variables will be approximately normally distributed, regardless of the underlying distribution

Probability Fundamentals

• Probability is always a number between 0 and 1, where 0 represents an impossible event and 1 represents a certain event
• The sum of the probabilities of all possible outcomes for a random event must equal 1
• Independent events the occurrence of one event does not affect the probability of another event (rolling a die multiple times)
• Dependent events the occurrence of one event influences the probability of another event (drawing cards from a deck without replacement)
• Mutually exclusive events cannot occur simultaneously (rolling a 1 and a 6 on a single die roll)
• Conditional probability the probability of an event occurring given that another event has already occurred, calculated using the formula $P(A|B) = \frac{P(A \cap B)}{P(B)}$
• Bayes' Theorem a formula used to calculate the probability of an event based on prior knowledge and new evidence, expressed as $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$

Types of Probability Distributions

• Bernoulli distribution models a single trial with two possible outcomes (success or failure), with a fixed probability of success
• Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials
  • Characterized by two parameters the number of trials (n) and the probability of success (p)
  • Probability mass function $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$
• Poisson distribution models the number of events occurring in a fixed interval of time or space, given a known average rate of occurrence
  • Characterized by a single parameter the average rate of occurrence (λ)
  • Probability mass function $P(X = k) = \frac{e^{-λ} λ^k}{k!}$
• Normal distribution a continuous probability distribution that is symmetric and bell-shaped, with many real-world applications
  • Characterized by two parameters the mean (μ) and the standard deviation (σ)
  • Probability density function $f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x-μ)^2}{2\sigma^2}}$
• Exponential distribution models the time between events in a Poisson process, or the time until a specific event occurs
  • Characterized by a single parameter the rate parameter (λ)
  • Probability density function $f(x) = λe^{-λx}$ for $x \geq 0$
• Uniform distribution a continuous probability distribution where all values within a given range are equally likely
  • Characterized by two parameters the minimum value (a) and the maximum value (b)
  • Probability density function $f(x) = \frac{1}{b-a}$ for $a \leq x \leq b$

Properties of Distributions

• Mean the expected value or average of a probability distribution
  • For discrete distributions, calculated by summing the product of each value and its probability $E(X) = \sum x \cdot P(X = x)$
  • For continuous distributions, calculated by integrating the product of each value and its probability density $E(X) = \int x \cdot f(x) dx$
• Median the middle value of a distribution, such that half of the values are above and half are below
• Mode the most frequently occurring value in a distribution
• Skewness a measure of the asymmetry of a distribution
  • Positive skewness indicates a longer tail on the right side of the distribution
  • Negative skewness indicates a longer tail on the left side of the distribution
• Kurtosis a measure of the heaviness of the tails of a distribution compared to a normal distribution
  • Higher kurtosis indicates heavier tails and a higher probability of extreme values
• Moment-generating function a tool used to calculate the moments of a distribution (mean, variance, skewness, kurtosis)
  • Defined as $M_X(t) = E(e^{tX})$, where $t$ is a real number

Calculating Probabilities

• For discrete distributions, probabilities are calculated by summing the probability mass function over the desired range of values
  • $P(a \leq X \leq b) = \sum_{x=a}^b P(X = x)$
• For continuous distributions, probabilities are calculated by integrating the probability density function over the desired range of values
  • $P(a \leq X \leq b) = \int_a^b f(x) dx$
• Cumulative distribution function (CDF) gives the probability that a random variable is less than or equal to a given value
  • For discrete distributions $F(x) = P(X \leq x) = \sum_{t \leq x} P(X = t)$
  • For continuous distributions $F(x) = P(X \leq x) = \int_{-\infty}^x f(t) dt$
• Inverse CDF used to find the value of a random variable given a specific probability
  • For discrete distributions, find the smallest value $x$ such that $F(x) \geq p$
  • For continuous distributions, solve the equation $F(x) = p$ for $x$
• Standard normal distribution a normal distribution with a mean of 0 and a standard deviation of 1
  • Z-score measures the number of standard deviations a value is from the mean $Z = \frac{X - μ}{σ}$
  • Probabilities for the standard normal distribution can be found using a Z-table or statistical software

Statistical Inference and Hypothesis Testing

• Statistical inference drawing conclusions about a population based on a sample of data
• Point estimate a single value used to estimate a population parameter (sample mean, sample proportion)
• Confidence interval a range of values that is likely to contain the true population parameter with a specified level of confidence
  • Calculated using the point estimate, the standard error, and the desired confidence level
  • For a population mean $\bar{x} \pm z_{\alpha/2} \cdot \frac{σ}{\sqrt{n}}$ (known population standard deviation)
  • For a population mean $\bar{x} \pm t_{\alpha/2} \cdot \frac{s}{\sqrt{n}}$ (unknown population standard deviation)
• Hypothesis testing a statistical method for determining whether there is enough evidence to reject a null hypothesis in favor of an alternative hypothesis
  • Null hypothesis ($H_0$) the default assumption that there is no significant difference or effect
  • Alternative hypothesis ($H_a$ or $H_1$) the claim that there is a significant difference or effect
  • P-value the probability of observing a test statistic as extreme as the one calculated, assuming the null hypothesis is true
  • Significance level ($\alpha$) the threshold for rejecting the null hypothesis, typically set at 0.05
• Type I error rejecting the null hypothesis when it is actually true (false positive)
• Type II error failing to reject the null hypothesis when it is actually false (false negative)

Real-World Applications

• Quality control using probability distributions to model the likelihood of defective products and determine appropriate sampling plans
• Finance modeling stock prices, portfolio returns, and risk management using various probability distributions (normal, lognormal, t-distribution)
• Insurance using probability distributions to calculate premiums based on the likelihood and severity of claims (exponential, Pareto, Weibull)
• Epidemiology modeling the spread of diseases and the effectiveness of interventions using probability distributions (binomial, Poisson, exponential)
• Machine learning using probability distributions to build predictive models and make decisions based on uncertain data (Gaussian mixture models, Bayesian networks)
• Genetics using probability distributions to model the inheritance of traits and the occurrence of mutations (binomial, Poisson, hypergeometric)
• Telecommunications modeling the arrival of data packets and the reliability of networks using probability distributions (Poisson, exponential, Erlang)

Common Pitfalls and Misconceptions

• Confusing probability with certainty assuming that a high probability event will always occur or that a low probability event will never occur
• Misinterpreting conditional probabilities failing to account for the base rate or prior probability when calculating the probability of an event given another event
• Assuming independence assuming that events are independent when they are actually dependent, leading to incorrect probability calculations
• Misunderstanding the Law of Large Numbers believing that a small sample will always be representative of the population or that a streak of one outcome makes the opposite outcome more likely in the future
• Misinterpreting p-values interpreting a p-value as the probability that the null hypothesis is true, rather than the probability of observing the data given that the null hypothesis is true
• Overreliance on assumptions using probability distributions that do not accurately model the real-world situation, leading to faulty conclusions
• Ignoring the impact of sample size failing to consider how the sample size affects the precision of estimates and the power of hypothesis tests
• Misusing the Central Limit Theorem applying the theorem to non-random samples, dependent data, or small sample sizes