🧰Engineering Applications of Statistics Unit 2 – Probability Distributions

Key Concepts and Definitions

• Probability distribution a mathematical function describing the likelihood of different outcomes in a random experiment or process
• Random variable a variable whose value is determined by the outcome of a random event and can be discrete (countable) or continuous (uncountable)
• Probability density function (PDF) defines the probability of a continuous random variable falling within a particular range of values
  • Represented by the function $f(x)$ where the probability of $X$ falling between $a$ and $b$ is given by $P(a \leq X \leq b) = \int_a^b f(x) dx$
• Cumulative distribution function (CDF) gives the probability that a random variable $X$ takes a value less than or equal to a specific value $x$
  • Denoted by $F(x) = P(X \leq x)$ and is the integral of the PDF from $-\infty$ to $x$
• Expected value (mean) the average value of a random variable over a large number of trials, denoted by $E(X)$ or $\mu$
• Variance a measure of the spread or dispersion of a random variable around its mean, denoted by $Var(X)$ or $\sigma^2$
  • Calculated as $Var(X) = E[(X - \mu)^2]$
• Standard deviation the square root of the variance, denoted by $\sigma$, and provides a measure of the average distance between the values of a random variable and its mean

Types of Probability Distributions

• Discrete probability distributions describe random variables that can only take on a finite or countably infinite number of distinct values (integers, whole numbers)
  • Examples include the binomial, Poisson, and geometric distributions
• Continuous probability distributions describe random variables that can take on any value within a specified range or interval (real numbers)
  • Examples include the normal (Gaussian), exponential, and uniform distributions
• Joint probability distributions describe the probability of two or more random variables occurring simultaneously
  • Can be discrete, continuous, or a combination of both
• Marginal probability distributions derived from joint distributions by summing or integrating over the values of one or more random variables
• Conditional probability distributions describe the probability of one random variable given the value or range of values of another random variable
• Multivariate probability distributions describe the joint behavior of multiple random variables, often used in machine learning and data analysis

Properties of Distributions

• Non-negativity the probability density function (PDF) or probability mass function (PMF) must be non-negative for all values of the random variable
• Normalization the total probability of all possible outcomes must equal 1, i.e., $\int_{-\infty}^{\infty} f(x) dx = 1$ for continuous distributions and $\sum_{x} P(X = x) = 1$ for discrete distributions
• Symmetry a distribution is symmetric if its PDF or PMF is mirror-symmetric about a central value (mean)
  • Example: the standard normal distribution is symmetric about its mean of 0
• Skewness a measure of the asymmetry of a distribution, with positive skewness indicating a longer right tail and negative skewness indicating a longer left tail
• Kurtosis a measure of the "tailedness" of a distribution, with higher kurtosis indicating more outliers and heavier tails compared to a normal distribution
• Moments quantitative measures that describe the shape and properties of a distribution, such as the mean (first moment), variance (second central moment), skewness (third standardized moment), and kurtosis (fourth standardized moment)
• Central limit theorem states that the sum or average of a large number of independent and identically distributed random variables will converge to a normal distribution, regardless of the underlying distribution of the individual variables

Common Probability Distributions

• Normal (Gaussian) distribution a continuous probability distribution characterized by its bell-shaped curve, often used to model real-world phenomena such as heights, weights, and errors in measurements
  • PDF: $f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$, where $\mu$ is the mean and $\sigma$ is the standard deviation
• Binomial distribution a discrete probability distribution that describes the number of successes in a fixed number of independent trials, each with the same probability of success
  • PMF: $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$, where $n$ is the number of trials, $k$ is the number of successes, and $p$ is the probability of success in each trial
• Poisson distribution a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time or space, assuming a constant average rate and independent occurrences
  • PMF: $P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$, where $\lambda$ is the average number of events per interval
• Exponential distribution a continuous probability distribution that models the time between events in a Poisson process, often used to describe waiting times, failure rates, and radioactive decay
  • PDF: $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$, where $\lambda$ is the rate parameter
• Uniform distribution a continuous probability distribution where all values within a given range are equally likely
  • PDF: $f(x) = \frac{1}{b-a}$ for $a \leq x \leq b$, where $a$ and $b$ are the minimum and maximum values of the range
• Beta distribution a continuous probability distribution defined on the interval [0, 1], often used to model probabilities, proportions, and percentages
  • PDF: $f(x) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha, \beta)}$ for $0 \leq x \leq 1$, where $\alpha$ and $\beta$ are shape parameters and $B(\alpha, \beta)$ is the beta function
• Gamma distribution a continuous probability distribution used to model waiting times, time to failure, and other positive, continuous random variables
  • PDF: $f(x) = \frac{1}{\Gamma(\alpha)\theta^\alpha}x^{\alpha-1}e^{-\frac{x}{\theta}}$ for $x \geq 0$, where $\alpha$ is the shape parameter, $\theta$ is the scale parameter, and $\Gamma(\alpha)$ is the gamma function

Calculating Probabilities

• For discrete distributions, probabilities are calculated by summing the probability mass function (PMF) values for the desired outcomes
  • Example: $P(X = k)$ for a specific value $k$ or $P(a \leq X \leq b)$ for a range of values
• For continuous distributions, probabilities are calculated by integrating the probability density function (PDF) over the desired range of values
  • Example: $P(a \leq X \leq b) = \int_a^b f(x) dx$
• Cumulative distribution function (CDF) can be used to calculate probabilities for both discrete and continuous random variables
  • $P(X \leq x) = F(x)$, where $F(x)$ is the CDF evaluated at $x$
• Complement rule can be used to find the probability of an event not occurring
  • $P(X > x) = 1 - P(X \leq x)$
• For joint distributions, probabilities are calculated by summing or integrating the joint PMF or PDF over the desired ranges of the random variables involved
• Conditional probabilities are calculated using the definition $P(A|B) = \frac{P(A \cap B)}{P(B)}$, where $A$ and $B$ are events and $P(B) \neq 0$
• Independence of events can simplify probability calculations, as $P(A \cap B) = P(A)P(B)$ for independent events $A$ and $B$
• Software tools and libraries (MATLAB, Python's SciPy and NumPy, R) can be used to calculate probabilities for various distributions

Parameter Estimation

• Method of moments estimators are obtained by equating sample moments (mean, variance, etc.) to their theoretical counterparts and solving for the distribution parameters
  • Example: for a normal distribution, $\hat{\mu} = \bar{x}$ and $\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n (x_i - \bar{x})^2$
• Maximum likelihood estimation (MLE) involves finding the parameter values that maximize the likelihood function, which is the joint probability density of the observed data treated as a function of the parameters
  • Log-likelihood function $\ell(\theta) = \log L(\theta)$ is often used for computational convenience
• Bayesian estimation incorporates prior knowledge about the parameters in the form of a prior distribution and updates this knowledge using observed data to obtain a posterior distribution
  • Posterior distribution: $p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)}$, where $p(\theta)$ is the prior, $p(x|\theta)$ is the likelihood, and $p(x)$ is the marginal likelihood (normalizing constant)
• Confidence intervals provide a range of plausible values for a parameter with a specified level of confidence (e.g., 95%)
  • For a normal distribution with unknown mean and known variance, a 95% confidence interval for the mean is $\bar{x} \pm 1.96\frac{\sigma}{\sqrt{n}}$
• Hypothesis testing involves making decisions about population parameters based on sample data
  • Null hypothesis ($H_0$) represents the status quo or default assumption, while the alternative hypothesis ($H_a$) represents the claim or research question
  • Type I error (false positive) occurs when rejecting a true null hypothesis, while Type II error (false negative) occurs when failing to reject a false null hypothesis

Applications in Engineering

• Quality control and process monitoring using control charts based on the normal, binomial, and Poisson distributions to detect anomalies and ensure product consistency
• Reliability engineering and failure analysis using the exponential, Weibull, and lognormal distributions to model time to failure, estimate reliability metrics, and optimize maintenance strategies
• Queuing theory and network analysis using the Poisson and exponential distributions to model arrival processes, service times, and system performance in manufacturing, telecommunications, and transportation systems
• Measurement and instrumentation using the normal distribution to quantify uncertainty, establish tolerance intervals, and determine the required sample size for accurate estimation
• Signal processing and communication systems using the Gaussian, Rayleigh, and Rice distributions to model noise, fading, and interference in wireless channels and to design optimal receivers and detectors
• Structural reliability and risk assessment using the normal, lognormal, and extreme value distributions to model loads, material properties, and environmental factors and to estimate failure probabilities and safety margins
• Stochastic modeling and simulation using various probability distributions to represent input variables, propagate uncertainty, and analyze the performance of complex engineering systems (Monte Carlo methods, sensitivity analysis)
• Machine learning and data analysis using probability distributions to model data, estimate parameters, assess goodness-of-fit, and make predictions or decisions based on statistical inference techniques

Practice Problems and Examples

• A machine produces bolts with lengths that follow a normal distribution with a mean of 10 cm and a standard deviation of 0.5 cm. What is the probability that a randomly selected bolt has a length between 9.5 cm and 10.5 cm?
• The number of defects in a 100 m^2 sheet of metal follows a Poisson distribution with an average of 2 defects per sheet. Calculate the probability of finding exactly 3 defects in a randomly selected sheet.
• The time between arrivals of customers at a service counter follows an exponential distribution with a mean of 5 minutes. What is the probability that the time between two consecutive arrivals is less than 3 minutes?
• A company produces light bulbs with lifetimes that follow a Weibull distribution with shape parameter $\beta = 2$ and scale parameter $\theta = 1000$ hours. Estimate the probability that a randomly selected light bulb will last more than 800 hours.
• The weights of apples in a harvest follow a gamma distribution with shape parameter $\alpha = 3$ and scale parameter $\theta = 0.5$. Find the mean and variance of the apple weights.
• A quality control inspector randomly selects 50 items from a production line and finds that 3 of them are defective. Assuming a binomial distribution, estimate the 95% confidence interval for the true proportion of defective items produced by the line.
• The breaking strength of a type of steel cable follows a normal distribution. A sample of 20 cables has a mean breaking strength of 10,000 N and a standard deviation of 500 N. Test the hypothesis that the true mean breaking strength of the cables is greater than 9,800 N at a significance level of 0.05.
• A company wants to estimate the average time its customers spend on its website. A random sample of 100 customers has a mean time of 12 minutes with a standard deviation of 3 minutes. Construct a 99% confidence interval for the true mean time spent on the website.