🎲Mathematical Probability Theory Unit 11 – Stochastic Processes

Key Concepts and Definitions

• Stochastic process is a collection of random variables indexed by time or space representing the evolution of a random system
• State space is the set of all possible values that a stochastic process can take at any given time or position
• Sample path (or realization) refers to a single possible outcome or trajectory of a stochastic process over time or space
• Stationarity implies that the statistical properties of a stochastic process do not change over time (time-invariant)
  • Strict stationarity requires the joint probability distribution to be invariant under time shifts
  • Weak stationarity (or covariance stationarity) only requires the mean and covariance to be time-invariant
• Ergodicity is a property where the statistical properties of a stochastic process can be inferred from a single, sufficiently long realization
• Martingale is a stochastic process whose expected value at any future time, given the current state, is equal to its current value
  • Submartingales have expected future values greater than or equal to the current value
  • Supermartingales have expected future values less than or equal to the current value

Types of Stochastic Processes

• Discrete-time processes have random variables indexed by discrete time steps (integers) while continuous-time processes have random variables indexed by a continuous time parameter (real numbers)
• Markov processes are memoryless stochastic processes where the future state depends only on the current state, not on the past states
  • Markov chains are discrete-time Markov processes with a countable state space
  • Continuous-time Markov chains have a countable state space but continuous time parameter
• Gaussian processes are stochastic processes where any finite collection of random variables has a multivariate normal distribution
  • Brownian motion (or Wiener process) is a continuous-time Gaussian process with independent increments
• Poisson processes model the occurrence of rare events in continuous time with a constant average rate
• Renewal processes generalize Poisson processes by allowing the inter-arrival times between events to have any distribution (not necessarily exponential)
• Birth-death processes are continuous-time Markov chains used to model population dynamics with birth and death rates

Probability Spaces and Random Variables

• Probability space $(\Omega, \mathcal{F}, P)$ consists of a sample space $\Omega$ (set of all possible outcomes), a $\sigma$-algebra $\mathcal{F}$ of events (subsets of $\Omega$), and a probability measure $P$ assigning probabilities to events
• Random variable $X$ is a measurable function from the sample space $\Omega$ to the real numbers $\mathbb{R}$, assigning a numerical value to each outcome
  • Discrete random variables take countable values (integers) while continuous random variables take uncountable values (real numbers)
• Probability distribution of a random variable $X$ is a function that assigns probabilities to the possible values or ranges of values that $X$ can take
  • Probability mass function (PMF) for discrete random variables: $P(X = x)$
  • Probability density function (PDF) for continuous random variables: $f_X(x)$ such that $P(a \leq X \leq b) = \int_a^b f_X(x) dx$
• Expected value (or mean) of a random variable $X$ is the average value it takes, denoted by $\mathbb{E}[X]$
  • For discrete $X$: $\mathbb{E}[X] = \sum_x x P(X = x)$
  • For continuous $X$: $\mathbb{E}[X] = \int_{-\infty}^{\infty} x f_X(x) dx$
• Variance of a random variable $X$ measures its spread around the mean, denoted by $\text{Var}(X) = \mathbb{E}[(X - \mathbb{E}[X])^2]$

Markov Chains

• Markov chain is a discrete-time stochastic process with the Markov property: the future state depends only on the current state, not on the past states
  • State space $S$ is the set of possible values the Markov chain can take at each time step (countable)
  • Transition probability $p_{ij}$ is the probability of moving from state $i$ to state $j$ in one time step: $p_{ij} = P(X_{n+1} = j | X_n = i)$
• Transition probability matrix $P = (p_{ij})$ contains all the transition probabilities, with rows summing to 1
• $n$-step transition probability $p_{ij}^{(n)}$ is the probability of moving from state $i$ to state $j$ in $n$ time steps, obtained by taking the $n$-th power of the transition probability matrix: $P^n = (p_{ij}^{(n)})$
• Stationary distribution $\pi = (\pi_1, \pi_2, \ldots)$ is a probability distribution over the states that remains unchanged under the transition probabilities: $\pi P = \pi$
  • If a Markov chain is irreducible (all states communicate) and aperiodic, it has a unique stationary distribution
• Absorbing Markov chains have one or more absorbing states that, once entered, cannot be left
  • Fundamental matrix $N = (I - Q)^{-1}$ gives the expected number of visits to each transient state before absorption, where $Q$ is the submatrix of transient-to-transient transition probabilities

Poisson Processes

• Poisson process is a continuous-time stochastic process that models the occurrence of rare events with a constant average rate $\lambda > 0$
  • Inter-arrival times between events are independent and exponentially distributed with mean $1/\lambda$
  • Number of events in any interval of length $t$ follows a Poisson distribution with mean $\lambda t$: $P(N(t) = k) = \frac{(\lambda t)^k e^{-\lambda t}}{k!}$
• Superposition of independent Poisson processes with rates $\lambda_1, \lambda_2, \ldots, \lambda_n$ is also a Poisson process with rate $\lambda = \sum_{i=1}^n \lambda_i$
• Thinning (or splitting) a Poisson process with rate $\lambda$ into two independent Poisson processes with rates $p\lambda$ and $(1-p)\lambda$, where $0 < p < 1$, is done by assigning each event to the first process with probability $p$ and to the second process with probability $1-p$
• Non-homogeneous Poisson process has a time-varying rate function $\lambda(t)$, with the expected number of events in an interval $[a, b]$ given by $\int_a^b \lambda(t) dt$
• Compound Poisson process associates a random variable (mark) with each event in a Poisson process, representing the event's magnitude or cost

Brownian Motion

• Brownian motion (or Wiener process) is a continuous-time stochastic process $\{B(t), t \geq 0\}$ with the following properties:
  • $B(0) = 0$ (starts at the origin)
  • Independent increments: for any $t_1 < t_2 \leq t_3 < t_4$, $B(t_4) - B(t_3)$ is independent of $B(t_2) - B(t_1)$
  • Stationary increments: for any $s < t$, $B(t) - B(s)$ has a normal distribution with mean 0 and variance $t-s$
  • Sample paths are continuous almost surely (with probability 1)
• Standard Brownian motion has unit variance per unit time, while a general Brownian motion can have any constant variance $\sigma^2$ per unit time
• Brownian bridge is a Brownian motion conditioned to start and end at specified values, often used to model random processes with fixed endpoints
• Geometric Brownian motion is a stochastic process $\{S(t), t \geq 0\}$ where the logarithm of $S(t)$ follows a Brownian motion with drift: $d\log S(t) = \mu dt + \sigma dB(t)$
  • Used to model stock prices in the Black-Scholes option pricing model
• Fractional Brownian motion is a generalization of Brownian motion with correlated increments, characterized by the Hurst parameter $H \in (0, 1)$
  • For $H = 1/2$, it reduces to standard Brownian motion
  • For $H > 1/2$, increments are positively correlated (persistent)
  • For $H < 1/2$, increments are negatively correlated (anti-persistent)

Applications in Real-World Scenarios

• Finance: modeling stock prices, option pricing, portfolio optimization, risk management
  • Geometric Brownian motion for stock price dynamics (Black-Scholes model)
  • Stochastic volatility models (Heston model) for time-varying volatility
  • Lévy processes for modeling jumps and heavy-tailed distributions in asset returns
• Queueing theory: analyzing waiting lines and service systems in operations research
  • M/M/1 queue: Poisson arrivals, exponential service times, single server
  • M/M/c queue: Poisson arrivals, exponential service times, $c$ parallel servers
  • M/G/1 queue: Poisson arrivals, general service time distribution, single server
• Reliability engineering: modeling the failure and repair of complex systems
  • Alternating renewal process for a system with exponential failure and repair times
  • Non-homogeneous Poisson process for modeling time-varying failure rates (bathtub curve)
• Epidemiology: modeling the spread of infectious diseases in a population
  • SIR (Susceptible-Infected-Recovered) model as a continuous-time Markov chain
  • Branching processes for modeling the early stages of an epidemic
• Machine learning: Gaussian processes for regression and classification
  • Kriging (Gaussian process regression) for spatial interpolation and surrogate modeling
  • Gaussian process classification for binary or multi-class classification problems

Advanced Topics and Extensions

• Martingales and their applications in stochastic calculus and financial mathematics
  • Doob's optional stopping theorem for the expected value of a stopped martingale
  • Azuma-Hoeffding inequality for concentration bounds on martingales with bounded differences
• Stochastic differential equations (SDEs) for modeling continuous-time stochastic processes driven by Brownian motion
  • Itô's lemma for computing the differential of a function of a stochastic process
  • Girsanov's theorem for changing the probability measure of an SDE
  • Feynman-Kac formula for solving certain partial differential equations using SDEs
• Lévy processes as generalizations of Brownian motion with jumps
  • Poisson process, compound Poisson process, and gamma process as examples of Lévy processes
  • Lévy-Khintchine formula for the characteristic function of a Lévy process
  • Subordinators as increasing Lévy processes used for time-changing other stochastic processes
• Stochastic calculus for jump processes and its applications
  • Itô's formula for jump processes, extending Itô's lemma to include jump terms
  • Stochastic differential equations driven by Lévy processes
  • Applications in finance (jump-diffusion models) and insurance mathematics (risk processes)
• Measure-valued processes and their applications in population genetics and statistical physics
  • Fleming-Viot process for modeling allele frequencies in a population under genetic drift and mutation
  • Dawson-Watanabe process (or super-process) for modeling branching populations with spatial structure
  • Interacting particle systems (voter model, contact process) for modeling the spread of opinions or infections on a lattice