19.2 Prior and posterior distributions

Bayesian inference updates our beliefs about parameters using data. It combines prior knowledge with observed evidence to form a posterior distribution, allowing us to make informed decisions in uncertain situations.

The process involves selecting appropriate priors, calculating posteriors using Bayes' theorem, and analyzing the impact of priors. As more data is gathered, the posterior converges towards the true parameter value, regardless of the initial prior.

Bayesian Inference

Prior vs posterior distributions

Top images from around the web for Prior vs posterior distributions

• 

  bayesian - Is it possible to calculate numerically the posterior distribution with a known prior ... View original

  Is this image relevant?

• 

  Frontiers | The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations ... View original

  Is this image relevant?

• 

  Help me understand Bayesian prior and posterior distributions - Cross Validated View original

  Is this image relevant?

• 

  bayesian - Is it possible to calculate numerically the posterior distribution with a known prior ... View original

  Is this image relevant?

• 

  Frontiers | The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations ... View original

  Is this image relevant?

1 of 3

Top images from around the web for Prior vs posterior distributions

• 

  bayesian - Is it possible to calculate numerically the posterior distribution with a known prior ... View original

  Is this image relevant?

• 

  Frontiers | The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations ... View original

  Is this image relevant?

• 

  Help me understand Bayesian prior and posterior distributions - Cross Validated View original

  Is this image relevant?

• 

  bayesian - Is it possible to calculate numerically the posterior distribution with a known prior ... View original

  Is this image relevant?

• 

  Frontiers | The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations ... View original

  Is this image relevant?

1 of 3

• Prior distribution represents initial beliefs or knowledge about a parameter before observing data
  • Denoted as $P(\theta)$, where $\theta$ is the parameter of interest (coin bias, disease prevalence)
  • Based on domain knowledge, previous studies, or expert opinion
• Posterior distribution represents updated beliefs or knowledge about a parameter after observing data
  • Denoted as $P(\theta|X)$, where $X$ is the observed data (coin flips, patient test results)
  • Combines prior distribution and likelihood function using Bayes' theorem

Selection of prior distributions

• Informative priors used when prior knowledge or information about the parameter is available
  • Beta distribution for parameters bounded between 0 and 1 (success probability)
  • Normal distribution for parameters with known mean and variance (average height)
• Non-informative priors used when little or no prior knowledge about the parameter is available
  • Aim to minimize the influence of the prior on the posterior distribution
  • Uniform distribution over the parameter space (any value equally likely)
  • Jeffreys prior, proportional to square root of Fisher information (invariant under reparameterization)

Calculation of posterior distributions

• Bayes' theorem: $P(\theta|X) = \frac{P(X|\theta)P(\theta)}{P(X)}$
  • $P(X|\theta)$ is likelihood function, probability of observing data $X$ given parameter $\theta$
  • $P(X)$ is marginal likelihood, normalizing constant
• Steps to calculate posterior distribution:
  1. Specify prior distribution $P(\theta)$
  2. Determine likelihood function $P(X|\theta)$ based on observed data
  3. Calculate marginal likelihood $P(X)$ by integrating or summing over all possible values of $\theta$
  4. Apply Bayes' theorem to obtain posterior distribution $P(\theta|X)$

Impact of priors on posteriors

• Sensitivity analysis investigates how choice of prior distribution affects posterior distribution
  • Compare posterior distributions obtained using different priors (skeptical vs optimistic)
• Influence of prior distribution
  • Strong prior with narrow distribution or high confidence can heavily influence posterior (expert opinion)
  • Weak prior with wide distribution or low confidence allows data to have more impact on posterior (uninformative)
• Asymptotic behavior
  • As sample size increases, influence of prior diminishes (law of large numbers)
  • Posterior distribution converges to true parameter value, regardless of choice of prior (consistency)