6.1 Least squares and maximum likelihood estimation

Mathematical biology uses estimation techniques to understand complex biological systems. Least squares and maximum likelihood estimation are two key methods for fitting models to data and inferring parameters.

These techniques have different strengths and applications in biology. Least squares is simpler and works well for linear models, while maximum likelihood is more flexible and can handle various probability distributions.

Estimation Techniques in Mathematical Biology

Principles of least squares estimation

Top images from around the web for Principles of least squares estimation

• 

  Frontiers | The (Mathematical) Modeling Process in Biosciences View original

  Is this image relevant?

• 

  Regression Modelling View original

  Is this image relevant?

• 

  Linear Regression (2 of 4) | Concepts in Statistics View original

  Is this image relevant?

• 

  Frontiers | The (Mathematical) Modeling Process in Biosciences View original

  Is this image relevant?

• 

  Regression Modelling View original

  Is this image relevant?

1 of 3

Top images from around the web for Principles of least squares estimation

• 

  Frontiers | The (Mathematical) Modeling Process in Biosciences View original

  Is this image relevant?

• 

  Regression Modelling View original

  Is this image relevant?

• 

  Linear Regression (2 of 4) | Concepts in Statistics View original

  Is this image relevant?

• 

  Frontiers | The (Mathematical) Modeling Process in Biosciences View original

  Is this image relevant?

• 

  Regression Modelling View original

  Is this image relevant?

1 of 3

• Least squares estimation minimizes sum of squared differences between observed and predicted values
• Objective function: $S = \sum_{i=1}^n (y_i - f(x_i, \beta))^2$ where $y_i$ are observed values, $f(x_i, \beta)$ are predicted values, and $\beta$ are model parameters
• Applied in linear regression, nonlinear curve fitting, and model calibration (population growth models)
• Assumes normally distributed errors and homoscedasticity (constant error variance)
• Computationally efficient and provides unbiased estimates under certain conditions
• Sensitive to outliers and may not be optimal for non-Gaussian error distributions (skewed data)

Application of maximum likelihood estimation

• Maximum likelihood estimation (MLE) finds parameter values maximizing likelihood of observing given data
• Likelihood function: $L(\theta|x) = P(x|\theta)$ where $\theta$ are model parameters and $x$ is observed data
• MLE process:
  1. Define data probability distribution
  2. Construct likelihood function
  3. Take logarithm of likelihood function
  4. Find maximum by setting derivatives to zero
• Used in population genetics (allele frequency estimation), phylogenetic tree reconstruction, and epidemiological models (disease transmission rates)
• Asymptotically efficient and consistent estimator
• Computationally intensive for complex models and requires knowledge of underlying probability distribution

Least squares vs maximum likelihood

• Both estimate model parameters for linear and nonlinear models
• Least squares assumes normally distributed errors, MLE accommodates various distributions
• MLE more efficient for large samples, least squares more robust for small samples
• Least squares simpler to implement, MLE more computationally demanding
• MLE flexible in handling different data types, least squares primarily for continuous data
• Least squares provides easily interpretable fit measures (R-squared), MLE offers likelihood-based model comparison (AIC, BIC)

Implementation of estimation algorithms

• Programming languages: Python, R, MATLAB
• Libraries: NumPy, SciPy (numerical computations), Statsmodels (statistical modeling), Scikit-learn (machine learning)
• Least squares algorithms:
  • Ordinary least squares (OLS) for linear models
  • Nonlinear least squares (NLS) for nonlinear models
  • Iterative methods (Gauss-Newton, Levenberg-Marquardt)
• MLE algorithms:
  • Newton-Raphson method
  • Expectation-Maximization (EM) algorithm
  • Gradient descent and variants
• Optimization techniques: Conjugate gradient method, Quasi-Newton methods (BFGS)
• Model diagnostics: Residual analysis, goodness-of-fit tests, cross-validation techniques (k-fold)