13.2 Model calibration, validation, and uncertainty analysis

Model calibration is crucial for improving hydrologic simulations. By adjusting parameters and using optimization algorithms, we can minimize differences between simulated and observed values. This process involves selecting objective functions, defining parameter ranges, and evaluating performance metrics.

Validation and uncertainty analysis are essential for assessing model reliability. Validation tests the calibrated model on independent data, while uncertainty analysis quantifies the impact of input and parameter variations. These steps help communicate model limitations and guide future improvements.

Model Calibration

Process of model calibration

Top images from around the web for Process of model calibration

• 

  HESS - On the choice of calibration metrics for “high-flow” estimation using hydrologic models View original

  Is this image relevant?

• 

  HESS - Technical note: Hydrology modelling R packages – a unified analysis of models and ... View original

  Is this image relevant?

• 

  HESS - Technical note: Hydrology modelling R packages – a unified analysis of models and ... View original

  Is this image relevant?

• 

  HESS - On the choice of calibration metrics for “high-flow” estimation using hydrologic models View original

  Is this image relevant?

• 

  HESS - Technical note: Hydrology modelling R packages – a unified analysis of models and ... View original

  Is this image relevant?

1 of 3

Top images from around the web for Process of model calibration

• 

  HESS - On the choice of calibration metrics for “high-flow” estimation using hydrologic models View original

  Is this image relevant?

• 

  HESS - Technical note: Hydrology modelling R packages – a unified analysis of models and ... View original

  Is this image relevant?

• 

  HESS - Technical note: Hydrology modelling R packages – a unified analysis of models and ... View original

  Is this image relevant?

• 

  HESS - On the choice of calibration metrics for “high-flow” estimation using hydrologic models View original

  Is this image relevant?

• 

  HESS - Technical note: Hydrology modelling R packages – a unified analysis of models and ... View original

  Is this image relevant?

1 of 3

• Adjust model parameters to improve agreement between simulated and observed hydrologic variables (streamflow, groundwater levels)
  • Objective functions quantify difference between simulated and observed values
    • Root mean square error (RMSE), Nash-Sutcliffe efficiency (NSE), percent bias (PBIAS)
  • Optimization algorithms search parameter space to find best set of parameters that minimize objective function
    • Gradient-based methods (Levenberg-Marquardt), global optimization methods (genetic algorithms, particle swarm optimization)
  • Performance metrics assess goodness-of-fit between simulated and observed values
    • Coefficient of determination ($R^2$), NSE, RMSE, PBIAS
• Calibration process involves:
  1. Select appropriate objective functions and performance metrics based on study objectives and available data
  2. Define reasonable ranges for model parameters based on prior knowledge or literature values
  3. Run optimization algorithm to find best set of parameters that minimize objective function
  4. Evaluate calibrated model's performance using performance metrics and visual inspection of simulated and observed hydrographs

Model Validation and Uncertainty Analysis

Model validation with datasets

• Assess performance of calibrated model on independent dataset not used in calibration process
  • Split-sample validation: divide available data into calibration and validation periods
  • Proxy-basin validation: apply calibrated model to similar watershed with comparable characteristics (area, land use, climate)
• Use validation metrics similar to calibration process ($R^2$, NSE, RMSE, PBIAS)
• Well-performing model should have similar performance metrics for both calibration and validation periods
• Poor performance during validation may indicate model overfitting, inadequate model structure, or differences in watershed characteristics between calibration and validation periods

Uncertainty quantification in models

• Sensitivity analysis assesses impact of changes in model inputs or parameters on model outputs
  • Local sensitivity analysis: vary one parameter at a time while keeping others constant
  • Global sensitivity analysis: vary multiple parameters simultaneously to assess their relative importance
    • Sobol' indices, Morris screening, Latin hypercube sampling
• Monte Carlo simulation propagates uncertainties in model inputs and parameters through model to obtain distribution of model outputs
  • Randomly sample input and parameter values from respective probability distributions
  • Run model multiple times using sampled values to generate distribution of model outputs
• Bayesian inference combines prior knowledge about model parameters with observed data to estimate posterior parameter distributions
  • Markov Chain Monte Carlo (MCMC) methods sample from posterior distribution to characterize parameter uncertainties
    • Metropolis-Hastings algorithm, Gibbs sampling
• Assess uncertainty in model structure by comparing performance of different model formulations or using multi-model ensembles

Interpretation of uncertainty analysis

• Uncertainty analysis provides range of plausible model outcomes rather than single deterministic prediction
• Consider central tendency (mean, median) and spread (standard deviation, interquartile range) of model output distribution for decision-making
• Construct confidence intervals or prediction intervals to communicate uncertainty associated with model predictions
• Reliability of model predictions depends on quality and quantity of input data, appropriateness of model assumptions, and robustness of calibration and validation process
• Clearly communicate limitations of model (inability to capture certain processes, applicable spatial and temporal scales, potential sources of uncertainty not accounted for)
• Use uncertainty analysis to guide future data collection efforts and model improvements by identifying most influential sources of uncertainty