🧰Engineering Applications of Statistics Unit 11 – Time Series Analysis & Forecasting

What's Time Series Analysis?

• Time series analysis involves analyzing data points collected over time to identify patterns, trends, and seasonality
• Focuses on understanding how a variable changes over time and making predictions about future values
• Utilizes historical data to build models that capture the underlying structure and patterns in the data
• Helps identify factors influencing the variable of interest and how they evolve over time
• Enables forecasting future values based on past observations and trends
• Finds applications in various domains such as finance, economics, weather forecasting, and engineering
• Involves techniques like decomposition, smoothing, and modeling to extract meaningful insights from time-dependent data

Key Components of Time Series

• Trend represents the long-term increase or decrease in the data over time (upward or downward movement)
• Seasonality refers to regular, predictable fluctuations that occur within a fixed period (yearly, monthly, or weekly patterns)
• Cyclical patterns are similar to seasonality but occur over longer periods and are less predictable (business cycles or economic cycles)
• Irregular or random fluctuations are unpredictable variations not captured by trend, seasonality, or cyclical components
• Level indicates the average value of the time series over a specific period
• Autocorrelation measures the relationship between a variable's current value and its past values at different lags
• Stationarity is a property where the statistical properties of the time series remain constant over time (mean, variance, and autocorrelation)

Trend, Seasonality, and Noise

• Trend represents the overall long-term direction of the time series (increasing, decreasing, or stable)
  • Can be linear, where the data follows a straight line, or non-linear, exhibiting curves or changing rates
• Seasonality captures regular, repeating patterns within a fixed time period
  • Examples include higher sales during holiday seasons or increased energy consumption during summer months
• Noise refers to the random, irregular fluctuations that are not explained by the trend or seasonality components
  • Noise can be caused by measurement errors, unexpected events, or other factors not captured in the model
• Decomposing a time series into trend, seasonality, and noise helps in understanding the underlying patterns and making accurate forecasts
• Techniques like moving averages, exponential smoothing, and seasonal decomposition can be used to separate these components

Stationarity and Why It Matters

• Stationarity is a crucial assumption in many time series analysis techniques
• A stationary time series has constant mean, variance, and autocorrelation over time
• Non-stationary time series exhibit changing statistical properties, making it challenging to model and forecast accurately
• Stationarity is important because many statistical methods and models assume that the data is stationary
• Non-stationary data can lead to spurious relationships and unreliable forecasts
• Techniques like differencing and transformation can be applied to convert non-stationary data into stationary data
• Tests such as the Augmented Dickey-Fuller (ADF) test and the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test can be used to assess stationarity

Popular Time Series Models

• Autoregressive (AR) models predict future values based on a linear combination of past values
  • AR models assume that the current value depends on a weighted sum of previous values and an error term
• Moving Average (MA) models predict future values based on a linear combination of past forecast errors
  • MA models capture the relationship between an observation and the past forecast errors
• Autoregressive Moving Average (ARMA) models combine both AR and MA components
  • ARMA models consider both the past values and the past forecast errors to make predictions
• Autoregressive Integrated Moving Average (ARIMA) models extend ARMA to handle non-stationary data
  • ARIMA models include differencing to remove trends and make the data stationary before applying ARMA
• Seasonal ARIMA (SARIMA) models incorporate seasonal components into the ARIMA framework
  • SARIMA models capture both non-seasonal and seasonal patterns in the data
• Exponential Smoothing models use weighted averages of past observations to make forecasts
  • Examples include Simple Exponential Smoothing (SES), Holt's Linear Trend, and Holt-Winters' Seasonal Method

Forecasting Techniques

• Naive forecasting assumes that the future values will be the same as the most recent observation
• Moving average forecasting uses the average of a fixed number of past observations to predict future values
• Exponential smoothing assigns exponentially decreasing weights to past observations, giving more importance to recent data
  • Simple Exponential Smoothing (SES) is suitable for data with no clear trend or seasonality
  • Holt's Linear Trend adds a trend component to capture increasing or decreasing patterns
  • Holt-Winters' Seasonal Method incorporates both trend and seasonality components
• ARIMA forecasting involves identifying the appropriate order of differencing, AR terms, and MA terms
  • Box-Jenkins methodology is used for model selection and parameter estimation in ARIMA
• Regression-based forecasting establishes a relationship between the variable of interest and other explanatory variables
  • Linear regression, polynomial regression, and multiple linear regression can be used for forecasting
• Machine learning techniques such as neural networks and support vector machines can also be employed for time series forecasting

Evaluating Forecast Accuracy

• Mean Absolute Error (MAE) measures the average absolute difference between the forecasted and actual values
  • MAE provides an intuitive understanding of the forecast errors in the original units of the data
• Mean Squared Error (MSE) calculates the average squared difference between the forecasted and actual values
  • MSE penalizes larger errors more heavily and is sensitive to outliers
• Root Mean Squared Error (RMSE) is the square root of MSE and is commonly used to measure forecast accuracy
  • RMSE is in the same units as the original data and provides a measure of the typical forecast error
• Mean Absolute Percentage Error (MAPE) expresses the forecast error as a percentage of the actual values
  • MAPE is useful for comparing forecast accuracy across different time series or models
• Theil's U statistic compares the forecast accuracy of a model to that of a naive forecast
  • A value less than 1 indicates that the model outperforms the naive forecast
• Residual analysis involves examining the differences between the forecasted and actual values
  • Residuals should be randomly distributed, uncorrelated, and have constant variance for a good forecast model

Real-World Engineering Applications

• Demand forecasting in supply chain management to optimize inventory levels and production planning
  • Forecasting customer demand helps in efficient resource allocation and reduces stockouts or overstocking
• Predictive maintenance in industrial equipment to anticipate failures and schedule maintenance activities
  • Analyzing sensor data and historical maintenance records to predict when equipment is likely to fail
• Energy load forecasting to balance electricity supply and demand in power grids
  • Forecasting short-term and long-term energy consumption helps in efficient power generation and distribution
• Traffic flow prediction for intelligent transportation systems and urban planning
  • Analyzing historical traffic data to forecast congestion levels and optimize traffic management strategies
• Weather forecasting for agricultural planning, renewable energy generation, and disaster management
  • Predicting temperature, precipitation, and wind patterns to make informed decisions in various domains
• Financial market forecasting to support investment decisions and risk management
  • Analyzing stock prices, exchange rates, and economic indicators to forecast future market trends
• Sales forecasting for businesses to plan marketing strategies, resource allocation, and revenue projections
  • Forecasting product demand, customer behavior, and market trends to make data-driven business decisions