8.5 Linear quadratic regulator (LQR)

Linear quadratic regulator (LQR) is a powerful optimal control technique used in modern control systems. It systematically designs state feedback controllers to optimize system performance while considering control effort and state deviations.

LQR finds applications in aerospace, robotics, and process control. It determines the best control inputs to minimize a quadratic cost function, balancing system state regulation and control effort. LQR assumes linear, time-invariant, fully observable systems.

Overview of linear quadratic regulator (LQR)

• LQR is a powerful optimal control technique widely used in modern control systems to regulate the behavior of dynamic systems and minimize a quadratic cost function
• It provides a systematic approach to designing state feedback controllers that optimize system performance while considering control effort and state deviations
• LQR has found applications in various domains, including aerospace, robotics, process control, and autonomous systems, where it helps achieve desired system behavior and robustness

Definition of LQR

Top images from around the web for Definition of LQR

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

1 of 3

Top images from around the web for Definition of LQR

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

• 

  Optimization of the linear quadratic regulator (LQR) control quarter car suspension system using ... View original

  Is this image relevant?

1 of 3

• LQR is an optimal control method that determines the best control inputs to minimize a quadratic cost function subject to the system dynamics described by a set of linear differential equations
• The quadratic cost function penalizes both the deviations of the system states from their desired values and the control effort required to achieve the desired system behavior
• LQR assumes that the system is linear, time-invariant, and fully observable, meaning that all states can be measured or estimated accurately

Applications of LQR in control systems

• LQR is extensively used in aerospace applications, such as aircraft flight control systems, to stabilize and control the aircraft's attitude, altitude, and trajectory
• In robotics and autonomous systems, LQR is employed for motion planning, trajectory tracking, and stabilization of robotic manipulators and mobile robots
• LQR finds applications in process control industries, such as chemical plants and manufacturing processes, to maintain desired operating conditions and optimize production efficiency
• Other applications include power systems, automotive control, and structural vibration suppression, where LQR helps achieve optimal performance and robustness

Mathematical formulation of LQR

• The mathematical formulation of LQR involves representing the system dynamics in state-space form, defining a quadratic cost function, and formulating the optimal control problem
• The state-space representation captures the evolution of the system states over time, while the quadratic cost function quantifies the performance objectives and control effort
• The optimal control problem seeks to find the control input that minimizes the quadratic cost function subject to the system dynamics and initial conditions

State-space representation

• The state-space representation describes the system dynamics using a set of first-order linear differential equations in the form $\dot{x}(t) = Ax(t) + Bu(t)$, where $x(t)$ is the state vector, $u(t)$ is the control input vector, and $A$ and $B$ are constant matrices
• The state vector $x(t)$ represents the internal variables of the system that fully characterize its behavior at any given time (position, velocity, etc.)
• The control input vector $u(t)$ represents the external signals that can be manipulated to influence the system's behavior (forces, torques, etc.)

Quadratic cost function

• The quadratic cost function in LQR is defined as $J = \int_{0}^{\infty} (x^T(t)Qx(t) + u^T(t)Ru(t)) dt$, where $Q$ and $R$ are positive definite weighting matrices
• The matrix $Q$ penalizes the deviations of the system states from their desired values, while the matrix $R$ penalizes the control effort
• The choice of $Q$ and $R$ matrices allows the designer to balance the trade-off between state regulation and control effort, depending on the specific control objectives

Optimal control problem formulation

• The optimal control problem in LQR aims to find the control input $u(t)$ that minimizes the quadratic cost function $J$ subject to the system dynamics and initial conditions
• The problem can be formulated as a constrained optimization problem, where the goal is to determine the optimal control input that satisfies the system equations and minimizes the cost function
• The solution to the optimal control problem leads to the optimal state feedback control law, which expresses the control input as a linear function of the system states

LQR controller design

• LQR controller design involves solving the optimal control problem to obtain the optimal state feedback gain matrix, which determines the control input based on the current system states
• The design process requires solving the algebraic Riccati equation, a matrix equation that arises from the necessary conditions for optimality
• The resulting LQR controller guarantees closed-loop system stability and exhibits robustness properties against parameter variations and external disturbances

Algebraic Riccati equation

• The algebraic Riccati equation (ARE) is a key component in the LQR design process and is given by $A^TP + PA - PBR^{-1}B^TP + Q = 0$, where $P$ is the symmetric positive definite solution matrix
• Solving the ARE yields the matrix $P$, which is used to compute the optimal state feedback gain matrix $K = R^{-1}B^TP$
• The ARE can be solved using various numerical methods, such as the eigenvector method, the Schur method, or iterative techniques like the Newton-Kleinman algorithm

Optimal state feedback gain

• The optimal state feedback gain matrix $K$ is obtained by solving the ARE and is given by $K = R^{-1}B^TP$
• The optimal control input is then computed as $u(t) = -Kx(t)$, which means that the control input is a linear function of the current system states
• The state feedback gain matrix $K$ determines how the control input should be adjusted based on the deviations of the system states from their desired values

Closed-loop system stability

• The LQR controller guarantees closed-loop system stability, meaning that the system states will converge to their desired values over time when the optimal control input is applied
• The stability of the closed-loop system can be analyzed by examining the eigenvalues of the closed-loop system matrix $A - BK$
• If all the eigenvalues of $A - BK$ have negative real parts, the closed-loop system is asymptotically stable, and the system states will converge to zero asymptotically

Robustness properties of LQR

• LQR controllers exhibit inherent robustness properties against parameter variations and external disturbances
• The robustness of LQR can be attributed to the optimal nature of the control law, which minimizes the quadratic cost function and provides a certain level of tolerance to modeling uncertainties
• LQR controllers have guaranteed gain and phase margins, which quantify the system's ability to maintain stability and performance in the presence of uncertainties and disturbances

LQR design considerations

• When designing an LQR controller, several key considerations need to be taken into account to achieve the desired system performance and robustness
• The selection of weighting matrices $Q$ and $R$ plays a crucial role in shaping the LQR controller's behavior and balancing the trade-off between control effort and state deviation
• Tuning the LQR performance involves iteratively adjusting the weighting matrices and evaluating the resulting system response to meet the specific control objectives
• It is important to be aware of the limitations of the LQR approach, such as its reliance on accurate system models and the assumption of full state feedback

Selection of weighting matrices

• The choice of the weighting matrices $Q$ and $R$ in the quadratic cost function significantly influences the LQR controller's behavior and performance
• The matrix $Q$ determines the relative importance of each state variable in the cost function, while the matrix $R$ determines the relative importance of each control input
• Increasing the values in $Q$ penalizes state deviations more heavily, resulting in faster convergence of the states to their desired values but potentially requiring more control effort
• Increasing the values in $R$ penalizes control effort more heavily, resulting in slower convergence of the states but smoother and less aggressive control actions

Balancing control effort vs state deviation

• One of the key trade-offs in LQR design is balancing the control effort required to achieve the desired system performance and the allowable state deviations from their desired values
• A higher emphasis on state regulation (larger values in $Q$) will result in faster convergence of the states but may require more control effort and potentially lead to actuator saturation
• A higher emphasis on control effort minimization (larger values in $R$) will result in smoother control actions but may allow larger state deviations and slower convergence
• The designer must carefully balance these competing objectives based on the specific requirements and constraints of the control problem

Tuning LQR performance

• Tuning the LQR controller involves iteratively adjusting the weighting matrices $Q$ and $R$ to achieve the desired system performance and robustness
• The tuning process typically involves simulating the closed-loop system with different sets of weighting matrices and evaluating the resulting system response
• Performance metrics such as settling time, overshoot, steady-state error, and control effort can be used to assess the LQR controller's performance and guide the tuning process
• Systematic tuning approaches, such as the Bryson's rule or the pole placement technique, can be employed to provide initial guesses for the weighting matrices and facilitate the tuning process

Limitations of LQR approach

• While LQR is a powerful and widely used optimal control technique, it has certain limitations that should be considered when applying it to practical control problems
• LQR assumes that the system model is accurate and linear, which may not always hold in real-world systems with nonlinearities, uncertainties, and unmodeled dynamics
• LQR requires full state feedback, meaning that all the system states must be measured or estimated accurately, which may be challenging or infeasible in some applications
• The performance of LQR controllers may degrade in the presence of actuator saturation, measurement noise, or external disturbances that are not explicitly accounted for in the design process
• LQR does not inherently handle constraints on the system states or control inputs, which may require additional techniques such as model predictive control or constrained optimization

LQR extensions and variations

• Several extensions and variations of the standard LQR formulation have been developed to address specific control problems and enhance the capabilities of LQR controllers
• These extensions include infinite-horizon and finite-horizon LQR, discrete-time LQR, LQR with state constraints, and LQR with output feedback
• Each of these variations introduces additional considerations and modifications to the standard LQR design process to accommodate the specific requirements and constraints of the control problem

Infinite-horizon vs finite-horizon LQR

• The standard LQR formulation assumes an infinite-horizon cost function, where the control objective is to minimize the cost over an infinite time horizon
• In some applications, such as trajectory planning or time-critical control tasks, a finite-horizon cost function may be more appropriate
• Finite-horizon LQR involves minimizing the cost function over a fixed time interval $[0, T]$, where $T$ is the final time
• The optimal control solution for finite-horizon LQR is time-varying and can be obtained by solving the differential Riccati equation backward in time

Discrete-time LQR

• The standard LQR formulation is based on continuous-time systems, where the system dynamics and control inputs are defined in terms of differential equations
• In practice, many control systems are implemented using digital computers, which operate in discrete time
• Discrete-time LQR involves formulating the optimal control problem for systems described by difference equations, where the state and control variables are defined at discrete time instants
• The discrete-time LQR design process follows a similar approach to the continuous-time case, with modifications to the state-space representation, cost function, and Riccati equation

LQR with state constraints

• The standard LQR formulation does not explicitly handle constraints on the system states, such as physical limits or safety boundaries
• LQR with state constraints extends the LQR framework to incorporate state constraints into the optimal control problem formulation
• State constraints can be handled using techniques such as soft constraints, where the constraints are incorporated into the cost function as penalty terms, or hard constraints, where the constraints are enforced explicitly using optimization methods
• LQR with state constraints requires solving a constrained optimization problem, which can be computationally more demanding than the standard LQR problem

LQR with output feedback

• The standard LQR formulation assumes that all the system states are available for feedback, which may not always be feasible in practice
• LQR with output feedback addresses the situation where only a subset of the system states or linear combinations of the states (outputs) are measurable
• In LQR with output feedback, an observer or state estimator is designed to estimate the unmeasured states based on the available measurements
• The estimated states are then used in the LQR control law, resulting in a combined observer-controller design
• LQR with output feedback requires additional considerations, such as the observability of the system and the stability of the observer-controller loop

Numerical methods for solving LQR

• The LQR design process involves solving the algebraic Riccati equation (ARE) to obtain the optimal state feedback gain matrix
• Several numerical methods have been developed to efficiently solve the ARE and compute the LQR controller gains
• These methods include direct solution techniques, such as the eigenvector method and the Schur method, as well as iterative techniques like the Newton-Kleinman algorithm
• Matlab and Python provide built-in functions and libraries for solving LQR problems, making the implementation of LQR controllers more accessible and efficient

Solving Riccati equation numerically

• The algebraic Riccati equation (ARE) is a key component in the LQR design process and needs to be solved numerically to obtain the optimal state feedback gain matrix
• The ARE is a nonlinear matrix equation of the form $A^TP + PA - PBR^{-1}B^TP + Q = 0$, where $P$ is the symmetric positive definite solution matrix
• Numerical methods for solving the ARE exploit the structure and properties of the equation to efficiently compute the solution matrix $P$
• The eigenvector method, also known as the Potter's method, computes the solution matrix $P$ by solving an eigenvalue problem involving the Hamiltonian matrix associated with the ARE
• The Schur method, also known as the sign function method, computes the solution matrix $P$ by exploiting the invariant subspace property of the Hamiltonian matrix and using the sign function iteration

Matlab/Python implementation of LQR

• Matlab and Python provide powerful tools and libraries for implementing LQR controllers and solving LQR-related problems
• In Matlab, the

  lqr

  function is available in the Control System Toolbox, which takes the system matrices $A$, $B$, $Q$, and $R$ as inputs and returns the optimal state feedback gain matrix $K$
• Python's

  scipy.linalg

  module provides the

  solve_continuous_are

  function, which solves the continuous-time algebraic Riccati equation and returns the solution matrix $P$
• Both Matlab and Python offer additional functions and libraries for state-space modeling, simulation, and analysis of LQR-controlled systems
• These software tools greatly simplify the implementation of LQR controllers and enable rapid prototyping and evaluation of control designs

Computational complexity of LQR

• The computational complexity of solving the LQR problem depends on the size of the system (number of states and inputs) and the numerical method employed
• The eigenvector method for solving the ARE has a computational complexity of $O(n^3)$, where $n$ is the number of states in the system
• The Schur method has a similar computational complexity of $O(n^3)$ but may require fewer iterations to converge compared to the eigenvector method
• Iterative methods, such as the Newton-Kleinman algorithm, have a computational complexity of $O(n^3)$ per iteration and may require multiple iterations to converge to the solution
• For large-scale systems with a high number of states, the computational cost of solving the LQR problem can become significant
• Efficient numerical algorithms and software implementations are crucial for real-time applications and embedded systems with limited computational resources

LQR in practical applications

• LQR has found widespread application in various domains, including aerospace, robotics, process control, and autonomous systems
• In each application, LQR is used to design optimal controllers that regulate the system behavior, minimize performance criteria, and ensure robustness against uncertainties and disturbances
• Practical implementation of LQR controllers requires addressing real-world challenges, such as system identification, sensor and actuator limitations, and computational constraints
• Successful deployment of LQR in practical applications relies on a combination of theoretical understanding, simulation studies, and experimental validation

LQR for aircraft control

• LQR is extensively used in aircraft flight control systems to stabilize and control the aircraft's attitude, altitude, and trajectory
• In aircraft control, LQR is applied to design autopilots, stability augmentation systems, and trajectory tracking controllers
• The system states in aircraft control typically include the aircraft's position, velocity, orientation, and angular rates, while the control inputs are the deflections of control surfaces (ailerons, elevators, rudder) and thrust commands
• LQR controllers in aircraft control are designed to minimize tracking errors, reduce pilot workload, and ensure smooth and precise maneuvers
• Practical considerations in aircraft control include handling actuator saturation, sensor noise, and varying flight conditions (speed, altitude, weight, etc.)

LQR in robotics and autonomous systems

• LQR is widely used in robotics and autonomous systems for motion planning, trajectory tracking, and stabilization of robotic manipulators and mobile robots
• In robotic manipulators, LQR is applied to control the joint angles and end-effector position and orientation, while minimizing tracking errors and energy consumption
• In mobile robots, LQR is used for path following, obstacle avoidance, and stability control, considering the robot's dynamics and kinematic constraints
• LQR controllers in robotics are designed to achieve precise, smooth, and efficient motions, while ensuring robustness against external disturbances and model uncertainties