🧚🏽‍♀️Abstract Linear Algebra I Unit 3 – Linear Transformations & Matrices

Key Concepts

• Linear transformations map vectors from one vector space to another while preserving linear combinations
• Matrices can represent linear transformations, with matrix multiplication corresponding to composition of transformations
• Eigenvalues and eigenvectors capture important properties of linear transformations and matrices
  • Eigenvectors are vectors that remain in the same direction after a linear transformation is applied
  • Eigenvalues are the scaling factors associated with eigenvectors
• Determinants provide information about the invertibility and volume-scaling properties of matrices and linear transformations
• Similarity transformations allow for changing the basis of a matrix representation while preserving its eigenvalues
• Diagonalization is the process of finding a basis in which a matrix has a diagonal representation, simplifying computations
• Inner product spaces introduce the concept of angle and length, enabling the study of orthogonality and orthonormal bases

Definitions & Terminology

• Linear transformation: a function $T: V \to W$ between vector spaces $V$ and $W$ that satisfies $T(u+v) = T(u) + T(v)$ and $T(cv) = cT(v)$ for all vectors $u,v \in V$ and scalars $c$
• Matrix representation: a matrix $A$ that represents a linear transformation $T$ with respect to given bases of the domain and codomain
• Eigenvalue: a scalar $\lambda$ such that $Av = \lambda v$ for some nonzero vector $v$ and matrix $A$
• Eigenvector: a nonzero vector $v$ such that $Av = \lambda v$ for some scalar $\lambda$ and matrix $A$
• Characteristic polynomial: the polynomial $p(x) = \det(A - xI)$, where $A$ is a square matrix and $I$ is the identity matrix
  • The roots of the characteristic polynomial are the eigenvalues of $A$
• Diagonalizable matrix: a square matrix $A$ that can be written as $A = PDP^{-1}$, where $D$ is a diagonal matrix and $P$ is an invertible matrix
• Orthogonal vectors: vectors $u$ and $v$ such that their inner product $\langle u, v \rangle = 0$
• Orthonormal basis: a basis consisting of mutually orthogonal unit vectors

Properties & Theorems

• Linearity properties: for a linear transformation $T$ and vectors $u,v \in V$ and scalar $c$, $T(u+v) = T(u) + T(v)$ and $T(cv) = cT(v)$
• Matrix multiplication theorem: if $A$ and $B$ are matrices representing linear transformations $T_A$ and $T_B$, then $AB$ represents the composition $T_A \circ T_B$
• Eigenvalue-eigenvector equation: for a matrix $A$, a scalar $\lambda$, and a nonzero vector $v$, $Av = \lambda v$ if and only if $\lambda$ is an eigenvalue of $A$ and $v$ is an associated eigenvector
• Spectral theorem: a real symmetric matrix is always diagonalizable with real eigenvalues and orthogonal eigenvectors
• Cayley-Hamilton theorem: every square matrix satisfies its own characteristic equation, i.e., $p(A) = 0$, where $p(x)$ is the characteristic polynomial of $A$
• Orthogonality and inner products: for vectors $u,v \in V$, $\langle u, v \rangle = 0$ if and only if $u$ and $v$ are orthogonal
• Parseval's identity: for an orthonormal basis $\{e_1, \ldots, e_n\}$ and any vector $v$, $\|v\|^2 = \sum_{i=1}^n |\langle v, e_i \rangle|^2$

Matrix Representations

• Matrix of a linear transformation: given bases $\{v_1, \ldots, v_n\}$ and $\{w_1, \ldots, w_m\}$ for vector spaces $V$ and $W$, the matrix $A$ of a linear transformation $T: V \to W$ has entries $a_{ij} = \langle T(v_j), w_i \rangle$
• Change of basis matrix: a matrix $P$ that transforms coordinates from one basis to another, i.e., $[v]_B = P[v]_A$ for a vector $v$ and bases $A$ and $B$
  • The columns of $P$ are the coordinates of the basis vectors of $B$ with respect to the basis $A$
• Similarity transformation: matrices $A$ and $B$ are similar if there exists an invertible matrix $P$ such that $B = P^{-1}AP$
  • Similar matrices have the same eigenvalues and characteristic polynomials
• Diagonalization: a matrix $A$ is diagonalizable if it can be written as $A = PDP^{-1}$, where $D$ is a diagonal matrix and $P$ is an invertible matrix
  • The columns of $P$ are eigenvectors of $A$, and the diagonal entries of $D$ are the corresponding eigenvalues
• Orthogonal matrix: a square matrix $Q$ is orthogonal if $Q^T = Q^{-1}$, where $Q^T$ is the transpose of $Q$
  • Orthogonal matrices preserve inner products and lengths of vectors

Geometric Interpretations

• Linear transformations as geometric mappings: linear transformations can be visualized as mappings that preserve lines and the origin in a vector space
  • Examples include rotations, reflections, scaling, and shearing
• Eigenvectors as invariant directions: eigenvectors of a linear transformation or matrix represent directions that remain unchanged (up to scaling) under the transformation
• Eigenvalues as scaling factors: eigenvalues determine how much the corresponding eigenvectors are scaled under a linear transformation
• Determinants and volume scaling: the determinant of a matrix represents the factor by which the matrix scales the volume of a unit cube
  • A negative determinant indicates a reflection or orientation reversal
• Orthogonality and perpendicularity: orthogonal vectors can be interpreted as perpendicular vectors in a geometric sense
• Inner products and angles: the inner product of two vectors is related to the cosine of the angle between them, with orthogonal vectors having an inner product of zero

Applications & Examples

• Markov chains: matrices can represent transition probabilities in a Markov chain, with eigenvalues and eigenvectors providing information about long-term behavior and steady-state distributions
• Quantum mechanics: linear transformations and matrices are fundamental in describing quantum states, observables, and time evolution in quantum systems (Schrödinger equation)
• Computer graphics: linear transformations are used extensively in computer graphics for modeling, rendering, and animating 2D and 3D objects (affine transformations, homogeneous coordinates)
• Principal component analysis (PCA): eigenvalues and eigenvectors are used in PCA to identify the most important directions of variation in high-dimensional data sets, enabling dimensionality reduction and feature extraction
• Differential equations: linear transformations and matrices arise in the study of systems of linear differential equations, with eigenvalues and eigenvectors playing a crucial role in the solution and stability analysis (exponential of a matrix)
• Fourier analysis: linear transformations are at the heart of Fourier analysis, which decomposes functions into sums or integrals of simpler trigonometric functions (Fourier series, Fourier transform)

Computational Techniques

• Gaussian elimination: a method for solving systems of linear equations and finding the inverse of a matrix by performing row operations to transform the matrix into row echelon form
• Eigenvalue computation: various algorithms exist for computing eigenvalues and eigenvectors of matrices, such as the power iteration method, QR algorithm, and Jacobi method
  • The choice of algorithm depends on the matrix properties and the desired accuracy
• Matrix diagonalization: to diagonalize a matrix $A$, find a basis of eigenvectors and form the matrix $P$ whose columns are these eigenvectors; the diagonal matrix $D$ has the corresponding eigenvalues on its diagonal
• Orthogonalization: the Gram-Schmidt process is an algorithm for constructing an orthonormal basis from a given set of linearly independent vectors
  • This is useful for obtaining orthonormal bases in inner product spaces
• Numerical stability: when performing computations with matrices, it's important to consider numerical stability and the potential for round-off errors
  • Techniques such as pivoting and iterative refinement can help mitigate these issues

Common Pitfalls & Tips

• Not checking for linearity: when working with linear transformations, always verify that the linearity properties hold; not all functions between vector spaces are linear transformations
• Confusing matrix multiplication order: matrix multiplication is not commutative, so the order of multiplication matters; always pay attention to the order of composition when working with multiple linear transformations
• Forgetting to check for invertibility: when using matrices to represent linear transformations, ensure that the matrices are invertible (nonzero determinant) if you need to work with their inverses
• Misinterpreting eigenvalues and eigenvectors: remember that eigenvectors are only defined up to scalar multiplication, so they are not unique; also, not all matrices have a full set of eigenvectors (defective matrices)
• Mishandling complex eigenvalues: when working with real matrices, eigenvalues and eigenvectors may be complex; ensure you're using appropriate techniques for dealing with complex numbers
• Overlooking the importance of bases: always consider the bases of the vector spaces you're working with, as the matrix representation of a linear transformation depends on the choice of bases
• Not exploiting matrix properties: when working with specific types of matrices (symmetric, orthogonal, diagonal, etc.), take advantage of their special properties to simplify computations and proofs