4.3 Orthogonality and orthonormal bases

Inner product spaces introduce orthogonality, a concept that extends perpendicularity. This idea is crucial for understanding vector relationships and simplifying calculations. Orthogonal vectors are independent, allowing complex problems to be broken down into manageable parts.

Orthonormal bases, created through the Gram-Schmidt process, are key tools in inner product spaces. These bases simplify computations, improve numerical stability, and have wide-ranging applications in math and engineering. Understanding orthonormality is essential for mastering inner product spaces.

Orthogonality in Inner Product Spaces

Definition and Properties

Top images from around the web for Definition and Properties

• 

  Cross product - Wikiversity View original

  Is this image relevant?

• 

  Inner product space - Wikipedia View original

  Is this image relevant?

• 

  Hyperbolic orthogonality - Wikipedia View original

  Is this image relevant?

• 

  Cross product - Wikiversity View original

  Is this image relevant?

• 

  Inner product space - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Definition and Properties

• 

  Cross product - Wikiversity View original

  Is this image relevant?

• 

  Inner product space - Wikipedia View original

  Is this image relevant?

• 

  Hyperbolic orthogonality - Wikipedia View original

  Is this image relevant?

• 

  Cross product - Wikiversity View original

  Is this image relevant?

• 

  Inner product space - Wikipedia View original

  Is this image relevant?

1 of 3

• Orthogonality generalizes perpendicularity from Euclidean space to inner product spaces
• Two vectors u and v are orthogonal when their inner product equals zero ($<u, v> = 0$)
• Zero vector stands orthogonal to all vectors, including itself
• Orthogonality exhibits symmetry (if u ⊥ v, then v ⊥ u)
• Pythagorean theorem extends to inner product spaces ($||u + v||² = ||u||² + ||v||²$ for orthogonal u and v)
• Orthogonal set comprises vectors where $<vᵢ, vⱼ> = 0$ for all i ≠ j
  • Example: In ℝ³, vectors (1,0,0), (0,1,0), and (0,0,1) form an orthogonal set
• Non-zero orthogonal vectors always maintain linear independence
  • Example: In ℝ², vectors (3,4) and (-4,3) are orthogonal and linearly independent

Applications and Significance

• Orthogonality simplifies calculations in linear algebra and functional analysis
• Orthogonal vectors decompose complex problems into simpler, independent components
• Orthogonal matrices preserve inner products and vector lengths
  • Example: Rotation matrices in 2D and 3D are orthogonal
• Orthogonality plays a crucial role in signal processing and data compression
  • Example: Discrete Cosine Transform used in JPEG image compression relies on orthogonal basis functions

Gram-Schmidt Process for Orthonormal Bases

Process Description and Implementation

• Gram-Schmidt process converts linearly independent vectors into an orthonormal basis
• Process steps for vectors {v₁, ..., vₖ}:
  1. Normalize first vector: $u₁ = v₁ / ||v₁||$
  2. For i > 1, compute: $uᵢ = (vᵢ - Σⱼ₌₁ⁱ⁻¹ <vᵢ, uⱼ>uⱼ) / ||vᵢ - Σⱼ₌₁ⁱ⁻¹ <vᵢ, uⱼ>uⱼ||$
• Resulting orthonormal vectors {u₁, ..., uₖ} span the same subspace as original vectors
• Process applies to any basis of a finite-dimensional inner product space
  • Example: Converting standard basis {(1,0), (0,1)} in ℝ² to an orthonormal basis

Applications and Advantages

• Gram-Schmidt process finds use in various mathematical and engineering fields
• Orthonormal bases simplify computations involving inner products and projections
• Process aids in solving systems of linear equations (QR decomposition)
• Gram-Schmidt orthogonalization improves numerical stability in computer algorithms
  • Example: Enhancing accuracy in least squares fitting of data points

Uniqueness of Orthonormal Bases

Relationship Between Orthonormal Bases

• Orthonormal bases remain unique up to orthogonal transformations
• Any two orthonormal bases {e₁, ..., eₙ} and {f₁, ..., fₙ} relate through an orthogonal transformation T
• Orthogonal matrix A represents transformation T, satisfying $A^T = A^{-1}$
• Change of basis matrix between orthonormal bases always results in an orthogonal matrix
  • Example: Rotating orthonormal basis in ℝ² by 45° produces another orthonormal basis

Properties and Implications

• Number of vectors in any orthonormal basis equals the space's dimension
• Uniqueness ensures properties derived using one orthonormal basis hold for all others
• Orthonormal bases preserve inner products and norms under basis transformations
• Concept extends to infinite-dimensional Hilbert spaces with some modifications
  • Example: Fourier series uses orthonormal basis of trigonometric functions in L²[-π,π]

Orthonormal Basis Expansions

Vector Representation and Fourier Coefficients

• Any vector v expands uniquely as $v = Σᵢ₌₁ⁿ <v, eᵢ>eᵢ$ in orthonormal basis {e₁, ..., eₙ}
• Coefficients $<v, eᵢ>$ denote Fourier coefficients of v relative to the orthonormal basis
• Fourier coefficients minimize distance between v and its projection onto subspace span{e₁, ..., eₖ}
• Parseval's identity states $||v||² = Σᵢ₌₁ⁿ |<v, eᵢ>|²$ for any vector v
  • Example: In ℝ³ with orthonormal basis {e₁, e₂, e₃}, vector v = 2e₁ - 3e₂ + e₃ has ||v||² = 2² + (-3)² + 1² = 14

Applications and Extensions

• Orthonormal basis expansions facilitate inner product, norm, and projection computations
• Concept generalizes to infinite-dimensional Hilbert spaces, foundational in Fourier analysis
• Expansions find use in signal processing, quantum mechanics, and data compression
  • Example: Representing audio signals as sum of sine and cosine waves (Fourier series)
• Orthonormal basis expansions enable efficient data storage and transmission
  • Example: Compressing images by truncating expansions in wavelet bases