10.1 Hilbert space theory and orthonormal bases

Hilbert spaces are like infinite-dimensional versions of the spaces we're used to, but with special rules. They let us work with complex mathematical objects using familiar tools like distance and angles.

In this part, we'll learn about the building blocks of Hilbert spaces: orthonormal bases. These are sets of vectors that help us break down complicated objects into simpler pieces, making them easier to understand and work with.

Hilbert Space Fundamentals

Definition and Properties of Hilbert Spaces

Top images from around the web for Definition and Properties of Hilbert Spaces

• 

  Tensor Product of 2-Frames in 2-Hilbert Spaces View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Tensor Product of 2-Frames in 2-Hilbert Spaces View original

  Is this image relevant?

• 

  Tensor Product of 2-Frames in 2-Hilbert Spaces View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Definition and Properties of Hilbert Spaces

• 

  Tensor Product of 2-Frames in 2-Hilbert Spaces View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Tensor Product of 2-Frames in 2-Hilbert Spaces View original

  Is this image relevant?

• 

  Tensor Product of 2-Frames in 2-Hilbert Spaces View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

1 of 3

• Hilbert spaces generalize the notion of Euclidean space to infinite-dimensional vector spaces while preserving the structure of an inner product
• A Hilbert space $H$ is a complete inner product space, meaning it is a vector space equipped with an inner product $\langle \cdot, \cdot \rangle$ and is complete with respect to the norm induced by the inner product
• The inner product $\langle x, y \rangle$ is a function that assigns a scalar value to each pair of vectors $x, y \in H$ and satisfies the following properties:
  • Conjugate symmetry: $\langle x, y \rangle = \overline{\langle y, x \rangle}$
  • Linearity in the second argument: $\langle x, \alpha y + \beta z \rangle = \alpha \langle x, y \rangle + \beta \langle x, z \rangle$
  • Positive definiteness: $\langle x, x \rangle \geq 0$ and $\langle x, x \rangle = 0$ if and only if $x = 0$
• The norm $\|x\|$ of a vector $x \in H$ is defined as $\|x\| = \sqrt{\langle x, x \rangle}$ and measures the length or magnitude of the vector
  • The norm satisfies the triangle inequality: $\|x + y\| \leq \|x\| + \|y\|$ for all $x, y \in H$

Completeness and Convergence in Hilbert Spaces

• Completeness is a crucial property of Hilbert spaces ensures that every Cauchy sequence in the space converges to a limit within the space
  • A sequence $\{x_n\}$ in a Hilbert space $H$ is a Cauchy sequence if for every $\varepsilon > 0$, there exists an $N$ such that $\|x_n - x_m\| < \varepsilon$ for all $n, m \geq N$
• Convergence in a Hilbert space can be defined using the norm: a sequence $\{x_n\}$ converges to $x \in H$ if $\lim_{n \to \infty} \|x_n - x\| = 0$
  • This is equivalent to convergence with respect to the inner product: $\lim_{n \to \infty} \langle x_n - x, y \rangle = 0$ for all $y \in H$
• Examples of Hilbert spaces include:
  • The space of square-integrable functions $L^2([a, b])$ with the inner product $\langle f, g \rangle = \int_a^b f(x) \overline{g(x)} dx$
  • The space of square-summable sequences $\ell^2(\mathbb{N})$ with the inner product $\langle \{a_n\}, \{b_n\} \rangle = \sum_{n=1}^\infty a_n \overline{b_n}$

Orthogonality and Bases

Orthogonality and Orthonormal Bases

• Two vectors $x, y \in H$ are orthogonal if their inner product is zero: $\langle x, y \rangle = 0$
  • Orthogonality generalizes the notion of perpendicularity in Euclidean spaces to Hilbert spaces
• An orthonormal basis for a Hilbert space $H$ is a countable set of vectors $\{e_n\}$ that are orthogonal to each other, have unit norm, and span the entire space
  • Orthogonality: $\langle e_n, e_m \rangle = 0$ for all $n \neq m$
  • Unit norm: $\|e_n\| = 1$ for all $n$
  • Completeness: every vector $x \in H$ can be represented as a unique linear combination of the basis vectors: $x = \sum_{n=1}^\infty \langle x, e_n \rangle e_n$
• The coefficients $\langle x, e_n \rangle$ in the expansion of a vector $x$ with respect to an orthonormal basis $\{e_n\}$ are called the Fourier coefficients of $x$

Construction of Orthonormal Bases

• The Gram-Schmidt process is an algorithm for constructing an orthonormal basis from a linearly independent set of vectors $\{v_n\}$ in a Hilbert space $H$
  • The process works by sequentially orthogonalizing and normalizing the vectors:
    1. Set $e_1 = v_1 / \|v_1\|$
    2. For $n \geq 2$, define $u_n = v_n - \sum_{k=1}^{n-1} \langle v_n, e_k \rangle e_k$ and set $e_n = u_n / \|u_n\|$
  • The resulting set $\{e_n\}$ is an orthonormal basis for the subspace spanned by $\{v_n\}$
• Examples of orthonormal bases include:
  • The standard basis $\{(1, 0, 0, \ldots), (0, 1, 0, \ldots), (0, 0, 1, \ldots), \ldots\}$ for the space of square-summable sequences $\ell^2(\mathbb{N})$
  • The trigonometric basis $\{1, \cos(nx), \sin(nx) : n \in \mathbb{N}\}$ for the space of square-integrable functions on $[0, 2\pi]$

Fourier Series in Hilbert Spaces

Fourier Series Expansion

• A Fourier series is an expansion of a periodic function $f$ in a Hilbert space $H$ as an infinite linear combination of orthonormal basis functions
  • The basis functions are typically trigonometric functions (sines and cosines) or complex exponentials
• Given an orthonormal basis $\{e_n\}$ for a Hilbert space $H$ and a vector $f \in H$, the Fourier series of $f$ with respect to the basis $\{e_n\}$ is the infinite sum: $f = \sum_{n=1}^\infty \langle f, e_n \rangle e_n$
  • The coefficients $\langle f, e_n \rangle$ are the Fourier coefficients of $f$ and can be computed using the inner product
• The Fourier series converges to $f$ in the norm of the Hilbert space: $\lim_{N \to \infty} \|f - \sum_{n=1}^N \langle f, e_n \rangle e_n\| = 0$

Completeness and Convergence of Fourier Series

• The completeness of the orthonormal basis $\{e_n\}$ ensures that the Fourier series of any vector $f \in H$ converges to $f$ in the norm of the Hilbert space
  • This means that the partial sums $\sum_{n=1}^N \langle f, e_n \rangle e_n$ approximate $f$ arbitrarily well as $N$ increases
• The convergence of the Fourier series can be understood in terms of the inner product: for any $g \in H$, $\lim_{N \to \infty} \langle f - \sum_{n=1}^N \langle f, e_n \rangle e_n, g \rangle = 0$
  • This implies that the Fourier series converges to $f$ in a weak sense, which is a more general notion of convergence than norm convergence
• Examples of Fourier series expansions include:
  • The Fourier series of a square-integrable periodic function $f$ on $[0, 2\pi]$ with respect to the trigonometric basis $\{1, \cos(nx), \sin(nx) : n \in \mathbb{N}\}$: $f(x) = \frac{a_0}{2} + \sum_{n=1}^\infty (a_n \cos(nx) + b_n \sin(nx))$ where $a_n = \frac{1}{\pi} \int_0^{2\pi} f(x) \cos(nx) dx$ and $b_n = \frac{1}{\pi} \int_0^{2\pi} f(x) \sin(nx) dx$
  • The Fourier series of a square-summable sequence $\{a_n\}$ with respect to the standard basis of $\ell^2(\mathbb{N})$: $\{a_n\} = \sum_{n=1}^\infty a_n e_n$ where $e_n$ is the sequence with a 1 in the $n$-th position and 0 elsewhere