Orthonormal basis

Bessel's Inequality

Let \H be a Hilbert space, and let $\{e_k{\}}_{k=1}^{\infty }$​ be an orthonormal set in \H. For any $f\in H$, there holds ${\sum }_{k=1}^{\infty }|⟨f,e_k⟩{|}^2\le \Vert f{\Vert }^2.$

Characterisation of Orthonormal Basis in Hilbert Space

The following properties of a countable orthonormal set $\{e_i{\}}_{i\in I}$ in a Hilbert space \H are equivalent:

• $\{e_i{\}}_{i\in I}$ forms a basis.
• The span of $\{e_i{\}}_{i\in I}$ is dense in \H.
• If $f\in H$ and $⟨f,e_i⟩=0$ for all $i\in I$, then $f=0$.
• If $f\in H$, and $S_n(f):={\sum }_{i=1}^n⟨f,e_i⟩e_i$, then $S_n(f)\to f$ as $n\to \infty$.
• For all $f\in H$, $\Vert f{\Vert }^2={\sum }_{i=1}^{\infty }|⟨f,e_i⟩{|}^2$. This is called the Parseval’s identity

Proof The Parseval’s identity is a direct result of the Pythagorean theorem.

Theorem

A Hilbert space has an orthonormal basis if and only if it is separable.

Proof Suppose \H is a separable Hilbert space and $\{f_1,f_2,\dots \}$ is a countable subset of \H whose closure equals \H. We will inductively define an orthonormal basis $\{e_i{\}}_{i=1}^{\infty }$ such that $\text{span}\{f_1,\dots ,f_n\}\subseteq \text{span}\{e_1,\dots ,e_n\}$ for all $n\in \mathbb{N}$. This will imply that \overline{\span\{e_{i}\}_{i=1}^{\infty}}=\H, which will mean that $\{e_i{\}}_{i=1}^{\infty }$ is an orthonormal basis. Without loss of generality, assume that $f_1\ne 0$. First set $e_1:=f_1/\Vert f_1\Vert$. Then suppose for some $n\in \mathbb{N}$, $\{e_1,\dots ,e_n\}$ is an orthonormal set such that $\text{span}\{f_1,\dots ,f_n\}\subseteq \text{span}\{e_1,\dots ,e_n\}$. If $f_k\in \text{span}\{e_1,\dots ,e_n\}$ for every $k\in \mathbb{N}$, then $\{e_1,\dots ,e_n\}$ is an orthonormal basis of \H and the process should be stopped. Otherwise, let $m$ be the smallest positive integer such that $f_m\in \{e_1,\dots ,e_n\}.$Define $e_{n+1}$ as follows: $e_{n+1}=\frac{f_m-⟨f_m,e_1⟩e_1-\cdots -⟨f_m,e_n⟩e_n}{\Vert f_m-⟨f_m,e_1⟩e_1-\cdots -⟨f_m,e_n⟩e_n\Vert }.$Clearly $\Vert e_{n+1}\Vert =1$, $⟨e_{n+1},e_k⟩=0$ for all $k\le n$, our choice of $f_m$ guarantees there is no division by $0$, and $\mathrm{span}\{f_1,\dots ,f_{n+1}\}\subseteq \mathrm{span}\{e_1,\dots ,e_{n+1}\},$completing the induction. $\square$

Same Gram-Schmidt Procedure in Linear Algebra

This is actually the same Gram-Schmidt procedure in linear algebra.