Orthogonality and Bounded Linear Maps

Closed Subspaces and Orthogonality

Proposition

In any finite dimensional inner product space, every subspace is closed.

Proof Suppose $V$ is a finite dimensional inner product space and $W$ is a subspace of $V$. Then $W$ is finite dimensional. Since every finite dimensional normed space is complete, it follows that $W$ is closed in $V$. $\square$

Remark

However, this is not true in general for infinite dimensional spaces. For example, in an infinite dimensional Hilbert space with basis $\{e_i{\}}_{i=1}^{\infty }$, the subspace $\mathrm{span}\{e_1,e_2,\dots \}$, which is the space of all finite linear combinations of the basis vectors, is not closed. To see this, consider the sequence $f_n={\sum }_{i=1}^n{e}_i$, which converges to $f={\sum }_{i=1}^{\infty }{e}_i$, which is not in $\text{span}\{e_1,e_2,\dots \}$.

Lemma

Suppose \renewcommand{\S}{\mathcal{S}} \S\subset \H is a closed subspace of a Hilbert space \H, and f\in \H. Then

• There exists a unique element $g\in \S$ which is closest to $f$, in the sense that $\Vert f-g\Vert ={\inf }_{h\in \S }\Vert f-h\Vert$.
• $f-g$ is perpendicular to $\S$. i.e. $⟨f-g,h⟩=0$ for all $h\in \S$.

Proposition

If $\S$ is a closed subspace of a Hilbert space \newcommand{\H}{\mathcal{H}}\H, then \H is a direct sum of $\S$ and its orthogonal: $\stackrel{˝}{=}\S \oplus {\S }^{\perp }.$

Proposition

Suppose $U\subset V$ is a subspace of an (possibly infinite dimensional) inner product space, then $U^{\perp }$ is always closed, and $(U^{\perp }{)}^{\perp }=\bar{U}$, where $\bar{U}$ denotes the closure of $U$.

Proof For all $\{f_n\}\subset U^{\perp }$ that converges to $f$, we claim that $f\in U^{\perp }$. Note that for all $g\in U$ and some $n\in \mathbb{N}$, we have $|⟨f_n,g⟩-⟨f,g⟩|=|⟨f_n-f,g⟩|\le \Vert f_n-f\Vert \cdot \Vert g\Vert ,$so $\{⟨f_n,g⟩{\}}_{n=1}^{\infty }$ converges to $⟨f,g⟩$. Note that since each $⟨f_n,g⟩=0$, it converges to $0\in \text{C}$. Hence, $⟨f,g⟩=0$ for all $g\in U$. This shows that $f\in U^{\perp }$. So $U^{\perp }$ is closed. Thus $(U^{\perp }{)}^{\perp }$ is closed, and $\bar{U}\subset (U^{\perp }{)}^{\perp }$. Now for any $h\in (U^{\perp }{)}^{\perp }$, since $\bar{U}$ is closed, $\stackrel{˝}{=}\bar{U}\oplus {\bar{U}}^{\perp }$, so we can write $h=h_1+h_2$, where $h_1\in \bar{U}$ and $h_2\in {\bar{U}}^{\perp }$. For any $g\in {\bar{U}}^{\perp }\subset U^{\perp }$, there holds $0=⟨h,g⟩=⟨h_1,g⟩+⟨h_2,g⟩.$As $h_1\in \bar{U}$, $⟨h_1,g⟩=0$. So $⟨h_2,g⟩=0$ for all $g\in {\bar{U}}^{\perp }$, which implies $⟨h_2,h_2⟩=0$, hence $h_2=0$. Therefore, $h=h_1\in \bar{U}$, which implies that $(U^{\perp }{)}^{\perp }=\bar{U}$. $\square$

Corollary

Suppose $U\subset V$ is a subspace of an (infinite dimensional) inner product space, then $U$ is dense in $V$ iff $U^{\perp }=\{0\}$.

Proof This is straightforward from the previous proposition. $\square$

e.g. Consider the Hilbert space $L^2(S^1)$, has an orthonormal basis $\{e^{in\theta }{\}}_{n\in \mathbb{Z}}$, where

Bounded Linear Operators

Bounded Linear Operator

Suppose $V$ and $W$ are normed spaces, then a linear operator $T:V\to W$ is bounded if there exists $M>0$ such that $\Vert T(v){\Vert }_W\le M\Vert v{\Vert }_V,\text{for all }v\in V.$The operator norm of $T$ is defined as $\Vert T\Vert :={\sup }_{\Vert f{\Vert }_V=1}\Vert T(f){\Vert }_W.$

Lemma

For any bounded linear map T\colon \H_{1}\to\H_{2} between two Hilbert spaces, we have $\Vert T\Vert ={\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨Tf,g⟩|.$

Proof By Cauchy-Schwarz inequality, for all g\in\H_{2} with $\Vert g\Vert =1$, we have $|⟨Tf,g⟩|\le \Vert Tf\Vert \Vert g\Vert =\Vert Tf\Vert \le \Vert T\Vert ,$ and so $\Vert T\Vert \ge {\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨Tf,g⟩|$. Conversely, $\Vert Tf\Vert =⟨Tf,\frac{Tf}{\Vert Tf\Vert }⟩\le {\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨Tf,g⟩|,$hence, we have $\Vert T\Vert \le {\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨Tf,g⟩|$, which implies that $\Vert T\Vert ={\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨Tf,g⟩|$. $\square$

Proposition

A linear operator T\colon\H_{1}\to\H_{2} is bounded if and only if it is continuous w.r.t the norm topology.

Proof If $T$ is bounded, then for all $\{f_n{\}}_{n=1}^{\infty }$ in \H_{1}, we have $\Vert T(f_n)-T(f)\Vert \le M\Vert f_n-f\Vert$ for all $n$. Taking the limit as $n\to \infty$, we have $\Vert T(f_n)-T(f)\Vert \to 0$ as $n\to \infty$, so $T$ is continuous. Conversely, if $T$ is continuous, assume that $T$ is unbounded. Then for each $n>0$, there exists some $g_n$ such that $\Vert T(g_n)\Vert >n\Vert g_n\Vert$. Let $h_n:=g_n/(n\Vert g_n\Vert )$, then $\Vert h_n\Vert =1/n$, so $h_n\to 0$. Since $T$ is continuous at $0$, we have $T(h_n)\to 0$, however, $\Vert T(h_n)\Vert >1$, which is a contradiction. Therefore, $T$ must be bounded. $\square$

Remark

There are many types of continuity. In fact, we have the following chain of implications: $\Vert T_n-T\Vert \to 0⟹\forall f,T_{n}f\to Tf⟹\forall g,⟨T_{n}f,g⟩\to ⟨Tf,g⟩.$

Proposition

For any bounded linear operator T\colon \H_{1}\to \H_{2}, there holds $\Vert T{\Vert }^2=\Vert T^{\ast }{\Vert }^2=\Vert T\circ T^{\ast }\Vert =\Vert T^{\ast }\circ T\Vert .$

Proof We shall first show that $\Vert T\Vert =\Vert T^{\ast }\Vert$. In fact, by the lemma $\Vert T\Vert ={\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨Tf,g⟩|={\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨f,T^{\ast }g⟩|={\sup }_{\Vert f\Vert =1,\Vert g\Vert =1}|⟨T^{\ast }g,f⟩|=\Vert T^{\ast }\Vert .$Now, for any $\Vert f\Vert =1$ and $\Vert g\Vert =1$, we have $|⟨T(T^{\ast }f),g⟩|=|⟨T^{\ast }f,T^{\ast }g⟩|\le \Vert T^{\ast }f\Vert \Vert T^{\ast }g\Vert \le \Vert T^{\ast }{\Vert }^2,$thus $\Vert T\circ T^{\ast }\Vert =\sup |⟨T(T^{\ast }f),g⟩|\le \Vert T^{\ast }{\Vert }^2$. For the other direction, note that $\Vert T\circ T^{\ast }\Vert \ge ⟨T(T^{\ast }f),f⟩=⟨T^{\ast }f,T^{\ast }f⟩=\Vert T^{\ast }f{\Vert }^2,$so $\Vert T\circ T^{\ast }\Vert \ge \sup \Vert T^{\ast }f{\Vert }^2=\Vert T^{\ast }{\Vert }^2,$ hence $\Vert T\circ T^{\ast }\Vert =\Vert T^{\ast }{\Vert }^2$. The same argument applies to $\Vert T^{\ast }\circ T\Vert$. $\square$