Conditional Expectation

Theorem

Let $(\Omega ,\Sigma ,\mathbb{P})$ be a probability space and ${\Sigma }_0\subset \Sigma$ a sigma-algebra. There exists a unique linear continuous map $T:L^2(\Omega ,\Sigma ,\mathbb{P})\to L^2(\Omega ,{\Sigma }_0,\mathbb{P})$, such that for any $X\in L^2(\Omega ,\Sigma ,\mathbb{P})$ and $A\in {\Sigma }_0$, ${\int }_{A}Xd\mathbb{P}={\int }_{A}T(X)d\mathbb{P}.$

Proof As ${\Sigma }_0$ is a sub-$\sigma$-algebra of $\Sigma$, any ${\Sigma }_0$-measurable function is necessarily $\Sigma$-measurable. This allows us to consider the map

$$j:L^2(\Omega ,{\Sigma }_0,\mathbb{P})\to L^2(\Omega ,\Sigma ,\mathbb{P})$$

defined by

$$j(Y):=Y\text{for }Y\in L^2(\Omega ,{\Sigma }_0,\mathbb{P}).$$

For any $X\in L^2(\Omega ,\Sigma ,\mathbb{P})$, the map

$$l_X:L^2(\Omega ,{\Sigma }_0,\mathbb{P})\to \mathbb{C}$$

defined by

$$l_X(Y):=⟨j(Y),X⟩,Y\in L^2(\Omega ,{\Sigma }_0,\mathbb{P}),$$

is a bounded linear functional on $L^2(\Omega ,{\Sigma }_0,\mathbb{P})$.

According to the Riesz representation theorem, there exists a unique element

$$Y_X\in L^2(\Omega ,{\Sigma }_0,\mathbb{P})$$

such that

$$l_X(Y)=⟨Y,Y_X⟩$$

for all $Y\in L^2(\Omega ,{\Sigma }_0,\mathbb{P})$. Define

$$T:L^2(\Omega ,\Sigma ,\mathbb{P})\to L^2(\Omega ,{\Sigma }_0,\mathbb{P})$$

by $T(X):=Y_X.$ Then $T$ is the adjoint map of $j$, i.e. $T=j^{\ast }.$ For any $X\in L^2(\Omega ,\Sigma ,\mathbb{P})$ and $A\in {\Sigma }_0$,

$$\begin{array}{rl}{\int }_{A}Xd\mathbb{P}& =⟨j(1_A),X⟩\\ & =⟨1_A,T(X)⟩\\ & ={\int }_{A}T(X)d\mathbb{P}.\end{array}$$

Uniqueness follows directly from the Riesz representation theorem.

Conditional Expectation

The unique map $T$ satisfying previous theorem is called the conditional expectation with respect to the sigma-algebra ${\Sigma }_0$. For $X\in L^2(\Omega ,\Sigma ,\mathbb{P})$, the conditional expectation of $X$ given ${\Sigma }_0$ will be denoted by $E(X\mid {\Sigma }_0):=T(X)$.

Example

Conditioning Over a Random Variable Let $X_0:\Omega \to \mathbb{R}$ be a random variable and $\mathcal{B}$ denote the Borel sigma algebra of $\mathbb{R}$. For $X\in L^2(\Omega ,\Sigma ,\mathbb{P})$, define $E[X\mid X_0]:=E(X\mid \sigma (X_0)),$ where $\sigma (X_0):=\{X_0^{-1}(B):B\in \mathcal{B}\}$ is the sigma-algebra generated by $X_0$. $E(\cdot \mid X_0)$ is referred to as the conditional expectation given the random variable $X_0$.

Proposition

Let $(\Omega ,\Sigma ,\mathbb{P})$ be a probability space and ${\Sigma }_0\subset \Sigma$ a sub-sigma-algebra. The following properties hold for any $X$ and $Y$ in $L^2(\Omega ,\Sigma ,\mathbb{P})$. (i) Linearity: For any $a,b\in \mathbb{R}$, $E(aX+bY\mid {\Sigma }_0)=aE(X\mid {\Sigma }_0)+bE(Y\mid {\Sigma }_0)$ almost surely. (ii) Monotonicity: Suppose that $X\le Y$ almost surely. Then, $E(X\mid {\Sigma }_0)\le E(Y\mid {\Sigma }_0)$ almost surely. (iii) The Tower Property: Let ${\Sigma }_1$ be a sub-sigma-algebra of ${\Sigma }_0$. Then, $E(E(X\mid {\Sigma }_0)\mid {\Sigma }_1)=E(X\mid {\Sigma }_1)=E(E(X\mid {\Sigma }_1)\mid {\Sigma }_0).(1.5)$ (iv) Jensen’s Inequality For any convex $\varphi :\mathbb{R}\to \mathbb{R}$, $\varphi (E(X\mid {\Sigma }_0))\le E(\varphi (X)\mid {\Sigma }_0).(1.6)$ (v) If ${\Sigma }_0$ is the trivial sigma-algebra, ${\Sigma }_0=\{\varnothing ,\Omega \}$, then $E(X\mid {\Sigma }_0)=E(X)$. (vi)Law of Total Expectation: $E(E(X\mid Y))=E(X).(1.7)$ (vii)Taking Out What Is Known: If $Y$ is ${\Sigma }_0$-measurable, then $E(Y\cdot X\mid {\Sigma }_0)=Y\cdot E(X\mid {\Sigma }_0)$. (viii) Extension to $L^1$: We have that $\mathbb{E}[E(X\mid {\Sigma }_0)]\le \mathbb{E}|X|.$ Consequently, $E(\cdot \mid {\Sigma }_0)$ extends to a bounded linear map from $L^1(\Omega ,\Sigma ,\mathbb{P})$ to $L^1(\Omega ,{\Sigma }_0,\mathbb{P})$.

Proof

(ii) Define the set

$$A:=\{\omega :\mathbb{E}(Y-X\mid {\Sigma }_0)(\omega )<0\}.$$

From linearity, our claim will be proved if it can be shown that $A$ has zero measure. Argue by contradiction and suppose that $A$ has finite measure. As $\mathbb{E}(Y-X\mid {\Sigma }_0)$ is by definition ${\Sigma }_0$-measurable, it follows that $A\in {\Sigma }_0$. By the defining property of conditional expectation, $$ 0 \leq \int_A Y-X,d\mathbb{P}

\int_A \mathbb{E}(Y-X\mid\Sigma_0),d\mathbb{P} <0.$$ This is a contradiction. It can then be safely concluded that $A$ has zero measure and therefore

$$\mathbb{E}(X\mid {\Sigma }_0)\le \mathbb{E}(Y\mid {\Sigma }_0)$$

almost surely.

---

(iii) Fix $A\in {\Sigma }_1$. Then,

$$\begin{array}{rl}{\int }_A\mathbb{E}(\mathbb{E}(X\mid {\Sigma }_1)\mid {\Sigma }_0)d\mathbb{P}& ={\int }_A\mathbb{E}(X\mid {\Sigma }_1)d\mathbb{P}\\ & ={\int }_{A}Xd\mathbb{P}\\ & ={\int }_A\mathbb{E}(X\mid {\Sigma }_0)d\mathbb{P}\\ & ={\int }_A\mathbb{E}(\mathbb{E}(X\mid {\Sigma }_0)\mid {\Sigma }_1)d\mathbb{P},\end{array}$$

which demonstrates $(1.5)$.

---

(iv)

A well known property of convex functions states that $\varphi$ is convex if and only if it can be expressed as the supremum $\varphi (x)={\sup }_{L\in {\mathcal{L}}_{\varphi }}L(x),$ where ${\mathcal{L}}_{\varphi }$ is the collection of affine functions

$${\mathcal{L}}_{\varphi }:=\left\{L:\mathbb{R}\to \mathbb{R}:L(x)=ax+b\text{ for some }a,b\in \mathbb{R}\text{ and }L(x)\le \varphi (x)\forall x\in \mathbb{R}\right\}.$$

Then, by monotonicity and linearity,

$$\begin{array}{rl}\mathbb{E}(\varphi (X)\mid {\Sigma }_0)& =\mathbb{E}\left(\underset{L\in {\mathcal{L}}_{\varphi }}{\sup }L(X)\mid {\Sigma }_0\right)\\ & \ge \underset{L\in {\mathcal{L}}_{\varphi }}{\sup }\mathbb{E}(L(X)\mid {\Sigma }_0)\\ & =\underset{L\in {\mathcal{L}}_{\varphi }}{\sup }L(\mathbb{E}(X\mid {\Sigma }_0))\\ & =\varphi (\mathbb{E}(X\mid {\Sigma }_0)).\end{array}$$

---

(v) This follows from

$${\int }_{\Omega }\mathbb{E}(X)d\mathbb{P}=\mathbb{E}(X)={\int }_{\Omega }Xd\mathbb{P}.$$

---

(vi) From the tower property and $(v)$ we have

$$\begin{array}{rl}\mathbb{E}(\mathbb{E}(X\mid Y))& =\mathbb{E}(\mathbb{E}(X\mid Y)\mid \{\varnothing ,\Omega \})\\ & =\mathbb{E}(X\mid \{\varnothing ,\Omega \})\\ & =\mathbb{E}(X).\end{array}$$

---

(vii)

This relation will first be proved for characteristic functions. Let $Y=1_A$ for some $A\in {\Sigma }_0$. Then for $B\in {\Sigma }_0$,

$$\begin{array}{rl}{\int }_{B}Y\cdot \mathbb{E}(X\mid {\Sigma }_0)d\mathbb{P}& ={\int }_{A\cap B}\mathbb{E}(X\mid {\Sigma }_0)d\mathbb{P}\\ & ={\int }_{A\cap B}Xd\mathbb{P}\\ & ={\int }_B{1}_{A}Xd\mathbb{P}\\ & ={\int }_B\mathbb{E}(YX\mid {\Sigma }_0)d\mathbb{P}.\end{array}$$

It then follows from linearity that $\mathbb{E}(YX\mid {\Sigma }_0)=Y\cdot \mathbb{E}(X\mid {\Sigma }_0)$ when $Y$ is a ${\Sigma }_0$-simple function. That is, $Y$ is of the form

$$\sum _{j=1}^N{c}_j{1}_{A_j},$$

with $c_j\in \mathbb{R}$ and $A_j\in {\Sigma }_0$ for each $1\le j\le N$. Suppose that $Y$ is a general ${\Sigma }_0$-measurable function. It is well known that $Y$ can be approximated by ${\Sigma }_0$-simple functions in the $L^2$-norm. Fix $\epsilon >0$. Then there must exist a ${\Sigma }_0$-simple function, $\tilde{Y}$, such that $\Vert Y-\tilde{Y}{\Vert }_2<\epsilon .$ Then for $B\in {\Sigma }_0$ and $X\in L^2(\Omega ,\Sigma ,\mathbb{P})$,

$$\begin{array}{rl}\left|{\int }_B\mathbb{E}(YX\mid {\Sigma }_0)d\mathbb{P}-{\int }_{B}Y\mathbb{E}(X\mid {\Sigma }_0)d\mathbb{P}\right|& \le \left|{\int }_B\mathbb{E}((Y-\tilde{Y})X\mid {\Sigma }_0)d\mathbb{P}\right|\\ & +\left|{\int }_B\mathbb{E}(\tilde{Y}X\mid {\Sigma }_0)-\tilde{Y}\mathbb{E}(X\mid {\Sigma }_0)d\mathbb{P}\right|\\ & +\left|{\int }_B(\tilde{Y}-Y)\mathbb{E}(X\mid {\Sigma }_0)d\mathbb{P}\right|.\end{array}$$

From the previous argument, the second term in the above estimate is zero. On successively applying Cauchy—Schwarz and Jensen’s inequality,

\begin{aligned} & \left| \int_B \mathbb{E}(YX\mid\Sigma_0)\,d\mathbb{P} - \int_B Y\mathbb{E}(X\mid\Sigma_0)\,d\mathbb{P} \right|\\ &\leq \mathbb{E}\bigl(|Y-\widetilde{Y}|\,|X|\bigr) + \int_\Omega |\widetilde{Y}-Y| \,\mathbb{E}(X\mid\Sigma_0)\,d\mathbb{P}\\ &\leq \mathbb{E}\left(|\widetilde{Y}-Y|^2\right)^{1/2} \|X\|_2 + \mathbb{E}\left(|\widetilde{Y}-Y|^2\right)^{1/2} \left( \int_\Omega \mathbb{E}(X\mid\Sigma_0)^2 \right)^{1/2}\\ &\leq \mathbb{E}\left(|\widetilde{Y}-Y|^2\right)^{1/2} \|X\|_2 + \mathbb{E}\left(|\widetilde{Y}-Y|^2\right)^{1/2} \left( \int_\Omega \mathbb{E}(X^2\mid\Sigma_0) \right)^{1/2}\\ &\leq 2\|\widetilde{Y}-Y\|_2\|X\|_2\\ &\leq 2\varepsilon\|X\|_2. \end{aligned} $$As $\varepsilon$ can be made arbitrarily small, it follows that $\mathbb{E}(YX\mid\Sigma_0)= Y\,\mathbb{E}(X\mid\Sigma_0)$ for any $\Sigma_0$-measurable $Y$. --- ### (viii) This is a consequence of $(iv)$ and the density of simple functions in $L^1$. >[!proposition] Independence > >Let$X\in L^{2}(\Omega,\Sigma,P)$ and $\Sigma_{0}$ be a sub-sigma-algebra of $\Sigma$. Showthat if $X$is independent of Σ0 then $\mathbb{E}(X\mid\Sigma_0)=\mathbb{E}(X)$