Inner Product Space and Fourier Series

Inner Products Spaces

An inner product $\langle \cdot, \cdot\rangle :V\rightarrow \mathbb R$ is a function that satisfies

• positive definite $\langle x,x\rangle > 0$ and $\langle x,x\rangle =0$ IFF $x\equiv0$
• symmetry $\langle x,y\rangle =\langle y,x\rangle$
• bilinear $\langle ax+cy,z\rangle =a\langle x,z\rangle +c\langle y,z\rangle$

Example 1

On $\mathbb R^n, x\cdot y = \sum_1^n x_i y_i$

• $x\cdot x =\sum^n x_i^2 \geq 0$
• $x\cdot y = \sum x_i y_i = \sum y_i x_i = y\cdot x$
• $(ax+cy)\cdot z = \sum (ax_i + cy_i) z_i = a\sum x_i z_i + c\sum y_i z_i = ax\cdot z + cy\cdot z$

Example 2

Let $A_{n\times n}$ be a positive definite symmetric matrix, i.e. all its eigenvalues are strictly positive. Then $\langle x\cdot y\rangle _A = x A y^T$ is an inner product on $\mathbb R^n$

Note that such $A$ can be diagonalized into $A=PDP^T$

• $\langle x,x\rangle _A = xAx^T = xPDPx^T = vDv^T = \sum v_k^2 \lambda_k \rangle 0$ (since $A$ is definite, $\lambda_k \rangle 0$)
• $\langle x,x\rangle =0$ IFF $\sum v_k^2 \lambda_k = 0\Rightarrow v_k = 0 \Rightarrow xP = 0$, since $P$ is orthogonal, $x=0$

Thrm 1. Cauchy Schwarz inequality

For $x,y\in (V,\langle ,\rangle ), |\langle x,y\rangle |\leq \langle x,x\rangle ^2\langle y,y\rangle ^2$

proof. By positive definite, for $t\in\mathbb R$, $\langle x-ty, x-ty\rangle \geq 0$.
Take $t = \frac{\langle x,y\rangle }{y,y}$,

$$\begin{align*} \langle x,x\rangle - 2t |\langle x,y\rangle | + t^2 \langle y,y\rangle &= \langle x,x\rangle - 2\frac{\langle x,y\rangle ^2}{\langle y,y\rangle} + \frac{\langle x,y\rangle ^2}{\langle y,y\rangle }\\ =&\langle x,x\rangle - \frac{\langle x,y\rangle ^2}{\langle x,y\rangle } \geq 0\\ \implies &\langle x,y\rangle ^2 \leq \langle x,x\rangle \langle y,y\rangle\\ \end{align*}$$

Example 3

Over $C[a,b]$, $\langle f,g\rangle _{L^2} = \int_a^b f g dx$

• $\langle f,f\rangle _{L^2} = \int_a^b f^2 dx \geq 0$
• $\langle f,f\rangle _{L^2}=0$ IFF $\int_a^b |f|^2 dx = 0$ IFF $f(x)=0$ because $f$ is continuous.

Assume $f(x_0) > 0$, then by continuity, take $I_\delta(x_0):=[x_0-\delta, x_0+\delta], \forall x\in I_\delta(x_0). f(x) > \epsilon > 0$ so that $\int_a^b |f|^2 dx > \int_{I_\delta(x_0)} |f(x)|^2 dx > \epsilon^2$ contradicts with $\langle f,f\rangle _{L^2}=0$ (other details to be filled)

By Cauchy Schwartz inequality,

$$\begin{align*} &\int fg\leq \sqrt{\int f^2}\sqrt{\int g^2}\\ \implies &\langle f,g\rangle _{L^2} \leq \|f\|_{L^2}\|g\|_{L^2}\\ \implies &f\in L^2([a,b])\\ \implies &f\in L^1([a,b])\\ \end{align*}$$

Therefore, every inner product gives rise to a norm $\|x\|_v := \sqrt{\langle x,x\rangle }$

Def'n. Hilbert space

a complete inner product space is called Hilbert space

Completeness over inner product space A inner product space is complete if every Cauchy sequence $x_n$ has a limit in the space. i.e.

$$\|x_n - x\|_V^2 = \langle x_n - x, x_n - x\rangle \rightarrow 0$$

$$L^2([a,b]):= \{f:[a,b]\rightarrow \mathbb R: \int_a^b |f|^2 \langle \infty\}$$

Remark The space $S:=(C[a,b], \|\cdot\|_{L^2})$ is not complete, but $L^2([a,b])$ is the completion of $S$. which means taking any Cauchy sequence, then it has a limit in $L^2$. In another words, continuous functions can approximate any square integrable function in $L^2$-norm.

Thrm 2. Monotone Convergence Theorem

Take nonnegative $f_n(x)\geq 0$, if $f_1(x)\leq f_2(x) \leq \cdots$ then point-wise limit $f_n(x)\rightarrow f(x)$ i.e. $|f_n(x)-f(x)|\rightarrow 0$

Corollary By MCT, $\lim \int f_n = \int \lim f_n = \int f$

Thrm 3. Dominated/Integral Convergence Theorem

If $\{h_n\}$ satisfy

• $|h_n(x)|\leq B(x)$, which $B$ is an integrable function
• $h_n(x)\rightarrow^{\text{point-wise}} h(x)$

Then, $\lim \int f_n = \int \lim f_n = \int f$

Thrm 4. Completeness

The space $(L^2, \|\cdot\|_{L^2})$ is complete

proof. Take Cauchy sequence $f_n \in L^2([a,b])$. By Cauchy, take subsequence $f_{n,k}$ s.t. $\|f_{n,k} - f_{n,k+1}\| \leq 2^{-k}$.
Let $f(x):=f_{n,1}(x) + \sum (f_{n, k+1} - f_{n,k}(x))$. WTS $f\in L^2 (i)$, $f_{n,k}\rightarrow^{L^2} f (ii)$.
Define $g:= |f_n,1| + \sum|f_{n,k+1}- f_{n,k}|$, $S_M(g) = |f_n,1| + \sum^M|f_{n,k+1}- f_{n,k}|$, then $f\leq g, S_M(f) \leq S_M(g)$
Apply MCT to $\{S_M(g)\}, 0\leq |f_{n,1}| \leq |f_{n,2}-f_{n,1}|\leq ..., S_1(g) \leq S_2(g) \leq S_3(g)$
Then, WTS $S_M(g)\rightarrow g(x)$, which suffices to show that the sequence $g$ converges, a.k.a. $g\in L^2$

$$\begin{align*} \|S_M(g)\|_{L^2} &\leq \|f_{n,1}\|_{}L^2 + \sum^M \|f_{n,k+1} - f_{n,k}\| \\ &\leq \|f_{n,1}\| + \sum^M 2^{-k} &\text{tri. ineq.}\\ &<\infty \end{align*}$$

Thus we have that

$$\begin{align*} \int |g^2| dx &= \int \lim_{M\rightarrow \infty} |S_M(g)^2|dx\\ &=\lim_{n\rightarrow\infty} \int |S_M(g)|^2 dx &\text{MCT}\\ &\leq \lim_{M\rightarrow \infty} \|f_{n,1}\| + \sum^M 2^{-k} \leq \|f_{n,1}\| + 2 \\ &< \infty \end{align*}$$

We have Cauchy seuqnce $f_n \in L^2$, WTS $\|f_n -f\|_{L^2}\rightarrow 0$

(i) Since $|f|\leq g$ and $g\in L^2, \int |f|^2 \leq \int |g|^2 < \infty\implies f\in L^2$

(ii) $\|f_n - f\|_{L^2}^2 = \int |f_n - f|^2 dx\rightarrow 0$.

Let $h_k:=|f_{n,k}- f|^2$, then

$$h_k \leq |S_k(g)| + |g| \leq 2|g|\in L^2$$

so that $f_{n,k}\rightarrow^{\text{p.w.}} f$ since $g\in L^2$ so $\lim_{k\rightarrow \infty} f_{n,k}(x)=f(x)$ exists
BY DCT, $\lim_{k\rightarrow \infty} \int |h_k|^2 = \int \lim_{k\rightarrow \infty} |h_k|^2 =0$,

Therefore, $f_{n,k}\rightarrow^{L^2}f$

Then, expansion $f_{n,k}$ to $f_n$. i.e. $f_n \rightarrow^{L^2} f$.
By Cauchy sequence of $f_n$, if the sub-sequence converges to some limit, so that the whole sequence will do so.

Fourier Series

Def'n. Orthogonal

Given $x,y\ in (V,\langle\cdot, \cdot\rangle)$, x,y are orthogonal if $\langle x,y\rangle = 0$

Example

$x\circ y = \cos(\theta) |x||y|$, so when $\theta = \pi /2, x\circ y = 0$

Def'n. Orthogonal set

A collection of $\{e_k\}$ is called orthonormal if $\langle e_k,e_m\rangle = \mathbb I(k=m)$

Thrm 4.

Every Hilbert space has orthonormal basis, i.e. $\forall x\in H. x = \sum^\infty \langle x,e_k\rangle e_k$

Example

Take $e_k = (0,...,0,1,0)$ where $1$ is at $k$th position, $\{e_k\}$ is orthonormal set.

proof. $e_k \cdot e_m = \sum^n (e_k)_i (e_m)_i = 1\mathbb I(k=m)$

Example

$\{1, \sqrt2\cos(n\theta), \sqrt2\sin(n\theta)\}$ is orthogonal for $(C([-\pi, \pi]), \|\cdot\|_{L^2})$

proof. We have the followings hold

$$\begin{align*} &\frac{\sqrt2}{\pi}\int_{-\pi}^\pi \cos(n\theta)\cos(k\theta) = 1\mathbb I(n=k)\\ &\frac{\sqrt2}{\pi}\int_{-\pi}^\pi \sin(n\theta)\sin(k\theta) = 1\mathbb I(n=k)\\ &\frac{\sqrt2}{\pi}\int_{-\pi}^\pi \sin(n\theta)\cos(k\theta) = 0 \end{align*}$$

Def'n. Fourier Series

For function $f\in L^2([a,b])$, the Fourier series is defined to be

$$f(x):= a_0 + \sum a_n \frac{\cos(n\pi x)}{L} + b_n \frac{\sin(n\pi x)}{L}$$

where

$$a_n = \langle f, \frac{\cos(n\pi x)}{L})\rangle _{L^2}, b_n = \langle f, \frac{\sin(n\pi x)}{L}\rangle _{L^2}$$

Thrm. Carleson Hunt Theorem

If $f\in L^2([-L,L])$, then $f$ equals its Fourier series almost everywhere.
almost everywhere outside a measure zero set