08. Inner Product Spaces

8.1 Bilinear forms

Bilinear form

Let $V$ be a vector space over $\mathbf{F}$
A bilinear form $B$ on $V$ is a function $B:V\times V\to \mathbf{F}$ such that

1. $B({\alpha }_1{v}_1+{\alpha }_2{v}_2,v_3)={\alpha }_{1}B(v_1,v_3)+{\alpha }_{2}B(v_2,v_3)\text{ for all }{v}_1,v_2,v_3\in V\text{ and }{\alpha }_1,{\alpha }_2\in \mathbf{F}$
2. $B(v_1,{\alpha }_2{v}_2+{\alpha }_3{v}_3)={\alpha }_{2}B(v_1,v_2)+{\alpha }_{3}B(v_1,v_3)\text{ for all }{v}_1,v_2,v_3\in V\text{ and }{\alpha }_2,{\alpha }_3\in \mathbf{F}$

$(1)$ is linear in the first variable and $(2)$ is linear in the second variable

Link to original

Dot Product

For $\mathbf{x}=(x_1,\cdots ,x_n)\in {\mathbb{F}}^n$ and $\mathbf{y}=(y_1,\cdots ,y_n)\in {\mathbb{F}}^n$
Then we define

$$B(\mathbf{x},\mathbf{y})=x_1{y}_1+\cdots +x_n{y}_n=\mathbf{x}{\mathbf{y}}^T$$

Link to original

Gram Matrix

Let $V$ be a vector space over $\mathbf{F}$
Let $B$ be a bilinear form om $V$

For $v_1,\cdots ,v_n\in V$
The Gram matrix of $B$ with respect to $v_1,\cdots ,v_n$ is the $n\times n$ matrix

$$B_{GM}=(B(v_i,v_j))\in {\mathcal{M}}_{n\times n}(\mathbf{F})$$

That is the $(i,j)th\text{ coordinate of }{B}_{GM}\text{ is }B(v_i,v_j)$

Link to original

Bilinear Form of a Gram Matrix

Let $V$ be a finite-dimensional vector space over $\mathbf{F}$
Let $v_1,\cdots ,v_n$ be a basis for $V$
Let $B$ be a bilinear form on $V$
Let $A\in {\mathcal{M}}_{n\times n}(\mathbf{F})$ be the associated Gram matrix

For $X,Y\in V$
Let $\mathbf{x}=(x_1,\cdots ,x_n)\in {\mathbb{F}}^n$
Let $\mathbf{y}=(y_1,\cdots ,y_n)\in {\mathbf{F}}^n$
where $\mathbf{x},\mathbf{y}$ are the unique coordinate vectors such that

$$X=x_1{v}_1+\cdots +x_n{v}_n\text{ and }Y=y_1{v}_1+\cdots +y_n{v}_n$$

Then

$$B(X,Y)=\mathbf{x}A{\mathbf{y}}^T$$

Proof

$$\begin{array}{rl}B(X,Y)& =B\left(\sum _{i=1}^n{x}_i{v}_i,\sum _{j=1}^n{y}_j{v}_j\right)\\ & =\sum _{i=1}^n{x}_{i}B\left(v_i,\sum _{j=1}^n{y}_j{v}_j\right)\\ & =\sum _{i=1}^n{x}_i\sum _{j=1}^n{y}_{j}B(v_i,v_j)\\ & =\sum _{i=1}^n\sum _{j=1}^n{x}_i{a}_{ij}{y}_j\\ & =xA{y}^T\end{array}$$

Link to original

![[36 - Types of Bilinear Forms#^6ada2c|^6ada2c]]

---

8.2 Inner product spaces

Positive Definite

Let $V$ be a RELA vector space
Let $B:V\times V\to \mathbb{R}$

$$B\text{ is positive definite}\text{ if }B(v,v)\ge 0\text{ for all }v\in V$$

and

$$B(v,v)=0⟺v=0$$

Note that it is with the same vector!

Link to original

Inner Product

Let $V$ be a real vector space $V$
An inner product is a positive definite, symmetric bilinear form on $V$

It is typically denoted as $⟨x,y⟩$ rather than $B(x,y)$

Note that $⟨-,-⟩$ will denote an inner product

Link to original

Norm / Magnitude / Length

Let $V$ be an inner product space
For $v\in V$ the norm of $v$ is

$$||v||:=\sqrt{⟨v,v⟩}$$

And then the distance between two vectors $v,w\in V$ is

$$d(v,w)=||v-w||$$

Link to original

Cauchy-Schwarz Inequality

For $v,w$ in an inner product space $V$ then

$$|⟨v,w⟩|\le ||v||||w||$$

Equality holds if $v$ and $w$ are linearly independent

Proof

If $w=0$ then the result is obvious so assume $w\ne 0$
For $t\in \mathbb{R}$ then

$$\begin{array}{rl}0& \le ||v+tw|{|}^2\\ & =⟨v+tw,v+tw⟩\\ & =⟨v,v⟩+2t⟨v,w⟩+t^2⟨w,w⟩\\ & =||v|{|}^2+2t⟨v,w⟩+t^2||w|{|}^2\end{array}$$

As $|w|\ne 0$ then it is a quadratic in $t$ which is non-negative so discriminant is non-negative
Hence

$$\text{discriminant}=4⟨u,w{⟩}^2-4||w|{|}^2||v|{|}^2\le 0$$

And then the Cauchy-Schwarz inequality follows
For equality then the discriminant has to be zero there is a repeated root $t=t_0$
Then $||v+t_{0}w||$ and hence $v+t_{0}w=0_V$
Therefore $v,w$ are linearly independent

Link to original

Properties of Norm

Let $V$ be a inner product space
For $v,w\in V$ and $\alpha \in \mathbb{R}$

1. $||v||\ge 0$ and $||v||=0⟺v=0_V$
2. $||\alpha v||=|a|||v||$
3. $||v+w||\le ||v||+||w||$ (also known as the triangle inequality)

Proof - (3)

$$\begin{array}{rl}||v+w|{|}^2=⟨v+w,v+w⟩& \\ & =⟨v,v⟩+2⟨v,w⟩+⟨w,w⟩\\ & \le ||v|{|}^2+2||v||||w||+||w|{|}^2\\ & =(||v||+||w||{)}^2\end{array}$$

Hence the triangle inequality follows

Link to original

Properties of the distance function

For $u,v,w\in V$

1. $d(v,w)\ge 0$ and $d(v,w)=0⟺v=w$
2. $d(u,w)=d(w,u)$
3. $d(u,w)\le d(u,v)+d(v,w)$

Link to original

---

8.3 Orthogonal Maps

Orthogonal Linear Map

Let $\alpha :V\to V$ be a linear map of a inner product space $V$

$$\alpha \text{ is orthogonal }\text{ if }⟨\alpha (v),\alpha (w)⟩=⟨v,w⟩$$

for all $v,w\in V$

Link to original

Orthogonal Matrices preserve dot product

Let $\alpha :{\mathbb{R}}_{\mathrm{col}}^n\to {\mathbb{R}}_{\mathrm{col}}^n$ be a linear map
Let $A$ denote the matrix of $\alpha$ with respect to the standard basis

$$A\text{ is orthogonal with respect to the dot product}⟺A\text{ is an orthogonal matrix}$$

Proof

1. $⟹$
   Suppose that $\alpha$ is orthogonal with respect to the dot product
   Denote the standard basis as ${\mathbf{e}}_1,\cdots ,{\mathbf{e}}_n$
   Then

$${\delta }_{ij}={\mathbf{e}}_i\cdot {\mathbf{e}}_j=\alpha ({\mathbf{e}}_i)\cdot \alpha ({\mathbf{e}}_j)=(A{\mathbf{e}}_i{)}^T(A{\mathbf{e}}_j)={\mathbf{e}}_i^T{A}^{T}A{\mathbf{e}}_j$$

As ${\mathbf{e}}_i^T{A}^{T}A{\mathbf{e}}_j$ is the $(i,j)$th entry of $A^{T}A$ and ${\delta }_{ij}$ is the $(i,j)$th entry of $I_n$ then

$$A^{T}A=I_n$$

2. $⟸$
   Suppose that $A$ is orthogonal then $A^{T}A=I_n$

$$\begin{array}{rl}\alpha \left(\sum _{i=1}^n{u}_i{\mathbf{e}}_i\right)\cdot \alpha \left(\sum _{j=1}^n{v}_j{\mathbf{e}}_j\right)& =\sum _{i=1}^n\sum _{j=1}^n{u}_i{v}_j\alpha ({\mathbf{e}}_i)\cdot \alpha ({\mathbf{e}}_j)\\ & =\sum _{i=1}^n\sum _{j=1}^n{u}_i{v}_j{\delta }_{ij}\\ & =\sum _{i=1}^n{u}_i{v}_i\\ & \\ & =\left(\sum _{i=1}^n{u}_i{\mathbf{e}}_i\right)\cdot \left(\sum _{j=1}^n{v}_j{\mathbf{e}}_j\right)\end{array}$$

Hence $\alpha$ is orthogonal

Link to original

Relation between Orthogonal Maps and Inner Product Spaces

An orthogonal map is an isometry of an inner product space

Proof

Suppose $\alpha$ is orthogonal then

$$d(\alpha (u),\alpha (w){)}^2=||\alpha (v-w)|{|}^2=⟨\alpha (v-w),\alpha (v-w)⟩=⟨v-w,v-w⟩=||v-w|{|}^2=d(v,w{)}^2$$

Hence $\alpha$ is an isometry

Link to original

Orthonormal Set

Let $V$ be an inner product space
If $\{v_1,\cdots ,v_k\}\subseteq V$ is a orthonormal set if

$$⟨v_i,v_j⟩={\delta }_{ij}$$

for all $i,j$
i.e. the vectors are unit length and mutually perpendicular

Link to original

Orthonormal Sets are linearly independent in a inner product space

In an inner product space $V$, a orthonromal sets is linearly independent

Proof

Suppose $\{v_1,\cdots ,v_k\}$ is orthonormal and ${\alpha }_1,\cdots ,{\alpha }_k\in \mathbb{R}$ such that

$${\alpha }_1{v}_1+\cdots +{\alpha }_k{v}_k=0_V$$

For $1\le i\le k$ we havea

$$\begin{array}{rl}0=⟨0_V,v_i⟩& =⟨{\alpha }_1{v}_1+\cdots +{\alpha }_k{v}_k,v_i⟩\\ & ={\alpha }_1⟨v_1,v_i⟩+\cdots +{\alpha }_k⟨v_k,v_i⟩\\ & ={\alpha }_i\end{array}$$

Hence ${\alpha }_1=\cdots ={\alpha }_k=0$

Link to original

Equivalent Properties of an Inner Product

For $X\in {\mathcal{M}}_{n\times n}(\mathbb{R})$
Let ${\mathbb{R}}^n$ be a inner product space using the dot product

1. $X{X}^T=I_n$
2. $X^{T}X=I_n$
3. Rows of $X$ form an orthonormal basis of ${\mathbb{R}}^n$
4. Columns of $X$ form an orthonormal basis of ${\mathbb{R}}_{\mathrm{col}}^n$
5. For all $\mathbf{x},\mathbf{y}\in {\mathbb{R}}_{\mathrm{col}}^n$ then $X\mathbf{x}\cdot X\mathbf{y}=\mathbf{x}\cdot \mathbf{y}$

Link to original

---

8.4 Complex inner product spaces

Sesquilinear form

Let $V$ be a complex vector space
A function $⟨-,-⟩:V\times V\to \mathbb{C}$ is a sesquilinear form if

1. $⟨{\alpha }_1{v}_1+{\alpha }_2{v}_2,v_3⟩=\overline{{\alpha }_1}⟨v_1,v_3⟩+{\alpha }_2⟨v_2,v_3⟩$ for all $v_1,v_2,v_3\in V$ and ${\alpha }_1,{\alpha }_2\in \mathbb{C}$
2. $⟨v_1,v_2⟩=\overline{⟨v_2,v_1⟩}\text{ for all }{v}_1,v_2\in V$

Link to original