06. Linear Transformations

6.1 What is a linear transformation?

Linear Map

Let $V,W$ be vector spaces over $\mathbf{F}$
Map $T:V\to W$ is linear if

1. Preserves additive structure

$$T(v_1+v_2)=T(v_1)+T(v_2)\text{ for all }{v}_1,v_2\in V$$

2. Preserves scalar multiplication

$$T(\lambda v)=\lambda T(v)\text{ for all }v\in V\text{ and }\lambda \in \mathbf{F}$$

How to show a map is linear $T$ needs to satisfy

Map

$$T({\lambda }_1{u}_1+{\lambda }_2{u}_2)={\lambda }_{1}T(u_1)+{\lambda }_{2}T(u_2)$$

Basically a combination of the additive structure and scalar multiplication at the same time!

Link to original

Preservation of $0$ in Linear Maps

Let $V,W$ be vector spaces over $\mathbf{F}$
Let $T:V\to W$ be a linear map then

$$T(0_V)=0_W$$

Proof

$$T(0_V)+T(0_V)=T(0_V+0_V)=T(0_V)$$

Hence

$$T(0_V)=0_W$$

Link to original

Equivalent Properties of Linear Maps

Let $V,W$ be vector spaces over $\mathbf{F}$
Let $T:V\to W$

1. $T$ is linear (aka linear map)
2. $T(\alpha v_1+\beta v_2=\alpha T(v_1)+\beta T(v_2)$ for all $v_1,v_2\in V$ and $\alpha ,\beta \in \mathbf{F}$
3. For any $n\ge 1$ for $v_1,\cdots ,v_n\in V$ and ${\alpha }_1,\cdots ,{\alpha }_n\in \mathbf{F}$ then

$$T({\alpha }_1{v}_1+\cdots +{\alpha }_n{v}_n)={\alpha }_{1}T(v_1)+{\alpha }_{n}T(v_n)$$

Link to original

Identity Map

Let $V$ be a vector space then
The identity map is $i{d}_V:V\to V$ is defined by

$$i{d}_V(v)=v\text{ for all }v\in V$$

It is also a linear map

Link to original

Zero Map

Let $V,W$ be vector spaces
The zero map is $0:V\to W$ is defined by

$$0(v)=0_W\text{ for all }v\in V$$

It is also a linear map
Notation may look confusing but the map $0$ is a function

Link to original

Left Multiplication Map

For $m,n\ge 1$ with $A\in {\mathcal{M}}_{m\times n}(\mathbb{R})$
Left multiplication map $L_A:{\mathbb{R}}_{\mathrm{col}}^n$ by

$$L_A(\mathbf{v})=A\mathbf{v}\text{ for }\mathbf{v}\in {\mathbb{R}}_{\mathrm{col}}^n$$

Link to original

Right Multiplication Map

For $m,n\ge 1$ with $A\in {\mathcal{M}}_{m\times n}(\mathbb{R})$
Left multiplication map $L_A:{\mathbb{R}}_{\mathrm{col}}^n$ by

$$L_A(\mathbf{v})=A\mathbf{v}\text{ for }\mathbf{v}\in {\mathbb{R}}_{\mathrm{col}}^n$$

Link to original

Multiplication Maps with Matrices

Take $m,n,p\ge 1$ with $A\in {\mathcal{M}}_{m\times n}(\mathbb{R})$
The left multiplication map ${\mathcal{M}}_{n\times p}(\mathbb{R})\to {\mathcal{M}}_{m\times p}(\mathbb{R})$ sends $X$ to $AX$ is also a linear map

Link to original

Projection Map

Let $V$ be a vector spaces over $\mathbf{F}$ with subspaces $U,W$ such that $V=U\oplus W$
For $v\in V$ there are unique $u\in U,w\in W$ such that $v=u+w$
Then $P$ is defined as

$$P:V\to V\text{ by }P(v)=w$$

Where $P$ is the projection of $V$ onto $W$ along $U$

Proof that $P$ is a linear map

Take $v_1,v_2\in V$ and ${\alpha }_1,{\alpha }_2\in \mathbf{F}$
There are $u_1,u_2\in U,w_1,w_2\in W$ such that

$$v_1=u_1+w_1\text{ and }{v}_2=u_2+w_2$$

With

$$\begin{array}{rl}{\alpha }_1{v}_1+{\alpha }_2{v}_2& ={\alpha }_1(u_1+w_1)+{\alpha }_2(u_2+w_2)\\ & =({\alpha }_1{u}_1+{\alpha }_2{u}_2)+({\alpha }_1{w}_1+{\alpha }_2{w}_2)\end{array}$$

where ${\alpha }_1{u}_1+{\alpha }_2{u}_2\in U$ and ${\alpha }_1{w}_1+{\alpha }_2{w}_2\in W$
Hence by uniqueness

$$P({\alpha }_1{v}_1+{\alpha }_2{v}_2)={\alpha }_1{w}_1+{\alpha }_2{w}_2={\alpha }_{1}P(v_1)+{\alpha }_{2}P(v_2)$$

Link to original

Trace $A=(a_{ij})\in {\mathcal{M}}_{n\times n}(\mathbb{R})$

For

$$\text{trace}(A):=a_{11}+a_{22}+\cdots +a_{nn}$$

The sum of the entries on the main diagonal of $A$

Trace Linear Map

$$\text{trace}:{\mathcal{M}}_{n\times n}(\mathbb{R})\to \mathbb{R}$$

is a linear map

Link to original

Useful example of a Linear Maps

Let ${\mathbb{R}}_n[x]$ be the vector space of polynomials of degree at most $n$
Define $D:{\mathbb{R}}_n[x]\to {\mathbb{R}}_n[x]$ by $p(x)\mapsto p^{\prime }(x)$
that is

$$D(a_n{x}^n+\cdots +a_{1}x+a_0)=n{a}_n{x}^{n-1}+\cdots +a_1$$

It is a linear map from ${\mathbb{R}}_n[x]$ to ${\mathbb{R}}_n[x]$
Could also be a linear map from ${\mathbb{R}}_n[x]$ to ${\mathbb{R}}_{n-1}[x]$

Link to original

---

6.2 Combining Linear Transformations

Combining Linear Transformations (addition + scalar multiplication)

Let $V,W$ be vector spaces over a field $\mathbf{F}$
For $S,T:V\to W$ and $\lambda \in \mathbb{F}$ then linear maps $S+T$ and $\lambda S$ are defined by

$$\begin{array}{rlr}S+T:V\to W& \text{ by }(S+T)(v)=S(v)+T(v)& \text{ for }v\in V\\ \lambda S:V\to W& \text{ by }(\lambda S)(v)=\lambda S(v)& \text{ for }v\in V\end{array}$$

With these operations (including the Zero map) the set of linear transformations $V\to W$ forms a vector space denoted $\text{Hom}(V,W)$

Proof for Linear Maps

Showing that $S+T$ is a linear map

$$\begin{array}{rl}(S+T)({\alpha }_1{v}_1+{\alpha }_2{v}_2)& =S({\alpha }_1{v}_1+{\alpha }_2{v}_2)+T({\alpha }_1{v}_1+{\alpha }_2{v}_2)[\text{by definition}]\\ & ={\alpha }_{1}S(v_1)+{\alpha }_{2}S(v_2)+{\alpha }_{1}T(v_1)+{\alpha }_{2}T(v_2)[\text{by linearity}]\\ & ={\alpha }_1(S(v_1)+T(v_2))+{\alpha }_2(S(v_2)+T(v_2))[\text{rearranging}]\\ & ={\alpha }_1(S+T)(v_1)+{\alpha }_2(S+T)(v_2)[\text{by defintion}]\end{array}$$

Hence $S+T$ is linear
Showing that $\lambda S$ is a linear map

$$\begin{array}{rl}(\lambda S)({\alpha }_1{v}_1+{\alpha }_2{v}_2)& =\lambda S({\alpha }_1{v}_1+{\alpha }_2{v}_2)\\ & =\lambda ({\alpha }_{1}S(v_1)+{\alpha }_{2}S(v_2))\\ & ={\alpha }_1(\lambda S(v_1))+{\alpha }_2(\lambda S(v_2))\\ & ={\alpha }_1(\lambda S)(v_1)+{\alpha }_2(\lambda S)(v_2)\end{array}$$

Hence $\lambda S$ is linear

Link to original

Combining Linear Transformations (composition)

Let $U,V,W$ be vector spaces over $\mathbf{F}$
Let $S:U\to V$ and $T:V\to W$ be linear then

$$T\circ S:U\to W\text{ by }(T\circ S)(v)=T(S(v))$$

Note on notation

Sometimes $T\circ S$ is written as $TS$ but $T\circ S$ is removes any ambiguity

Proof for Linearity

Take $u_1,u_2\in U$ and ${\lambda }_1,{\lambda }_2\in \mathbb{F}$ then

$$\begin{array}{rl}(T\circ S)({\lambda }_2{u}_1+{\lambda }_2{u}_2)& =T(S({\lambda }_1{u}_1+{\lambda }_2{u}_2))\\ & =T({\lambda }_{1}S(u_1)+{\lambda }_{2}S(u_2))\\ & ={\lambda }_{1}T(S(u_1))+{\lambda }_{2}T(S(u_2))\\ & ={\lambda }_1(T\circ S)(u_1)+{\lambda }_2(T\circ S)(u_2)\end{array}$$

Hence $T\circ S$ is linear

Link to original

Invertible Linear Maps

Let $V,W$ be vector spaces and let $T:V\to W$ be linear
$T$ is invertible if there is linear transformation $S:W\to V$ such that

$$ST=i{d}_V\text{ and }TS=i{d}_W$$

where $i{d}_V$ and $i{d}_W$ are the identity maps on $V,W$ respectively
Then $S$ is called the inverse of $T$ (typically written as $T^{-1}$)

Note that invertible linear maps are an isomorphism (refer to groups)

Link to original

Property of Bijective Linear Maps

Let $V,W$ be vector spaces
Let $T:V\to W$ be linear

$$T\text{ is invertible }⟺T\text{ is bijective}$$

Proof

If $T$ is invertible, then it is certainly bijective (from Intro to Uni Maths)

Assume that $T$ is bijective
Then $T$ has an inverse function $S:W\to V$ (but we need to show that $S$ is linear)
For $w_1,w_2\in W$ and ${\lambda }_1,{\lambda }_2\in \mathbb{F}$
Let $v_1=S(w_1)$ and $v_2=S(w_2)$ then

$$T(v_1)=TS(w_1)=w_1\text{ and }T(v_2)=TS(w_2)=w_2$$

Hence

$$\begin{array}{rl}S({\lambda }_1{w}_1+{\lambda }_2{w}_2)& =S({\lambda }_{1}T(v_1)+{\lambda }_{2}T(V_2))\\ & =S(T({\lambda }_1{v}_1+{\lambda }_2{v}_2))[\text{since }T\text{ is linear}]\\ & ={\lambda }_1{v}_1+{\lambda }_2{v}_2[\text{as }S\text{ is inverse to }T]\\ & ={\lambda }_{1}S(W_1)+{\lambda }_{2}S(w_2)\end{array}$$

So $S$ is linear

Link to original

Inverse of Compositive Linear Maps

Let $U,V,W$ be vector spaces
Let $S:U\to V$ and $T:V\to W$ be invertible linear transformations
Then

$$TS:U\to W\text{ is invertible}\text{ and}(TS{)}^{-1}=S^{-1}{T}^{-1}$$

Link to original

Dimension Property of Invertible Linear Maps

1. Let $V,W$ be vector spaces with $V$ finite-dimensional
   If there is an invertible linear map $T:V\to W$ then $\dim V=\dim W$
2. Let $V,W$ be finite-dimsional vector spaces with $\dim V=\dim W$
   Then there is invertible linear map $T:V\to W$

Consequently $V$ and $W$ are isomorphic if and only if $\dim V=\dim W$

Proof

Let $v_1,\cdots ,v_n$ be a basis for $V$
For $v\in V$ it can be uniquely expressed as

$$v={\alpha }_1{v}_1+\cdots +{\alpha }_n{v}_n$$

for some ${\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_n$

Suppose $T(v_1),\cdots ,T(v_n)$ is a basis for $W$

As $T:V\to W$ is invertible then for $w\in W$ there exists $v\in V$ such that $T(v)=w$
So

$$w=T(v)=T({\alpha }_1{v}_1+\cdots +{\alpha }_n{v}_n)={\alpha }_{1}T(v_1)+\cdots +{\alpha }_{n}T(v_n)$$

Hence $T(v_1),\cdots ,T(v_n)$ is spanning

Finding a solution to ${\alpha }_{1}T(v_1)+\cdots +{\alpha }_{n}T(v_n)=0_W$

$$\begin{array}{rl}{\alpha }_{1}T(v_1)+\cdots +{\alpha }_{n}T(v_n)& =0_W\\ T({\alpha }_1{v}_1+\cdots +{\alpha }_n{v}_n)& =0_W\\ {\alpha }_1{v}_1+\cdots +{\alpha }_n{v}_n& =0_V\end{array}$$

Hence ${\alpha }_1={\alpha }_2=\cdots ={\alpha }_n=0$ as $v_1,\cdots ,v_n$ is a basis for $V$
Hence $T(v_1),\cdots ,T(v_n)$ is linearly independent
So $T(v_1),\cdots ,T(v_n)$ is a basis for $W$

Hence $\dim W=n=\dim V$

Suppose $n=\dim V=\dim W$
Let $v_1,\cdots ,v_n$ be a basis for $V$ and $w_1,\cdots ,w_n$ a basis for $W$
We need to show that

$$T:V\to W\text{ given by }T\left(\sum a_i{v}_i\right)=\sum {\alpha }_i{w}_i$$

is a well-defined, invertible linear map
TODO =_=

Link to original

---

6.3 Rank and nullity

Kernel / Null Space

Let $V,W$ be vector spaces
Let $T:V\to W$ be linear

$$\ker T:=\{v\in V\mid T(v)=0_W\}$$

In other words, $\ker T$ contains vectors that are sent to $0_W$ by the map $T$

Link to original

Image

Let $V,W$ be vector spaces
Let $T:V\to W$ be linear

$$\mathrm{Im}T:=\{T(v)\mid v\in V\}$$

In other words, $\ker T$ contains vectors that are sent to $0_W$ by the map $T$

Link to original

Properties of Rank and Kernel / Null Space

Let $V,W$ be vector spaces of $V$ over $\mathbf{F}$
Let $T:V\to W$ be linear

1. $\ker T$ is a subspace of $V$ and $\mathrm{Im}T$ is a subspace of $W$

2. $T$ is injective if and only if $\ker T=\{0_V\}$

3. If $A$ is a spanning set of $V$, then $T(A)$ is a spanning set of $\mathrm{Im}T$

4. If $V$ is finite-dimensional, then $\ker T$ and $\mathrm{Im}T$ are finite-dimensional

Proof (Image)

![[property-of-rank-and-kernel.png]]

Link to original

Row Space and Column Space corollary

Given a matrix $A$
Image of $L_A$ is $\text{Col}(A)$, the column space of $A$
Image of $R_A$ is $\text{Row}(A)$, the row space of $A$

Proof

The Canonical Basis ${\mathbf{e}}_1,\cdots ,{\mathbf{e}}_m$ spans ${\mathbb{R}}^m$
Hence the rows of $A$, ${\mathbf{r}}_1={\mathbf{e}}_{1}A,\cdots ,{\mathbf{r}}_m={\mathbf{e}}_{m}A$ spans $\mathrm{Im}{R}_A$
Hence $\mathrm{Im}{R}_A\subseteq \mathrm{Row}(A)$

Conversely if $\mathbf{v}\in \mathrm{Row}(A)$ then

$$\mathbf{v}={\alpha }_1{\mathbf{r}}_1+\cdots +{\alpha }_m{\mathbf{r}}_m=({\alpha }_1{\mathbf{e}}_{1+\cdots +{\alpha }_m{\mathbf{e}}_m})\in \mathrm{Im}{R}_A$$

Similarly then $\mathrm{Im}{L}_A=\mathrm{Col}(A)$

Link to original

Nullity

Let $V,W$ be vector spaces with $V$ finite-dimensional
Let $T:V\to W$ be linear

$$\mathrm{nullity}(T):=\dim (\ker T)$$

Link to original

Rank

Let $V,W$ be vector spaces with $V$ finite-dimensional
Let $T:V\to W$ be linear

$$\mathrm{rank}(T):=\dim (\mathrm{Im}T)$$

Link to original

07 - Rank-Nullity Theorem

Rank-Nullity Theorem

Let $V,W$ be vector spaces with $V$ finite-dimensional
Let $T:V\to W$ be linear

Then

$$\dim V=\mathrm{rank}(T)+\mathrm{nullity}(T)$$

Proof (Main)

Let $v_1,\cdots ,v_n$ be a basis for $\ker T$, where $n=\mathrm{nullity}(T)$
Extend this to a basis $v_1,\cdots ,v_n,v_1^{\prime },\cdots ,v_r^{\prime }$ so that $\dim V=n+r$
We claim that $B=\{T{v}_1^{\prime },\cdots ,T{v}_r^{\prime }\}$ is a basis for $\mathrm{Im}T$

1. $B$ spans $\mathrm{Im}T$
   By Properties of Rank and Kernel (3), $T(v_1),\cdots ,T(v_n),T(v_1^{\prime }),\cdots ,T(v_r^{\prime })$ spans $\mathrm{Im}T$
   But $v_1,\cdots ,v_n\in \ker T$, meaning $T(v_1)=\cdots =T(v_n)=0_W$
   so in fact $T{v}_1^{\prime },\cdots ,T{v}_r^{\prime }$ span $\mathrm{Im}T$
2. $B$ is linearly independent
   Take ${\alpha }_1,\cdots ,{\alpha }_r\in \mathbf{F}$ such that

$${\alpha }_{1}T(v_1^{\prime })+\cdots +{\alpha }_{r}T(v_r^{\prime })=0_W$$

As $T$ is linear then we can rewrite this to

$$T({\alpha }_1{v}_1^{\prime }+\cdots +{\alpha }_r{v}_r^{\prime })=0_W$$

So by definition then

$${\alpha }_1{v}_1^{\prime }+\cdots +\alpha v_r^{\prime }\in \ker T$$

As $v_1,\cdots ,v_n$ is a basis for $\ker T$, there are ${\beta }_1,\cdots ,{\beta }_n\in \mathbf{F}$ such that

$${\alpha }_1{v}_1^{\prime }+\cdots +{\alpha }_r{v}_r^{\prime }={\beta }_1{v}_1+\cdots +{\beta }_n{v}_n$$

As $v_1,\cdots ,v_n,v_1^{\prime },\cdots ,v_r^{\prime }$ is a basis then it is linearly independent hence

$${\alpha }_1=\cdots ={\alpha }_r={\beta }_1=\cdots ={\beta }_n=0$$

Hence $T{v}_1^{\prime },\cdots ,T{v}_r^{\prime }$ is linearly independent

So $B$ is a basis

So $\mathrm{rank}(T)=r$ and so $\dim (V)=n+r=\mathrm{nullity}(T)+\mathrm{rank}(T)$

Proof (Using Matrices)

Let $A$ be a $m\times n$ matrix and consider it’s reduced form $\mathrm{RRE}(A)$
There are as many leading $1s$ as there are non-zero rows, that is the row rank of $A$
The kernel of $A$ can be parameterised by assigning parameters to every column without a leading 1
Hence

$$\begin{array}{rl}\text{row rank of }A& =\text{number of leading }1s\\ \text{nullity of }A& =\text{number of leaing columns without leading }1s\end{array}$$

Therefore

$$\begin{array}{rl}n& =\dim {\mathbb{R}}_{col}^n\\ & =(\text{number of leading }1s)+(\text{number of columns without leading }1s)\\ & =(\text{row rank of }A)+(\text{nullity of}A)\end{array}$$

Link to original

Equivalent Statements for Rank and Nullity corollary

Let $V$ be a finite-dimensional vector space
Let $T:V\to V$ be linear

1. $T$ is invertible

2. $\mathrm{rank}(T)=\dim V$

3. $\mathrm{nullity}(T)=0$

Proof

$(1)⟹(3)$
Assume that $T$ is invertible.
Then $T$ is injective and hence surjective, so $\mathrm{Im}T=V$ hence $\mathrm{rank}(T)=\dim V$

$(2)⟹(3)$
Assume that $\mathrm{rank}(T)=\dim V$ so by Rank-Nullity Theorem, then $\mathrm{nullity}(T)=0$

$(3)⟹(1)$
Assume that $\mathrm{nullity}(T)=0$ so that $\ker T=\{0_V\}$ then $T$ is injective
Also, by Rank-Nullity, $\mathrm{rank}(T)=\dim V$ and $\mathrm{Im}T\le V$ so $\mathrm{Im}T=V$, so $T$ is surjective
So $T$ is bijective, so $T$ is invertible

Link to original

Uniqueness of Inverses of linear maps corollary

Let $V$ be a finite-dimensional vector space
Let $T:V\to V$ be linear
Then any one-sided inverse of $T$ is a two-sided inverse and hence the inverse of $T$ is unique

Proof

Suppose $T$ has a right inverse $S:V\to V$ so $T\circ S=i{d}_V$
Since $i{d}_V$ is surjective, $T$ is surjective, so $\mathrm{rank}(T)=\dim V$
So by Equivalent Statements for Rank and Nullity then $T$ is invertible
Suppose $T$ has a two-sided inverse $S^{\prime }$
Then

$$S^{\prime }=S^{\prime }\circ i{d}_V=S^{\prime }\circ (T\circ S)=(S^{\prime }\circ T)\circ S=i{d}_V\circ S=S$$

Hence $S$ is the unique two sided inverse

Link to original

Invertibility of Matrices from Inversible Product corollary

Let $A,B$ be square matrices of the same size
If $AB$ is invertible then

$$A\text{ is invertible and }B\text{ is invertible}$$

Link to original

Dimension Inequality for Linear Maps lemma

Let $V,W$ be vector spaces, with $V$ finite-dimensional
Let $T:V\to W$ be linear and $U\le V$ then

$$\dim U-\mathrm{nullity}(T)\le \dim T(U)\le \dim U$$

In particular, if $T$ is injective then $\dim T(U)=\dim U$

Proof

Let $S:U\to W$ be the restriction of $T$ to $U$ (that is, $S(u)=T(u)$ for all $u\in U$)
Then $S$ is linear, and $\ker S\le \ker T$ so $\mathrm{nullity}(S)\le \mathrm{nullity}(T)$
And $\mathrm{Im}S=T(U)$
By Rank-Nullity Theorem then

$$\begin{array}{rl}\dim T(U)& =\dim \mathrm{Im}S=\dim U-\mathrm{nullity}(S)\le \dim U\\ & \\ \dim T(U)& =\dim U-\mathrm{nullity}(S)\ge \dim U-\mathrm{nullity}(T)\end{array}$$

Note that if $T$ is injective than $\mathrm{nullity}T=0$, so $\dim T(U)=\dim U$

Link to original

Transclude of 08---Matrix-Inverses#^57bfa7