07. Linear Maps and Matrices

7.1 Representing linear maps with matrices

Matrix for $T$ with respect to bases $\mathcal{V}$ and $\mathcal{W}$

Let $V$ be a $n$-dimensional vector space over $\mathbf{F}$ with an ordered basis $\mathcal{V}$ of vectors $v_1,\cdots ,v_n$
Let $W$ be a $m$-dimensional vector space over $\mathbf{F}$ with an ordered basis $\mathcal{W}$ of vectors $w_1,\cdots ,w_m$
Hence every vector in $V$ and $W$ is represented by a coordinate vector in ${\mathbf{F}}^n$ and ${\mathbf{F}}^m$

Let $T:V\to W$ be a linear transformation
The matrix for $T$ with respect to the bases $\mathcal{V}$ and $\mathcal{W}$ is the matrix that takes the coordinate vector of $v$ to the coordinate vector of $Tv$

In other words this is the $m\times n$ matrix $A=(a_{ij})$ where

$$T(v_i)=\sum _{k=1}^m{a}_{ki}{w}_k$$

This is written as ${}_{\mathcal{W}}{T}_{\mathcal{V}}$ for this matrix $A$

Refer to the example below to get a better understanding!

Note about the property of the matrix $1\le i\le n$ then $T(v_i)$ can be uniquely expressed as a linear combination of $\mathcal{W}$

It is well defined and for each

Specifically, $a_{1i},\cdots ,a_{mi}$ are the coordinates of $T(v_i)$
These are the entries in the $i$th column of $A$
The coordinate column vector of $v_i$ is ${\mathbf{e}}_i^T$ and the $i$th column of $A$ is $A{\mathbf{e}}_i^T$
So the entries of $A$ are the coordinates of the image of the basis $\mathcal{V}$

Link to original

Order of the vectors in the basis is very crucial, if the order changes so does the matrix

Link to original

Matrix for $T$ with respect to this basis $V$

Same as above but with $V=W$
So we use the same ordered basis for both the domain and codomain of $T:V\to V$

Link to original

Finding Matrix with respect to a basis (1)

Let $T:{\mathbb{R}}^3\to {\mathbb{R}}^3$ be defined by $T(x,y,z)=(0,x,y)$
Define $\mathcal{E}$ as the Canonical Basis in ${\mathbb{R}}^3$
Define $\mathcal{F}$ as the ordered basis

$${\mathbf{f}}_1=(1,1,1),{\mathbf{f}}_2=(1,1,0),{\mathbf{f}}_3=(0,1,1)$$

Then

$$\begin{array}{rl}T(1,0,0)& =(0,1,0)=-{\mathbf{f}}_1+{\mathbf{f}}_2+{\mathbf{f}}_3\\ T(0,1,0)& =(0,0,1)={\mathbf{f}}_1-{\mathbf{f}}_2\\ T(0,0,1)& =(0,0,0)=0\end{array}$$

Hence

$${}_{\mathcal{F}}{T}_{\mathcal{E}}=\left(\begin{array}{ccc}-1& 1& 0\\ 1& -1& 0\\ 1& 0& 0\end{array}\right)$$

Link to original

Zero Map Matrix

Let $V$ be a $n$-dimensional vector space over $\mathbf{F}$ with ordered basis $\mathcal{V}$
Let $W$ be a m-dimensional vector space over $\mathbf{F}$ with ordered basis $\mathcal{W}$

Let ${}_{\mathcal{W}}{0}_{\mathcal{V}}$ be the matrix represented by the zero map
Then

$${}_{\mathcal{W}}{0}_{\mathcal{V}}=0_{m\times n}\text{ for any choice of }\mathcal{V}\text{ and }\mathcal{W}$$

Link to original

Identity Map Matrix

Let $V$ be a $n$-dimensional vector space over $\mathbf{F}$ with ordered basis $\mathcal{V}$

Let ${}_{\mathcal{V}}{I}_{\mathcal{V}}$ be the matrix represented by the identity map
Then

$${}_{\mathcal{V}}{\text{id}}_{\mathcal{V}}=I_n\text{ for any choice of }\mathcal{V}$$

Link to original

Linear Property of a Linear Map Matrix

Let $V$ be a $n$-dimensional vector space over $\mathbf{F}$ with ordered basis $\mathcal{V}$
Let $W$ be a m-dimensional vector space over $\mathbf{F}$ with ordered basis $\mathcal{W}$

If $S:V\to W,T:V\to W$ are linear and for $\alpha ,\beta \in \mathbf{F}$ then

$${}_{\mathcal{W}}(\alpha S+\beta T{)}_{\mathcal{V}}=\alpha {(}_{\mathcal{W}}{S}_{\mathcal{V}})+\beta {(}_{\mathcal{W}}{T}_{\mathcal{V}})$$

Link to original

08 - Matrix Representation of a Composition of Linear Maps

Matrix Representation of a Composition of Linear Maps

Let $U,V,W$ be finite-dimensional vector spaces over $\mathbf{F}$ of dimensions $m,n,p$ with ordered basis $\mathcal{U},\mathcal{V},\mathcal{W}$ respectively
Let $S:U\to V$ and $T:v\to W$ be linear

Let

$$A={}_{\mathcal{V}}{S}_{\mathcal{U}}\text{ and }B={}_{\mathcal{W}}{T}_{\mathcal{V}}$$

Then

$$BA={}_{\mathcal{W}}T{S}_{\mathcal{U}}$$

Illegal way to think but the middle vector space ($\mathcal{W}$) “cancels out” ;(

Proof

Note that $A$ is a $n\times m$ matrix and $B$ is a $p\times n$ matrix hence $BA$ is a $p\times m$ matrix
Let $\mathcal{U}$ be $u_1,\cdots ,u_m$
Let $\mathcal{V}$ be $v_1,\cdots ,v_n$
Let $\mathcal{W}$ be $w_1,\cdots ,w_p$

Suppose that $A=(a_{ij})$ and $B=(b_{ij})$ then by definition

$$\begin{array}{rl}S(u_i)& =\sum _{j=1}^n{a}_{ji}{v}_j\text{ for }1\le i\le m\\ T(v_j)& =\sum _{k=1}^p{b}_{kj}{w}_k\text{ for }1\le j\le n\end{array}$$

Then for $1\le i\le m$ we have

$$\begin{array}{rl}(T\circ S)(u_i)=T(S(u_i))& =T\left(\sum _{j=1}^n{a}_{ji}{v}_j\right)\\ & =\sum _{j=1}^n{a}_{ji}T(v_j)\text{ as }T\text{ is linear}\\ & =\sum _{j=1}^n{a}_{ji}\sum _{k=1}^p{b}_{kj}{w}_k\\ & =\sum _{k=1}^p\left(\sum _{j=1}^n{b}_{kj}{a}_{ji}\right)w_k\\ & =\sum _{k=1}^p(BA{)}_{ki}{w}_k\end{array}$$

Hence by definition ${}_{\mathcal{W}}T{S}_{\mathcal{U}}=BA$

Link to original

Associative Property of Matrices through Linear Maps corollary

Take $A\in {\mathcal{M}}_{m\times n}(\mathbf{F})$, take $B\in {\mathcal{M}}_{n\times p}(\mathbf{F})$, and take $C\in {\mathcal{M}}_{p\times q}(\mathbf{F})$
Then

$$A(BC)=(AB)C$$

Proof

Consider the left multiplication maps

$$L_A:{\mathbf{F}}_{\mathrm{col}}^n\to {\mathbf{F}}_{\mathrm{col}}^m\text{ and }{L}_B:{\mathbf{F}}_{\mathrm{col}}^p\to {\mathbf{F}}_{\mathrm{col}}^n\text{ and }{L}_C:{\mathbf{F}}_{\mathrm{col}}^q\to {\mathbf{F}}_{\mathrm{col}}^p$$

With respect to the standard bases of these spaces, then represent the matrices of $L_A,L_B,L_C$ as $A,B,C$ respectively
Then
By Matrix of a Composition of Linear Maps, $A(BC)$ and $(AB)C$ are matrices of

$$L_A\circ (L_B\circ L_C):{\mathbf{F}}_{\mathrm{col}}^q\to {\mathbf{F}}_{\mathrm{col}}^m\text{ and }(L_A\circ L_B)\circ L_C:{\mathbf{F}}_{\mathrm{col}}^q\to {\mathbf{F}}_{\mathrm{col}}^m$$

But composition of functions is associative, so

$$L_A\circ (L_B\circ L_C)=(L_A\circ L_B)\circ L_C$$

Hence

$$A(BC)=(AB)C$$

Link to original

Invertible Linear Maps and Matrices corollary

Let $V$ be a finite-dimensional vector space
Let $T:V\to V$ be an invertible linear transformation
Let $A$ be a matrix of $T$ with respect to an ordered basis (for both domain and codomain)

Then $A$ is invertible, and $A^{-1}$ is the matrix of $T^{-1}$ with respect to the same basis

Link to original

---

7.2 Change of basis

09 - Change of Basis Theorem

Change of Basis Theorem

Let $V$ be a finite-dimensional vector space over $\mathbf{F}$ with ordered $\mathcal{V},{\mathcal{V}}^{\prime }$
Let $W$ be a finite-dimensional vector space over $\mathbf{F}$ with ordered bases $\mathcal{W},{\mathcal{W}}^{\prime }$

Let $T:V\to W$ be a linear map
Then

$${}_{{\mathcal{W}}^{\prime }}{T}_{{\mathcal{V}}^{\prime }}=({}_{{\mathcal{W}}^{\prime }}{I}_{\mathcal{W}})({}_{\mathcal{W}}{T}_{\mathcal{V}})({}_{\mathcal{V}}{I}_{{\mathcal{V}}^{\prime }})$$

Follows from 08 - Matrix Representation of a Composition of Linear Maps =_=

Change of Basis Theorem V2

Let $V$ be a finite-dimensional vector space over $\mathbf{F}$ with ordered basis $\mathcal{V},{\mathcal{V}}^{\prime }$
Let $T:V\to V$ be a linear map
Then

$${}_{{\mathcal{V}}^{\prime }}{T}_{{\mathcal{V}}^{\prime }}={(}_{{\mathcal{V}}^{\prime }}{I}_{\mathcal{V}}){(}_{\mathcal{V}}{T}_{\mathcal{V}}){(}_{\mathcal{V}}{I}_{{\mathcal{V}}^{\prime }})$$

If we set $A={}_{{\mathcal{V}}^{\prime }}{T}_{{\mathcal{V}}^{\prime }},B={}_{\mathcal{V}}{T}_{\mathcal{V}},P={}_{\mathcal{V}}{I}_{{\mathcal{V}}^{\prime }}$
Then

$$A=P^{-1}BP$$

Special case from the proof above :)

Link to original

Similar Matrices

Take $A,B\in {\mathcal{M}}_{n\times n}(\mathbf{F})$
If there is an invertible $n\times n$ matrix $P$ such that $A=P^{-1}BP$ then

$$A\text{ and }B\text{ are similar}$$

It is also an equivalence relation

Link to original

Properties of Linear Maps

Let $A={}_{\mathcal{V}}{T}_{\mathcal{V}}$ and $B={}_{\mathcal{W}}{T}_{\mathcal{W}}$ be matrices represented $T$ with respect to two bases so that

$$A=P^{-1}BP\text{ for some invertible }P$$

1. $A$ is invertible if and only if $B$ invertible
2. Trace of $A$ equals the trace of $B$ (follows from identity $\mathrm{trace}(MN)=\mathrm{trace}(NM)$)
3. A functional identity satisfied by $A$, such as $A^2=A$, is also satisfied by $B$
4. Determinant of $A$ equals the determinant of $B$
5. Eigenvalues of $A$ equals the eigenvalues of $B$

Note that $(4)$ and $(5)$ will be formally defined in Linear Algebra II

Link to original

---

7.3 Matrices and Rank

Relation between $\mathrm{rowrank}$ and $\mathrm{colrank}$

From the definitions then $\mathrm{Col}(A)=\mathrm{Row}(A^T)$ so $\mathrm{colrank}(A)=\mathrm{rowrank}(A^T)$

Similarly
$\mathrm{Row}(A)=\mathrm{Col}(A^T)$ so $\mathrm{rowrank}(A)=\mathrm{colrank}(A^T)$

Link to original

Consistent Linear System lemma

$$(A\mid )\text{ is consistent}⟺\mathrm{Col}(A\mid \mathbf{b})=\mathrm{Col}(A)$$

Proof

Suppose that $A$ is a $m\times n$ matrix and denote the columns of $A$ as ${\mathbf{c}}_1,{\mathbf{c}}_2,\cdots ,{\mathbf{c}}_n$
Then

$$\begin{array}{rl}(A\mid \mathbf{b})\text{ is consistent }& ⟺A\mathbf{x}=\mathbf{b}\text{ for some }\mathbf{x}\in {\mathbf{F}}_{\mathrm{col}}^n\\ & ⟺x_1{\mathbf{c}}_1+\cdots +x_n{\mathbf{c}}_n=\mathbf{b}\text{ for some }\mathbf{x}\in {\mathbf{F}}_{\mathrm{col}}^n\\ & ⟺\mathbf{b}\in \mathrm{Col}(A)\\ & ⟺\mathrm{Col}(A\mid \mathbf{b})=\mathrm{Col}(A)\end{array}$$

Link to original

10 - Column Rank equals Row Rank

2> [!definition] Column Rank equals Row Rank

The column rank of a matrix equals its row rank

Proof

Link to original

11 - Row Rank Criterion on Product of Matrices

Row Rank Criterion on Product of Matrices

Let $P=QR$ where $P,Q,R$ are respectively $k\times l,k\times m,m\times l$ matrices over $\mathbf{F}$

1. $\mathrm{rowrank}(P)$ is at most $m$

2. Let $p=\mathrm{rowrank}(P)$, then $P$ can be written as a $k\times p$ and $p\times l$ matrix

3. The row rank and column rank of a matrix is equal

Proof

1. By Property of Row Space we have

$$\mathrm{Row}(P)=\mathrm{Row}(QR)\le \mathrm{Row}(R)$$

Hence

$$\mathrm{rowrank}(P)=\dim \mathrm{Row}(P)\le \dim \mathrm{Row}(R)=\mathrm{rowrank}(R)\le m$$

2. There is a $k\times k$ invertible matrix $E$ such that $EP=\mathrm{RRE}(P)$ and hence

$$P=E^{-1}\mathrm{RRE}(P)$$

Let $\tilde{E}$ denote the first $p$ columns of $E^{-1}$ and $\tilde{P}$ denote the first $p$ rows of $\mathrm{RRE}(P)$
As the last $k-p$ rows of $\mathrm{RRE}(P)$ are zero rows then

$$P=\tilde{E}\tilde{P}$$

where $\tilde{E}$ is a $k\times p$ and $\tilde{P}$ is a $p\times l$ matrix
3) From $(1)$ and $(2)$, the row rank of a $k\times l$ matrix $P$ is the minimal value $p$ such that
$P$ can be written as the product $QR$ of a $k\times p$ matrix and a $p\times l$ matrix
Whenever $P=QR$

$$\underset{l\times k}{P^T}=\underset{l\times p}{R^T}\underset{p\times k}{Q^T}$$

So the row rank of $P^T$ is similarly $p$. But the $\mathrm{rowrank}(P^T)=\mathrm{colrank}(P)$ as required

Link to original