What is this? ⭐

This is a first-principles build-up of the linear algebra definition, ending with linear maps

Definition: Fields

A field has at least two elements and has two binary operations +_ and *

• association, commutation, and distributive laws are satisfied with + and *

• a field has additive and multiplicative identities

• also has additive inverse of anything in the field, but it may not be unique

• it also has a multiplicative inverse (so integers are not a field)

The distributive property, additive identity, additive inverse are enough to qualify a field as a group and an abelian group if it's commutative

Finite fields

• finite number of elements in the set

• define $\mathbb{F}_p$ as the set $0, 1, 2,... p-1$ where $p$ is a prime number
  • yes, it needs to be a prime number; you can prove that if it isn't a prime number, it is not a field

Definition: Vector Spaces

A vector space is defined over a scalar field $\mathbb{F}$ and it contains a vector addition and scalar multiplication

• It is closed under vector addition and scalar multiplication

• vector space has unique additive identity (unlike fields; this is provable)

• vector space has unique additive inverses (this is provable)

Vector spaces can also just be matrices (i.e. your basis vectors are matrices). Vector spaces can be as broad as $R^n$, or it can be as narrow as $\{0\}$. Sometimes, they lie in-between, i.e. the overall space is $R^n$ but the vector space itself is some $\text{span}(v_1, …, v_k)$.

We define subspaces as vector spaces that are subsets of some other vector space. For example, $\text{span}(v_1, …, v_k)$ is the subspace of $R^n$, if $n> k$.

You can have vector spaces of functions, as long as the functions are linear

Function spaces

• define $\mathbb{F}^{[0,1]}$ as the set of all functions in the interval $[0,1]$

• this is a vector space because it has an additive operation and a scalar multiplication operation

• you can also think of this $\mathbb{F}^{[0,1]}$ as an uncountably-infinite entry vector, and $f(x)$ would be indexing the vector at location $x$.
  • this is why $\mathbb{R}^n$ is no different than how we defined a function space. We are mapping an n-tuple $1,2,3,,....n$ to the output n-tuple vector, and each "mapping" determines what the vector will be

Subspace

• A subspace $U$ of vector space $V$ is another vector space, but it is closed under addition and scalar multiplication

• this means that a vector subspace must include the zero vector

Subspace sum

The sum $U_1 + ... + U_m$ are the set of all possible sums of the elements from $U_1, ...U_m$.

Or, in mathematical terms: $U + W = \{u+ w | u\in U, w \in W\}$

• it is worth noting that it is NOT a union operation. Unions are not guarenteed to be a subspace
  • consider two perpendicular lines. Each line can be a subspace, but the two perpenduclar lines are not a subspace
  • a subspace sum will cover the whole area between the two perpendicular lines

Direct sums

• A direct sum $\oplus$ means that every vector in $U_1 \oplus ... \oplus U_m$ can be written as $u_1 + ... + u_m$ in only one way
  • this includes the zero vector; it is sufficient to show that the zero vector is uniquely made

• if $U, W$ are subspaces of $V$, $U+W$ is a direct sum if and only if $U\cap W = \{0\}$
  • this is kinda intuitive (if there were any other common elements then you would be able to construct non-unique combinations
  • in this case, we call $U, W$ to be complements of each other. Complements are not unique

Linear combination and span

• the span is just the set of all linear combinations of vectors $c_1v_1 + ... + c_n v_n$

• the span of any vectors forms a subspace (closed under addition and multiplication)

Linear independence

• A set of vectors are linearly independent if the only choice of $a_1, ...a_m$ that makes $a_1v_1 + ... + a_m v_m = 0$ is all zeros (i.e. only one trivial solution exists)

• In contrast, linear dependence is when you can find a non-trivial solution

Linear dependence lemma

f $v_1, ...v_m$ is linearly dependence, there must exist some $v_j$ in there such that $v_j \in span(v_1, ...v_{j-1})$

Length lemma

If $V$ is spanned by $n$ vectors there can't be more than $n$ linearly independent vectors

Basis

• a basis of a subspace $V$ is a list of linearly independent vectors whose span is $V$

• a list of vectors is a basis of $V$ iff every $v\in V$ can be written uniquely as a linear combination of the vectors in that list

Lemma: spanning sets

• spanning sets contain a basis

• in other words, every spanning list can be reduced to a basis (you do this by "kicking out" certain vectors until you get a linearly independent list of vectors

Lemma: Linearly independent lists

• every linearly indepdnent list of vectors in finite-dimensional vector space can be extended to a vector space

Lemma: direct sums

• for every subspace $U$ in vector space $V$, it is always possible to find a $W$ such that $U\oplus W = V$

• this makes sense, because you can think of this $W$ as being "perpendicular" to the $U$ (think about it in 3d and it makes sense)

Dimension

• the dimension of any vector space is the number of (linearly independent) vectors in the basis

Basis and dimension lemma

If V is finite-dimensional, then every linearly independent list of vectors in $V$ with length $\dim V$ is a basis of $V$

Sum and intersection lemma

$\dim(U_1 + U_2) = dim(U_1) + dim(U_2) - dim(U_1 \cap U_2)$

• this is intuitive because if there is any overlap, you end up double-counting the dimensions

Also: $\dim(U_1 \oplus U_2) = \dim(U_1) + \dim(U_2)$

Linear maps

Linear maps $T: V → W$ will map between $V$ and $W$, where V is the domain and W is the codomain

$T(u+v) = Tu + Tv$ and $T(\lambda v) = \lambda Tv$

To define a linear map all you need to do is create a collection of basis vectors in $V$ and define where the basis vectors go. this is because by the previous two properties, you can now transform any vector in $V$!

• $Tv = T\sum x_i v_i = \sum x_i Tv_i = \sum x_i w_i$

Product of linear maps

If $T \in \mathcal(U, V), S\in \mathcal(V, W)$, then $(ST)u = S(Tu)$. You compose linear maps like functions

Linear maps take 0 to 0 (required by its properties)

You can also think of matrix multiplication as a linear map $T: V → W$ where $V$ is a matrix and $W$ is another matrix.

Vector space of linear maps

$\mathcal{L}(V, W)$ is the set of all linear maps from V to W.

Interestingly, $\mathcal{L}$ is a vector space! This is because it's closed under addition ($(T_1 + T_2)(v) = T_1 v + T_2v$ and closed under scalar multiplication.

Linear maps and Matrices

Any linear map $T: F^n → F^m$ is a multiplication by a an $m\times n$ matrix $A \in F^{m\times n}$. More specifically:

If $T \in \mathcal{L}(V , W)$, and $v_1, ...v_n$ is a basis of V and $w_1, ...w_m$ is a basis of W, then the matrix has the following property:

• $Tv_k = A_{1, k}w_1 + ... + A_{m,k}w_m$

Every column in the matrix representation of $T$ represents the transformation $v_i → T(v_i)$, in terms of the basis vectors in $W$

Connecting to common matrices

Common matrices that you see in real life will be using the standard basis for both $V, W$. So this will be a bit easier.