A basic knowledge of linear algebra is essential to the study of differential geometry. The linearisation of a smooth map features heavily throughout differential geometry as do determinants, traces, eigenvalues and eigenvectors. Many of the fundamental objects studied in differential geometry are tensor fields which incorporate and extend undergraduate linear algebra.
Here we review some of the most relevant topics. Some topics may not be covered in standard courses, and the perspective taken as well as some of the notation may also be unfamiliar. Thus even if you have a solid background in the material, it's worth at least taking a look. Throughout the course if you encounter something you're not comfortable with, you can always return here to see if it's explained here.
A real vector space is a set \(V\) equipped with operations of addition, additive inverse, and scalar multiplication:
\begin{align*} + &: V \times V \to V \\ - &: V \to V \\ \cdot &: \mathbb{R} \times V \to V \end{align*}That is addition takes two vectors as input and produces a third vector as output, additive inverse takes a vector as input and produces a vector as output, while scalar multiplication takes a real number and a vector as input and produces a vector as output. Typically we write \(u + v\), \(-u\) and \(cv\) for the addition of vectors \(u,v\), additive inverse of a vector \(u\), and scalar multiplication of the vector \(v\) by the real number \(c\) respectively.
The operations of addition and scalar multiplication are required to satisfy the following axioms:
For compatibility, \((cd) u\) means to first multiply the real numbers \(c\) and \(d\) together and then to scalar multiply the result with the vector \(u\). On the other hand, \(c (du)\) means to first scalar multiply the real number \(d\) with the vector \(u\) and then to scalar multiply the resulting vector with the real number \(c\). The axiom asserts that these two are equal and hence there is no ambiguity in not distinguishing between scalar multiplication and multiplication of real numbers. Thus there is no ambiguity in writing both as juxtaposition, and indeed in writing expressions like \(cdu\) which is equal to \((cd) u\) and to \(c(du)\).
Similarly for distribution of scalar multiplication over addition, \((c+d)u\) means to first add the real numbers \(c,d\), then scalar multiply the result with the vector \(u\). On the other hand, \(cu + du\) means to scalar multiply \(c\) with \(u\) and \(d\) with \(u\), then vector add the resulting vectors. The axiom asserts that these are equal as thus there is no ambiguity in using the same symbol \(+\) for both scalar addition and vector addition.
For a real vector space, a scalar is a real number. Given scalars \(c, d\) and vectors \(u, v\), the vector \(cu + dv\) is called a linear combination .
The \(n\)-fold product of \(\mathbb{R}\) with itself is
\begin{equation*} \mathbb{R}^n := \{(u_1, \dots, u_n) : u_i \in \mathbb{R}\} \end{equation*}That is \(\mathbb{R}^n\) is the set of \(n\)-tuples of real numbers.
The vector space operations are defined component-wise; that is by defining them by the corresponding operations on the components
\begin{align*} (u_1, \dots, u_n) + (v_1, \dots, v_n) &= (u_1 + v_1, \dots, u_n + v_n) \\ -(u_1, \dots, u_n) &= (-u_1, \dots, -u_n) \\ c(u_1, \dots, u_n) &= (cu_1, \dots, cu_n) \end{align*}We define the zero-vector \(\vec{0} = (0, \dots, 0)\). Typically we will just write \(0\) in place of \(\vec{0}\) and the context should make it clear whether we mean the real number \(0\) or the vector \(\vec{0}\). For example, given a vector \(u\), \(u + 0\) means to add the zero-vector to \(u\) (the result of which is \(0\)), while \(0u\) means to multiply the vector \(u\) by the real number \(0\) (the result of which is the zero-vector).
Verifying the vector space axioms is straight forward directly from the definitions and the corresponding properties of real numbers. If you've never done this before, it's well worth spending a little time on it to ensure you understand and to convince yourself that it is indeed straightforward. For example, to verify commutativity of addition we have
\begin{align*} (u_1, \dots, u_n) + (v_1, \dots, v_n) &= (u_1 + v_1, \dots, u_n + v_n) \\ &= (v_1 + u_1, \dots, v_n + u_n) \\ &= (v_1, \dots, v_n) + (u_1, \dots, u_n). \end{align*}Thus for example in \(\mathbb{R}^2\), the linear combination \(2(1, 0) - 10(1,1/2)\) is the vector
\begin{align*} 2(1, 0) - 10(1,1/2) &= (2, 0) - (10, 5) \\ &= (2 - 10, 0 - 5) = (-8, -5) \\ &= -(8, 5) \end{align*}Let \(\mathbb{R}^n \to \mathbb{R}\) denote the set of real valued functions of \(n\) real variables. Here our vectors are functions. This is a vector space with pointwise operations; that is with operations define at each point \(x \in \mathbb{R}^n\):
\begin{align*} (f + g) (x) &:= f(x) + g(x) \\ (-f) (x) &:= - f(x) \\ (cf)(x) &:= c \times f(x). \end{align*}The zero-vector is the zero-function; that is, the constant function with value \(0\) for every \(x\).
Remember that addition takes to vectors (in this case function \(f, g\)) and produces a third function \(f + g\). This is the function that maps \(x \in \mathbb{R}^n\) to the value \(f(x) + g(x)\). Thus we define \(f + g\) by first evaluating \(f\) and \(g\) at the point \(x\), then by adding the resulting real numbers \(f(x) + g(x)\). Varying the point \(x\) gives the function \(f + g\).
For scalar multiplication, given a real number \(c\) and a function \(f\), \(cf\) is a new function defined by taking an input \(x \in \mathbb{R}^n\) and outputting the real number \(c \times f(x)\) - that is the product of the number \(c \in \mathbb{R}\) with the number \(f(x) \in \mathbb{R}\). Varying the point \(x\) gives the function \(cf\).
For example, the function \(2 \sin(x) - 3 e^x\) is a vector being a linear combination of the vectors \(\sin(x)\) and \(e^x\) with the scalars \(2, -3\).
Let \(C^k(\mathbb{R}^n \to \mathbb{R})\) denote the set of \(C^k\) real valued functions of \(n\) variables. That is, the set of functions \(f\) whose partial derivatives \(\partial_{i_1} \dots \partial_{i_j} f\) exist and are continuous for each \(j = 0, \dots, k\).
A \(C^0\) function is a continuous function.
A \(C^1\) function is a \(C^0\) function such that the partial derivatives \(\partial_1 f, \dots, \partial_n f\) exist and are continuous.
A \(C^2\) function is a \(C^1\) function such that for each \(i, j\) with \(1 \leq i, j \leq n\), the partial derivative \(\partial_{ij} f\) exists and is continuous.
The vector space structure is inherited from the vector space structure on \(\mathbb{R}^n \to \mathbb{R}\) of the previous example. The reason we obtain a vector space is that \(C^k(\mathbb{R}^n \to \mathbb{R})\) is closed under the vector space operations. That is if \(f, g\) are \(C^k\) functions then so too is \(f + g\). Likewise for \(-f\) and \(cf\). The zero function is also \(C^k\).
Let \(V\) denote the set of \(C^2\) solutions of the equation \[ y'' + y = 0 \] on \(\mathbb{R}\). That is \(V\) is the set of \(C^2\) functions \(y : \mathbb{R} \to \mathbb{R}\) such that \(y'' + y = 0\).
The vector space structure is inherited from \(\mathbb{R} \to \mathbb{R}\). The reason \(V\) is a vector space is because the equation \(y'' + y = 0\) is linear hence the set of solutions is closed under the vector space operations. For example, if \(y,z\) are solutions, then \[ (y + z)'' + (y + z) = y'' + y + z'' + z = 0 + 0 = 0 \] and hence \(y + z \in V\). Similarly \(cy \in V\) and the zero function \(0 \in V\).
Some example elements of \(V\) are \[ \sin x, \cos x, 3 \sin x - 5 \cos x. \]
Let \(V\) denote the set of \(C^k\) solutions to the linear equation \[ L(y) := c_k y^{(k)} + \dots + c_1 y^{(1)} + c_0 y = 0 \] where \(y^{(i)}\) denotes the \(i\)'th derivative of \(y\) for \(i = 0, \dots, k\).
Then \(V\) is a vector space in a manner analogous to the previous example of the specific case \(L(y) = y'' + y\).
Throughout this section \(V\) will denote a real vector space.
A subspace \(W\) of a vector space \(V\) is a subset \(W \subseteq V\) that is closed under the vector space operations of \(W\). That is
The subset \(W = \{(x, y, 0) : x, y \in \mathbb{R}\}\) is a subspace of \(\mathbb{R}^3\).
More generally, given any set of indices \(I = \{i_1, \dots, i_k\} \subseteq \{1, \dots, n\}\), the subset \[ W = \{(x_1, \dots, x_n) \in \mathbb{R}^n : x_i = 0, \quad \forall i \in I\} \] is a subspace of \(\mathbb{R}^n\).
The case of \(\{x, y, 0\} \subseteq \mathbb{R}^3\) corresponds to \(I = \{3\} \subseteq \{1, 2, 3\}\).
There is a chain of inclusions of subspaces \[ \dots \subseteq C^k(\mathbb{R}^n \to \mathbb{R}) \subseteq C^{k-1} (\mathbb{R}^n \to \mathbb{R}) \subseteq \dots \subseteq C^0(\mathbb{R}^n \to \mathbb{R}) \subseteq \mathbb{R}^n \to \mathbb{R}. \]
The vector space of \(C^2\) solutions to \(y'' + y = 0\) is a subspace of \(C^2\).
Show that a set \(W \subseteq V\) is a subspace if and only if it is closed under finite linear combinations. That is, given any \(c^i \in \mathbb{R}\), \(u_i \in W\) \(i = 1, \dots, k\), the linear combination \(\sum_{i=1}^k c^i u_i \in W\).
Let \(S \subseteq V\) be a subset of \(V\). The span of \(S\) is the set of all finite linear combinations of elements from \(S\): \[ \operatorname{span} S := \left\{\sum_{i=1}^k c_i v_i : c_i \in \mathbb{R}, v_i \in S : k = 1, 2, 3, \cdots\right\} \]
The set \(\operatorname{span} S\) is the smallest subspace of \(V\) containing \(S\).
To say that \(\operatorname{span} S\) is the smallest subspace of \(V\) containing \(S\) is to say two things:
First we show that \(\operatorname{span} S\) is a subspace. Let \(u, v \in \operatorname{span} S\) which we may write in the form, \[ u = \sum_{i=1}^k c^i u_i, \quad v = \sum_{j=1}^l d^j v_j \] for \(c^i, d^j \in \mathbb{R}\), \(u_i, v_j \in S\) and \(k,l \in \{1, 2, 3, \dots \}\). Note we don't claim any uniqueness of these expressions, just that by definition of \(\operatorname{span} S\), there is at least one way of writing \(u, v\) in this form.
Then
\begin{align*} u + v &= \sum_{i=1}^k c^i u_i + \sum_{j=1}^l d^j v_j \\ &= c^1 u_1 + \dots + c^k u_k + d^1 v_1 + \dots d^l v_l \end{align*}The last line is a linear combination of the elements \(u_1, \dots, u_k, v_1, \dots, v_l \in S\) hence \(u + v \in \operatorname{span} S\).
Thus \(\operatorname{span} S\) is closed under \(+\). We leave the remaining steps as an exercise for the reader.
Complete the proof that \(\operatorname{span} S\) is a subspace. That is, show that the set \(\operatorname{span} S\) satisfies the requirements in the definition of subspaces.
Let \(U, V\) be vector spaces. The direct sum , \(U \oplus V\) is the set of pairs of elements from \(U\) and \(V\). That is, \[ U \oplus V = \{(u, v) : u \in U, v \in V\}. \]
Given \(u \in U\) and \(v \in V\) we write, \[ u \oplus v = (u, v). \]
\[ \mathbb{R}^2 = \mathbb{R} \oplus \mathbb{R}. \] Elements are of the form \(x \oplus y\) where \(x, y \in \mathbb{R}\). For example, \((2, -3) = 2 \oplus -3\).
Let \(U, V\) be vector spaces. The direct sum \(U \oplus V\) is a vector space with operations
\begin{align*} (u_1, v_1) + (u_2, v_2) &= (u_1 + u_2, v_1 + v_2) \\ c (u, v) &= (cu, cv) \\ 0 &= (0_U, 0_V) \end{align*}We can extend this definition for any finite \(k\):
Let \(V_1, \dots, V_k\) be vector spaces. The direct sum \(V_1 \oplus \cdots \oplus V_k\) is the vector space \[ V_1 \oplus \cdots \oplus V_k = \{(v_1, \dots, v_k) : v_i \in V_i\} \] with vector space operations
\begin{align*} (u_1, \dots, u_k) + (v_1, \dots, v_k) &= (u_1 + v_1, \dots, u_k + v_k) \\ c (v_1, \dots, v_k) &= (cv_1, \dots, cv_k) \\ 0 &= (0_1, \dots, 0_k) \end{align*}Verify that \(V_1 \oplus \cdots \oplus V_k\) is a vector space.
Let \(u_1, u_2 \in U\) and \(v_1 v_2 \in V\). Show that \[ c (u_1 \oplus u_2) = cu_1 \oplus cu_2 \] and that \[ (u_1 \oplus v_1) + (u_2 \oplus v_2) = (u_1 + u_2) \oplus (v_1 + v_2). \]
For any vector space \(V\), the \(k\)-fold product is \[ V^k = \underbrace{V \oplus \dots \oplus V}_{\text{\(k\) times}} \]
Throughout this section \(V\) will denote a vector space.
A set of vectors \(S \subseteq V\) is called a generating set if \(\operatorname{span} S = V\). Given a vector \(u \in V\), an expression \(u = \sum_{i=1}^k c^i u_i\) with \(c^i \in \mathbb{R}\) and \(u_i \in S\) is called an expansion of \(u\) in the generating set \(S\). The elements \(c^i \in \mathbb{R}\) are called the coefficients of the expansion.
The set \(S = \{e_1 = (1, 0), e_2 = (0, 1)\}\) is a generating set for \(\mathbb{R}^2\) as are the sets \(\{e_1 - e_2, e_1 + e_2\}\), \(\{e_1, e_2, e_1 + e_2\}\). In the latter case, an expansion of a vector is not unique in general. For example, \((2, 0) = 2e_1 = 2(e_1 + e_2) - 2 e_2\).
Let \(S \subseteq T \subseteq V\). Show that \(\operatorname{span} S = \operatorname{span} T\).
Adding elements of \(\operatorname{span} S\) to \(S\) does not change the span. This is because any element of \(\operatorname{span} S\) is already a finite linear combination of elements of \(S\), and \(\operatorname{span} S\) is closed under finite linear combinations. The next exercise verifies this claim.
Let \(S \subseteq V\) and let \(u \in \operatorname{span} S\). Let \(S' = S \cup \{u\}\). Show that \[ \operatorname{span} S' = \operatorname{span} S. \]
Let \(S \subseteq V\) be a set of vectors.
We say \(S\) is linearly dependent if there is some finite subset \(u_1, \dots, u_k \subseteq S\) and non-zero real numbers \(c^1, \dots, c^k\) with \[ \sum_{i=1}^k c^i u_i = 0. \]
Otherwise we say \(S\) linearly independent .
If \(S\) is linearly dependent this means that at least one element of \(S\) may be written as a linear combination of other elements from \(S\). This follows since by definition we have \[ c^1 u_1 + \dots + c^k u_k = 0 \] with each \(c^i \neq 0\). In particular \(c^1 \neq 0\) so that \[ u_1 = - \frac{1}{c^1} \left(c^2 u_2 + \dots + c^k u_k\right). \] That is, \(u_1\) is a linear combination of \(u_2, \dots, u_k \in S\).
On the other hand, if \(S\) is linearly independent, then for every finite linear combination \(\sum_{i=1}^k c^i u_i\) of elements \(u_i \in S\) we have the implication \[ \sum_{i=1}^k c^i u_i = 0 \Rightarrow c^i = 0 \quad \forall i = 1, \dots, k. \] In words, \(S\) is linearly independent if the only way to obtain \(0\) through a linear combination of vectors from \(S\) is by the trivial linear combination; that is with all coefficients equal to zero. Equivalently, no \(u \in S\) may be written as a linear combination of the other elements in \(S \backslash \{u\}\).
Let \(e_1 = (1, 0)\) and \(e_2 = (0, 1)\). Then \(S = \{e_1 + e_2, e_1 - e_2\}\) is a linearly independent set. Indeed, suppose there exists real numbers \(A, B\) such that \[ A(e_1 + e_2) + B(e_1 - e_2) = 0. \] Then \[ (0, 0) = (A+B) e_1 + (A-B)e_2 = (A+B, A-B) \] hence \(A + B = A - B = 0\) and therefore \(A = B = 0\).
Linear independence is a maximally efficient way of spanning a subspace. For example, \(S = \{e_1 + e_2, e_1 - e_2\}\) spans \(\mathbb{R}^2\) as does \(T = \{e_1 + e_2, e_1 - e_2, e_1\}\). The former has only \(2\) elements, whilst the latter has \(3\) elements. The difference is that \(T\) is not a linearly independent set.
Show that \(T = \{e_1 + e_2, e_1 - e_2, e_1\}\) is not a linearly independent set.
Let \(S = \{u_1, \dots, u_k\}\) be a linearly independent set. Let \(S_i = S \backslash \{u_i\} = \{u_1, \dots, u_{i-1}, u_{i+1}, \dots, u_k\}\). Then \(\operatorname{span} S_i\) is a proper subspace of \(\operatorname{span}S\).
\(\operatorname{span} S_i\) is a subspace of \(\operatorname{span}S\) since \(S_i \subseteq S\), hence every linear combination of elements from \(S_i\) is linear combination of elements of \(S\).
Suppose that \(S_i\) was not a proper subset; that is \(S_i = S\). Then \(u_i \in \operatorname{span} S = \operatorname{span} S_i\) so that there are constants \(c^1, \dots, c^{i-1}, c^{i+1}, \dots, c^k\) such that \[ u_i = c^1 u_1 + \dots + c^{i-1}u_{i-1} + c^{i+1} u_{1+1} + \dots + c^k u_k = 0. \] That is \[ c^1 u_1 + \dots + c^{i-1}u_{i-1} - u_i + c^{i+1} u_{1+1} + \dots + c^k u_k = 0 \] contradicting that \(u_1,\dots, u_k\) is linearly independent. Thus \(S_i\) is a proper subset of \(S\).
Let \(S\) be linearly independent and let \(S' \subsetneq S\).
We now have an important result regarding the number of elements of linearly independent sets.
Let \(S = \{u_1, \dots, u_k\}\) and \(T = \{v_1, \cdots, v_l\}\) be linearly independent sets such that \(\operatorname{span} S \subseteq \operatorname{span} T\). Then \(l \geq k\); that is \(T\) has at least as many elements as \(S\).
The proof is by induction on \(k\).
The base case, \(k = 0\) is trivial. While it is perfectly valid to use \(k = 0\) as the base case, it may feel a little unsatisfying. So let's verify \(k = 1\) for the base case as well. We have that \(S = \{u_1\}\) and \(\operatorname{span} S = \{cu : c \in \mathbb{R}\} \neq \emptyset\). Then since \(\operatorname{span} T\) contains \(\operatorname{span} S\), it must be non-empty, hence \(T\) is non-empty, hence \(l \geq 1 = k\).
For the induction step, suppose that the result is true for \(k-1\); that is if \(S'\) is a linearly independent set of \(k-1\) elements and \(T'\) is any finite linearly independent set with \(\operatorname{span} S' \subseteq \operatorname{span} T'\), then \(T'\) has at least \(k-1\) elements.
Now, let \(S = \{u_1, \dots, u_k\}\) and \(T = \{v_1, \dots, v_l\}\) be linearly independent sets with \(\operatorname{span} S \subseteq \operatorname{span} T\). Since each \(u_i \in \operatorname{span} S \subseteq \operatorname{span} T\), we may expand each \(u_i\) in terms of elements of \(T\): \[ u_i = \sum_{j=1}^l c^j_i v_j \] for coefficients \(c^j_i \in \mathbb{R}\).
If \(c^l_i = 0\) for every \(i\), then each \(u_i\) is a linear combination of the elements \(v_1, \dots, v_{l-1}\); that is \(\operatorname{span} S \subseteq \operatorname{span} \{v_1, \dots, v_{l-1}\}\). Repeating, we eventually get \(\operatorname{span} S \subseteq \operatorname{span} \{v_1, \dots, v_m\}\) with \(1 \leq m \leq l\) and such that \(c^m_i \neq 0\) for at least one \(i\). We will show that \(m \geq k\) from which we conclude the required result \(l \geq m \geq k\).
Without loss of generality (i.e. reordering the \(u_i\) if necessary) we assume that \(c^m_k \neq 0\). Then we may rearrange \[ u_k = \sum_{j=1}^{m-1} c^j_k v_j + c^m_k v_m \] to obtain \[ v_m = \frac{1}{c^m_k} \left(u_k - \sum_{j=1}^{m-1} c^j_k v_j\right). \] Substituting into \(u_i = \sum_{j=1}^m c^j_i v_j\) we obtain that
\begin{align*} u_i &= \sum_{j=1}^{m-1} c^j_i v_j + c^m_i v_m \\ &= \sum_{j=1}^{m-1} c^j_i v_j + c^m_i \frac{1}{c^m_k} \left(u_k - \sum_{j=1}^{m-1} c^j_k v_j\right). \end{align*}Thus for \(i = 1, \dots, k-1\), \[ u_i - \frac{c^m_i}{c^m_k} u_k = \sum_{j=1}^{m-1} (c^j_i - c^j_k) v_j. \] That is, \[ u_i - \frac{c^m_i}{c^m_k} u_k \in \operatorname{span} \{v_1, \dots, v_{m-1}\} \]
But now \(\{u_1 - \frac{c^m_1}{c^m_k} u_k, \dots, u_{k-1} - \frac{c^m_{k-1}}{c^m_k} u_k\}\) is a linearly independent set (see exercise below) of \(k-1\) vectors contained in the span of the linearly independent set \(\{v_1, \dots, v_{m-1}\}\). The inductive hypothesis applies giving \(m-1 \geq k-1\) and hence that \[ l \geq m \geq k \] as required.
Let \(\{v_1, \dots, v_m\}\) be a set of linearly independent vectors. Show that for any real numbers \(a_j\) \(1 \leq j \leq m-1\), the set \(\{v_1 - a_1 v_m, \dots, v_{l-1} - a_{l-1} v_m\}\) is a linearly independent set.
A set of vectors \(S \subseteq V\) is called a basis if it is a linearly independent generating set.
Let \(S\) be a basis for \(V\). Then for every \(u \in V\) there is a unique finite set of vectors \(u_i \in S\) and non-zero coefficients \(c^i \in \mathbb{R}\), \(i=1, \dots, k\) such that \[ u = \sum_{1=1}^k c^i u_i. \] In words, if \(S\) is a basis, then for every vector \(u \in V\) there is a unique expansion of \(u\) as finite linear combination of elements of \(S\).
Since \(S\) is a generating set, such a set of vectors and coefficients exists.
For uniqueness, suppose that \(u = \sum_{i=1}^k a^i u_i = \sum_{j=1}^l b^i v_i\) with \(a_i, b_j \in \mathbb{R}\) and \(u_i, v_j \in S\). The sets \(A = \{u_i\}_{i=1}^k\) and \(B = \{v_j\}_{j=1}^l\) are both subsets of \(S\). A priori, these sets need not be equal - they may intersect, one may be contained in the other, or they may indeed be disjoint. In any event their union \(C = A \bigcup B\) is a finite set so we may write \(C = A \bigcup B = \{w_m\}_{m=1}^N\).
Define
\begin{equation*} c^m = \begin{cases} a^i & w_m = u_i \in A \\ 0 & \text{otherwise} \end{cases} \end{equation*}That is, if \(w^m \in A\), then \(w_m = u_i\) for some \(i\), and we define \(c^m\) to be the corresponding coefficient \(a_i\) from the expansion of \(u\) as a linear combination of elements in \(A\). Otherwise, if \(w^m\) does not appear in the expansion of \(u\) (i.e. \(w^m \in B \backslash A\)) we just set the coefficient \(w^m\) to \(0\). Thus we may write \[ u = \sum_{m=1}^N c^m w_m. \] Likewise we may write \[ u = \sum_{m=1}^N d^m w_m \] where \(d_m = b_j\) if \(w^m = v_j \in B\) and \(d_m = 0\) otherwise.
Then \[ \sum_{m=1}^N (c^m - d^m) w_m = \sum_{m=1}^N c^m w_m - \sum_{m=1}^N d^m w_m = 0. \] Since \(S\) is linearly independent and each \(w_m \in S\), we must have \(c^m = d^m\) and hence the expansion is unique.
We say that \(V\) is a finite dimensional vector space is there is a finite basis \(S\).
Let \(V\) be a finite dimensional vector space. Then every basis has the same (necessarily finite) number of elements.
Let \(S = \{u_1, \dots, u_k\}\) be a finite basis for \(V\) and let \(T = \{v_1, \dots, v_l\}\) be any other basis. Then \(\operatorname{span} S \subseteq \operatorname{span} T\) and hence \(k \leq l\). Likewise, \(\operatorname{span} T \subseteq \operatorname{span} S\) and \(l \leq k\). Thus \(l = k\).
Let \(V\) be a finite dimensional vector space. The dimension of \(V\) is the number of elements in a basis for \(V\). If \(V = \{0\}\), we say \(V\) has dimension \(0\).
By the theorem, all bases have the same number of elements, hence the dimension is well defined.
Let \(V = \mathbb{R}^n\) and let \[ e_1 = (1, 0, \dots, 0), e_2 = (0, 1, 0, \dots, 0), \dots, e_n = (0, \dots, 0, 1) \] Show that \(S = \{e_1, \dots, e_n\}\) is a basis and hence that the dimension of \(\mathbb{R}^n\) is \(n\).
The set of \(C^2\) solutions of \(y'' + y = 0\) has basis \(u_1 = \sin x, u_2 = \cos x\) hence has dimension \(2\).
Let \(V\) be a vector space of dimension \(k\). Show that a subset \(S \subseteq V\) is a basis if and only \(S\) is linearly independent and has \(k\) elements. Given example of an \(S\) that has \(k\) elements but is not a basis, and give an example of an \(S\) that is linearly independent but is not a basis.
A basis for \(\mathbb{R}^2\) is \((1, 0), (0, 1)\). The set \(S = \{(1, 1), (1, -1)\}\) is a linearly independent subset of \(\mathbb{R}^2\) with two elements, hence is a basis by the previous exercise.
Let \(U \subseteq V\) be a subspace of a \(n\)-dimensional vector space \(V\). Then \(U\) is finite dimensional with dimension \(k \leq n\).
The proof is by a finite induction argument.
If \(U = \{0\}\), then \(U\) has \(0\) dimension which is finite.
Otherwise, there exists \(u_1 \in U\) with \(u_1 \neq 0\). If \(\operatorname{span} \{u_1\} = U\) then \(U\) has finite dimension \(1\).
Otherwise, there exists a \(u_2 \in U \backslash \operatorname{span} \{u_1\}\). Thus \(u_2\) is not linearly dependent on \(u_2\) hence is linearly independent. Then \(\{u_1, u_2\}\) is a linearly independent set. If \(\operatorname{span} \{u_1, u_2\} = U\), then \(\{u_1, u_2\}\) is also a spanning set hence a basis, hence \(U\) has dimension \(2\).
Now suppose that we have \(\{u_1, \dots, u_{l-1}\}\) linearly independent with \(\operatorname{span} \{u_1, \dots, u_{l-1}\} \subsetneq U\). Thus there is a \(u_l \in U\) linearly independent of \( \{u_1, \dots, u_{l-1}\}\). Then \(\{u_1, \dots, u_l\}\) is linearly independent. If \(\operatorname{span} \{u_1, \dots, u_l\} = U\), then \(U\) has dimension \(l\).
If at any step \(\operatorname{span} \{u_1, \dots, u_{l}\} \neq U\) we can add a new element \(\{u_{l+1}\}\) to obtain a linearly independent set of \(l + 1\) elements. This process cannot continue indefinitely since \(V\) has dimension \(n\) and any linearly independent set has at most \(n\) elements. Thus the process terminates after a finite number of steps - say \(k \leq n\) steps - at which point we have obtained a basis for \(U\) and the dimension of \(U\) is \(k \leq n\).
Let \(U \subseteq V\) be a subspace of an \(n\)-dimensional vector space. Using a similar argument as the lemma, show that any basis \(\{u_1, \dots, u_k\}\) for \(U\) may be extended to a basis \(\{u_1, \dots, u_k, u_{k+1}, \dots, u_n\}\) for \(V\).
Let \(U = \{(x, y, 0\}\} \subseteq \mathbb{R}^3\). Then \(\{(e_1 = (1, 0, 0), e_2 = (0, 1, 0)\}\) is a basis for \(U\) which we extend to the basis \(\{(e_1 = (1, 0, 0), e_2 = (0, 1, 0), e_3 = (0, 0, 1)\}\) for \(\mathbb{R}^3\).
Let \(U = \{y \in C^2(\mathbb{R} \to \mathbb{R}): y'' + y = 0, y'(0) = 0\}\). Show that \(U\) is a subspace of \(V\), the set of all \(C^2\) solutions of \(y'' + y = 0\). Determine a basis for \(U\) and hence determine the dimension of \(U\). Extend this basis to a basis for \(V\).
Let \(V^1, V^2\) be vector spaces of dimensions \(k_1\) and \(k_2\) respectively. Let \(\{v^1_1, \dots, v^1_{k_1}\}\) and \(\{v^2_1, \dots, v^2_{k_2}\}\) be bases for \(V^1, V^2\) respectively. Show that \(V^1 \oplus V^2\) has dimension \(k_1 + k_2\) by showing that a basis for \(V^1 \oplus V^2\) is \(\{v^1_1 \oplus 0, \dots, v^1_{k_1} \oplus 0, 0 \oplus v^2_1, \dots, \oplus v^2_{k_2}\}\)
Let \(V, W\) be vector spaces. A linear map is a function \(F : V \to W\) such that
That is, \(F\) is a linear map if it preserves the vector space operations of addition and scalar multiplication.
Let \(V = \mathbb{R}^2\) and \(W = \mathbb{R}^3\) and let \[ F(x, y) = (x + y, 0, x - y). \]
Then
\begin{align*} F((x,y) + (w,z)) &= F((x+w, y+z)) \\ &= \big((x+w) + (y+z), 0, (x+w) - (y+z)\big) \\ &= \big(x+y, 0, x-y\big) + \big(w+z, w-z\big) \\ &= F(x, y) + F(w, z). \end{align*} \begin{align*} F(c(x, y)) &= F(cx, cy) = \big(cx+cy,cx-cy\big) \\ &= c \big(x+y, x-y\big) \\ &= c F(x, y). \end{align*}Thus \(F\) is linear.
Let \(F : V \to W\) be a linear map. Show that
Thus \(F\) preserves all the structure (addition, additive identity, additive inverse, scalar multiplication) just from the assumption that it preserves addition and scalar multiplication.
Let \(F : V \to W\) be linear. Show that for any finite linear combination we have \[ F(c^1 u_1 + \cdots c^k u_k) = c^1 F(u_1) + \dots + c^k F(u_k). \]
Let \(F : V \to W\) be a linear map. Show that the range \(F(V) = \{F(v) : v \in V\}\) is a subspace of \(W\).
Let \(F : V \to W\) be any function.
We say \(F\) is injective if \[ [F(u) = F(v)] \Rightarrow [u = v]. \] Equivalently, \(F\) is injective if \[ [u \neq v] \Rightarrow [F(u) \neq F(v)]. \]
We say \(F\) is surjective if for every \(w \in W\), there exists at least one \(u \in V\) such that \(F(u) = w\). That is if the range of \(F\) equals \(W\): \(F(V) = W\).
A function that is both injective and surjective is called a bijection.
Let \(w \in W\). Then \(F\) is injective means that there is at most one solution \(u\) of \(F(u) = w\), while \(F\) is surjective means that there exists at least one solution of \(F(u) = w\). In other words, injectivity is equivalent to uniqueness of solutions while surjectivity is equivalent to existence of solutions. A bijection is equivalent to both the existence and uniqueness of solutions.
Show that \(F : V \to W\) is a bijection if and only if there exists a function \(G: W \to V\) such that \(F \circ G = \operatorname{Id}_W\) and \(G \circ F = \operatorname{Id}_V\). The function \(G\) is denoted the inverse function and is usually written as \(F^{-1}\).
Let \(F : V \to W\) be a linear map. Then \(F\) is injective if and only if \([F(v) = 0] \Rightarrow [v = 0]\). That is, \(F\) is injective if and only if the only solution to \(F(v) = 0\) is the trivial solution \(v = 0\).
By an exercise above, \(F(0) = 0\).
Suppose \(F\) is injective. Then if \(F(v) = 0\) we have \(F(v) = F(0)\) hence by definition of injectivity, \(v = 0\).
Conversely, assume that \(F(v) = 0\) implies that \(v = 0\). For any \(u, w\) with \(F(u) = F(w)\) we then have \[ 0 = F(u) - F(w) = F(u - w). \] Thus by assumption \(u - w = 0\) and hence \(u = w\).
Let \(V = \mathbb{R}^2\) and \(W = \mathbb{R}^3\) and let \[ F(x, y) = (x + y, 0, x - y). \]
Then \(F\) is not surjective since \(F(x, y) = (0, 1, 0)\) has no solution \((x,y)\).
Suppose that \(F(x, y) = (0, 0, 0)\). Then \(x+y = 0\) and \(x-y = 0\) hence \(x = y = 0\). That is \((x, y) = (0, 0)\) and thus \(F\) is injective by the lemma.
A linear map \(F : V \to W\) is an isomorphism if there is a linear map \(G : W \to V\) such that \(F \circ G = \operatorname{Id}_W\) and \(G \circ F = \operatorname{Id}_V\). The function \(G\) is denoted the inverse function and is usually written as \(F^{-1}\). We say that \(F\) is invertible.
The linear map \(F(x, y) = (x + y, x - y)\) from \(\mathbb{R}^2\) to \(\mathbb{R}^2\) is an isomorphism. To see this we need to determine an inverse function \(F^{-1}\). Given \((u, v) \in \mathbb{R}^2\) to define the inverse \(F^{-1}(u, v)\), we need to solve \(F(x, y) = (u, v)\) for \((x, y)\) in terms of \((u, v)\). That is we need to solve
\begin{equation*} \begin{cases} x + y &= u \\ x - y &= v \end{cases} \end{equation*}The second equation gives \(y = x - v\). Subbing into the first equation gives \(x + (x-v) = u\) so that \(x = \tfrac{1}{2}(u + v)\). Then \(y = x - v = \tfrac{1}{2}(u - v)\). Notice that \((x, y)\) is determined uniquely. Thus we may unambiguously define \[ F^{-1}(u, v) = \left(\tfrac{1}{2}(u+v), \tfrac{1}{2}(u-v)\right). \] Direct substitution confirms that \(F \circ F^{-1} = \operatorname{Id}\) and \(F^{-1} \circ F = \operatorname{Id}\).
Let \(F : V \to W\) be a linear map. Then \(F\) is an isomorphism if and only if \(F\) is bijective.
Suppose \(F\) is an isomorphism. By an exercise above, \(F\) is bijective.
Conversely, suppose \(F\) is a bijective linear map. Then by the same exercise, the inverse \(F^{-1}\) exists. We just need to show that \(F^{-1}\) is linear. That is, we need to show that for any linear combination of two vectors, we have \[ F^{-1}(au + bv) = aF^{-1} (u) + b F^{-1} (v) \] To see this, we use linearity of \(F\) and bijectivity as follows:
\begin{align*} F\big(F^{-1}(au + bv)\big) &= au + bv \\ &= aF\big(F^{-1}(u)\big) + b F\big(F^{-1}(v)\big) \\ &= F\big(aF^{-1}(u) + b F^{-1}(v)\big). \end{align*}Thus since \(F\) is injective we have \[ F^{-1}(au + bv) = aF^{-1} (u) + b F^{-1} (v) \] as required.
Let \(F : V \to W\) be a linear map.
The linear map \(F(x, y) = (x + y, x - y)\) from \(\mathbb{R}^2\) to \(\mathbb{R}^2\) is an isomorphism by an exercise above. Consider the basis \(e_1 = (1, 0), e_2 = (0, 1)\). Then
\begin{align*} F(e_1) &= F(1, 0) = (1, 1) \\ F(e_2) &= F(0, 1) = (1, -1) \end{align*}By the previous lemma, we conclude that \(\{(1, 1), (1, -1)\}\) is basis.
For sums, there are inclusion maps:
Let \(V_1, \dots, V_k\) be a finite set of vector spaces. For each \(i = 1, \dots, k\) define the inclusions \(\iota_i : V_i \to V_1 \oplus \cdots \oplus V_k\)
\begin{align*} \iota_1(v_1) \in V_1 &\mapsto (v_1, 0, \dots, 0) \in V_1 \oplus \cdots \oplus V_k \\ & \vdots \\ \iota_i(v_i) \in V_i &\mapsto (0, \dots, 0, v_i, 0, \dots, 0) \in V_1 \oplus \cdots \oplus V_k \\ & \vdots \\ \iota_k(v_k) \in V_k &\mapsto (0, \dots, 0, v_k) \in V_1 \oplus \cdots \oplus V_k \end{align*}Show that each \(\iota_i\) is an injective linear map, hence isomorphic with it's image. Thus we may identify each \(V_i\) with it's image \(\iota_i(V_i)\). Verify that
\begin{align*} \iota_1(v_1) &= v_1 \oplus 0 \cdots \oplus 0 \\ & \vdots \\ \iota_i(v_i) &= 0 \oplus 0 \oplus v_i \oplus 0 \oplus \cdots \oplus 0 \\ & \vdots \\ \iota_n(v_n) &= 0 \oplus 0 \cdots \oplus 0 \oplus v_n\\ \end{align*}We also have projection maps.
Let \(V_1, \dots, V_k\) be a finite set of vector spaces. For each \(i = 1, \dots, k\) define the projections \(\pi_i : V_1 \oplus \cdots \oplus V_k \to V_i\) \[ \pi_i(v_1, \dots, v_k) \in V_1 \oplus \cdots \oplus V_k \mapsto v_i \in V_i \]
For finite dimensional vector spaces, situation sums have projection maps. For general vector spaces (i.e. not necessarily finite dimensional) this is no longer true. In that situation, one needs to work with products rather than sums. In finite dimensions sums and products are the same thing, so we won't belabour the point here. Just be aware that you may come across the distinction elsewhere and that isn't inconsistent with what we have done here.
Show that each \(\pi\) is a linear surjection.
Show that \(\pi_i\) is a right-inverse to \(\iota_i\); that is \(\pi_i \circ \iota_i = \operatorname{Id}_{V_i}\). Show that if \(k > 2\), the converse is not true; that is \(\iota_i \circ \pi_i \neq \operatorname{Id}_{V_1 \oplus \dots \oplus V_k}\).
Let \(F : V \to W\) be a linear map. The kernel of \(F\) is the set \[ \ker F = \{v \in V : F(v) = 0\}. \]
Show that \(F\) is injective if and only if \(\ker F = \{0\}\).
Let \(F(x, y) = (x+y, 0, x-y)\). Then \[ \ker F = \{(0, 0)\}. \] Thus \(F\) is injective.
Let \(F(x, y, z) = (x + z, y)\). Then \(F(x, y, z) = 0\) if and only if \(x + z = 0\) and \(y = 0\). Thus \[ \ker F = \{(x, y, z) : z = x, y = 0\} \] Then \(F\) is not injective.
Let \(F : V \to W\) be linear. Then \(\ker F\) is a subspace of \(V\).
Let \(u, v \in \ker F\) so that \(F(u) = F(v) = 0\). Then \[ F(u + v) = F(u) + F(v) = 0 + 0 = 0. \] Thus \(u + v \in \ker F\).
For any \(c \in \mathbb{R}\) we have \[ F(cu) = cF(u) = 0 \] hence \(cu \in \ker F\).
By a previous exercise, \(F(0) = 0\) and \(F(-u) = - F(u) = 0\) hence \(0 \in \ker F\) and \(-u \in \ker F\). Thus \(\ker F\) is closed under addition, additive inverse and scalar multiplication and contains the additive identity \(0\). Thus \(\ker F\) is a subspace.
The set \(\{(x, y, z) : z = -x, y = 0\}\) is a subspace of \(\mathbb{R}^3\) since it is the kernel of the linear map \(F(x, y, z) = (x+z, y)\).
Recall that for \(F : V \to W\) linear, the range of \(F\), \(F(V) = \{F(v) : v \in V\}\) is a subspace. Thus both the range \(F(V)\) and the kernel \(\ker F\) are subspaces. They are closely related via the following important theorem:
Let \(F : V \to W\) be a linear map between finite dimensional vector spaces. Then \[ \operatorname{dim} \ker F + \operatorname{dim} F(V) = \operatorname{dim} V. \]
Let \(V\) have dimension \(n\) and let \(\{u_1, \dots, u_k\}\) be a basis for \(\ker F\) and let \(\{v_1, \dots, v_l\}\) be a basis for \(F(V)\). By definition, there exists \(w_j \in V\) such that \(F(w_j) = v_j\), \(j = 1, \dots, l\). We claim that \(S = \{u_1, \dots, u_k, w_1, \dots, w_l\}\) is a basis for \(V\). Then since all bases have size \(n\) we will then obtain \(k + l = n\) as required.
To prove the claim, we will show that \(S\) is a linearly independent spanning set.
For linear independence, suppose there are constants \(c^1, \dots, c^k, d^1, \dots, d^l\) such that \[ c^1 u_1 + \cdots + c^k u_k + d^1 w_1 + \cdots + d^l w_l = 0. \] Then
\begin{align*} 0 &= F\big(c^1 u_1 + \cdots + c^k u_k + d^1 w_1 + \cdots + d^l w_l\big) \\ &= c^1 F(u_1) + \cdots + c^k F(u_k) + d^1 F(w_1) + \cdots + d^l F(w_l) \\ &= d^1 v_1 + \cdots + d^l v_l \end{align*}since \(u_i \in \ker F\). Since \(v_j\) are linearly independent we have \(d^j = 0\) for each \(j\) and hence \[ c^1 u_1 + \cdots + c^k u_k = 0. \] Since \(u_i\) are linearly independent, we have \(c^i = 0\) for each \(i\). Thus \(S\) is a linearly independent set.
To show \(S\) is a spanning set, let \(u \in V\) be arbitrary. Expand \(F(u)\) in the basis for \(F(V)\): \[ F(u) = \sum_{j=1}^l d^j v_j. \] Then note that \[ F\left(\sum_{j=1}^l d^j w_j\right) = \sum_{j=1}^l d^j F(w_j) = \sum_{j=1}^l d^j v_j = F(u). \] Thus \[ F\left(u - \sum_{j=1}^l d^j v_j\right) = 0 \] hence \[ u - \sum_{j=1}^l d^j v_j \in \ker F. \] Expanding in terms of the basis for \(\ker F\) we obtain \[ u - \sum_{j=1}^l d^j v_j = \sum_{i=1}^k c^i u_i. \] That is, \[ u = \sum_{i=1}^k c^i u_i + \sum_{j=1}^l d^j v_j \] and hence \(S\) is a spanning set.
Thus \(S\) is a basis and hence must have \(n\) elements since \(V\) is \(n\)-dimensional and the number of elements in any basis equals the dimension. On the other hand, \(S\) has \(k+l\) elements. Thus \(k + l = n\).
In the proof we did not use that \(W\) is finite dimensional, hence the result is true even if \(W\) is not finite dimensional. The important thing is that \(V\) is finite dimensional.
The rank-nullity theorem allows us to understand the space of solutions to linear equations.
Let \(T : V \to W\) be a linear map with \(V\) finite dimensional, and let \(w \in W\). Show that \(T(v) = w\) has a solution \(v\) if and only if \(w \in \operatorname{range} T = \{T(u) : u \in V\}\). Show moreover that the space of solutions is a vector subspace of \(V\) with dimension equal to \(\operatorname{dim} \ker T\).
Let \(V\) be a finite dimensional vector space and let \(B = \{e_1, \dots, e_n\}\) and \(C = \{u_1, \dots, u_n\}\) be bases for \(V\). The change of basis \(P_{BC}\) is the map defined by \(P_{BC}(e_i) = u_i\).
The change of basis \(P_{BC}\) is invertible.
For convenience, we write \(P\) for \(P_{BC}\). The image of \(P\) is the span of \(\{P(e_i) = u_i\}\), which equals \(V\) since \(\{u_i\}\) is a basis. Thus \(P\) is surjective and hence \(\dim \operatorname{range} P = n\). By rank-nullity, \[ n = \operatorname{dim} \ker P + \operatorname{dim} \operatorname{range} P = \operatorname{dim} \ker P + n. \] Thus \(\dim \ker P = 0\) and \(P\) is also injective.
Let \(T : V \to V\) be a linear map. Let \(B = \{e_1, \dots, e_n\}\) and \(C = \{u_1, \dots, u_n\}\) be bases for \(V\) with change of basis \(P_{BC}\). Let \(T_B\) and \(T_C\) denote the matrices with respect to the bases \(B, C\). Write \(P\) for the matrix \(P^j_i = \theta^j(P(e_i))\) where \(\{\theta^j\}\) is the dual basis to \(\{u_1, \dots, u_n\}\). Then \[ T_B = P^{-1} T_C P \] as a product of matrices.
Let \(X_1, \dots, X_n \in \mathbb{R}^n\). We write \(M(X_1, \dots, X_n)\) for the matrix with columns \(X_1, \dots, X_n\).
Let
\begin{equation*} X_1 = \begin{pmatrix} 2 \\ 0\end{pmatrix}, \quad X_2 = \begin{pmatrix} -5 \\ 1\end{pmatrix} \end{equation*}Then
\begin{equation*} M(X_1, X_2) = \begin{pmatrix} 2 & -5 \\ 0 & 1 \end{pmatrix} \end{equation*}Then there is a unique map \[ \det : \operatorname{Mat}_{n \times n} \to \mathbb{R} \] such that
The third condition says that whenever we swap two arguments, we introduce a minus. For example, with \(n = 3\) we have \[ \det M(X_1, X_3, X_2) = - \det M(X_1, X_2, X_3). \] Here we swapped the last two arguments (\(i=2, j=3)\) and so introduced a minus sign.
If \(X_i = X_j\) for some \(i \neq j\), then \(\det M = 0\). For example if \(X_1 = X_2\), then by swapping the first two arguments and then using \(X_1 = X_2\) we get, \[ \det M(X_1, X_2, X_3) = -\det M(X_2, X_1, X_3) = -\det M(X_1, X_2, X_3). \] Thus the real number \(\det M(X_1, X_2, X_3)\) equals it's own negative hence equals \(0\).
To prove the theorem we need the notion of permutation.
Let \(S = \{1, \dots, n\}\). A permutation of \(S\) is an invertible function \(\sigma : S \to S\). The set of all such permutations is denoted \(S_n\).
Think of permutations as rearranging the elements of \(S\): \((1, \dots, n) \mapsto (\sigma_1, \dots, \sigma_n)\). Here it's customary to write \(\sigma_i\) for \(\sigma(i))\). The fact that \(\sigma\) is invertible says that \(\sigma(S) = S\). That is, \(\{\sigma_1, \dots, \sigma_n\} = \{1, \dots, n\}\) but with the elements reordered.
Let \(S = \{1, 2, 3\}\) and let \(\sigma \in S_3\) be defined by
\begin{align*} \sigma(1) &= 2 \\ \sigma(2) &= 1 \\ \sigma(3) &= 3. \end{align*}The example is an example of a transposition . We transposed the elements \(1\) and \(2\) and left \(3\) untouched.
A transposition is a permutation \(\tau \in S_n\) such that \(\tau(i) = \tau(j)\) and \(\tau(j) = \tau(i)\) for some \(i \neq j\) and \(\tau\) is the identity on \(S \backslash \{i, j\}\).
The third condition on the map \(\det\) in the theorem above is that for any transposition \(\tau \in S_n\), \[ \det M (X_{\tau_1}, \dots, X_{\tau_n}) = -\det M(X_1, \dots, X_n). \]
Here is a fundamental result regarding permutations.
Let \(\sigma \in S_n\) be a permutation of a finite set. Then there exists transpositions \(\tau_1, \dots, \tau_k \in S_n\) representing \(\sigma\). That is, \[ \sigma = \tau_k \circ \cdots \circ \tau_1. \]
Moreover, the parity of the natural number \(k\) of transpositions is independent of the particular choice of transpositions \(\tau_1, \dots, \tau_k\).
Here the parity of a natural number \(k\) is defined to be
\begin{equation*} \operatorname{parity} k = \begin{cases} 0, & k \text{ even} \\ 1, & k \text{ odd} \end{cases} \end{equation*}Regarding the last statement of the lemma, note that the lemma asserts the existence of transpositions but makes no assertion about uniqueness. Indeed if say \(\sigma = \tau_1 \circ \cdots \circ \tau_k\) and \(\tau\) is any transposition then \[ \sigma = \tau \circ \tau \circ \tau_1 \circ \cdots \circ \tau_k \] expresses \(\sigma\) as a composition of \(k+2\) transpositions.
The last statement of the lemma asserts that although there may be many ways to write \(\sigma\) as a composition of a number of transpositions, the parity of this number is the always the same. Thus if \(\sigma\) may be written as a composition of an even number of transpositions, then every other possible way to write \(\sigma\) as a composition of transpositions also requires an even number of transpositions. Likewise for an odd number.
Let \(\sigma\) be a permutation of a finite set. The sign, \(\operatorname{sgm} \sigma\) of \(\sigma\) is \((-1)^p\) where \(p\) is the parity of the number of transpositions representing \(\sigma\). It is convenient to also define \(\operatorname{sgn} (\sigma) = 0\) if \(\sigma : X \to X\) is not invertible.
Thus for any map \(\sigma\) from a finite set to itself,
\begin{equation*} \operatorname{sgn} (\sigma) = \begin{cases} 1, & \sigma \text{ is an even composition of transpositions} \\ -1, & \sigma \text{ is an odd composition of transpositions} \\ 0, & \sigma \text{ is not invertible.} \end{cases} \end{equation*}Consider the permutation, \[ \sigma : (1, 2, 3, 4) \mapsto (2, 3, 1, 4). \] Then \[ \sigma = \tau_1 \circ \tau_2 \] where
\begin{align*} \tau_1 : (1, 2, 3, 4) &\mapsto (2, 1, 3, 4) \\ \tau_2 : (1, 2, 3, 4) &\mapsto (1, 3, 2, 4) \end{align*}Thus \(\operatorname{sgn} \sigma = (-1)^2 = (-1)^0 = 1\).
Finally, let's introduce some convenience notation.
Let \(X_1, \dots, X_n \in \mathbb{R}^n\) and \(\sigma \in S_n\). Then we write \[ \sigma(X_1, \dots, X_n) = (X_{\sigma_1}, \dots, X_{\sigma_n}). \]
Here we are now thinking of \(\sigma\) permuting the \(X_i\).
Let \(\sigma : (1, 2, 3) \mapsto (3, 1, 2)\). Then \[ \sigma(X_1, X_2, X_3) = (X_3, X_1, X_2). \]
Now we may prove the theorem. For ease of reference, here's the theorem again:
Then there is a unique map \[ \det : \operatorname{Mat}_{n \times n} \to \mathbb{R} \] such that
The strategy of proof is a common one: we will show that if such a map \(\det\) exists satisfying the properties 1,2,3 then it is given by an explicit formula. This establishes uniqueness but not existence (why?). Then we verify that the formula does indeed define a map \(\det\) satisfying the properties 1,2,3 hence also establishing existence.
Proof of Theorem
Let \(\{e_1, \dots, e_n\}\) be the standard basis. Given vectors \(X_1, \dots, X_n\) expand each vector in the basis to write \[ X_i = \sum_{j=1}^n X^j_i e_i \] with unique coefficients \(X^j_i \in \mathbb{R}\). Thus
\begin{equation*} M(X_1, \dots, X_n) = \begin{pmatrix} X_1^1 & \dots & X_n^1 \\ \vdots & \ddots & \vdots \\ X_1^n & \dots & X_n^n \\ \end{pmatrix} \end{equation*}Suppose that \(\det\) is a map satisfying the properties 1,2,3.
First, consider \(\det M(e_{i_1}, \dots, e_{i_n})\) for \(1 \leq i_1, \dots, i_n \leq n\). If we have any repeats, e.g. \(i_1 = i_2\), then since \(\det\) is alternating (property 3),
\begin{align*} \det M (e_{i_1}, e_{i_2}, \dots, e_{i_n}) &= -\det M (e_{i_2}, e_{i_1}, \dots, e_{i_n}) \\ &= - \det M (e_{i_1}, e_{i_2}, \dots, e_{i_n}). \end{align*}Therefore \[ \det M(e_{i_1}, e_{i_2}, \dots, e_{i_n}) = 0. \] More generally, given \(e_{i_1}, \dots, e_{i_n}\), if \(i_k = i_j\) for some \(k \neq j\), then \[ \det M(e_{i_1}, \dots, e_{i_n}) = 0 = \operatorname{sgn} \sigma_{i_1, \dots, i_n} \] where \(\sigma_{i_1, \dots, i_n}\) is the map \[ \sigma_{i_1,\dots, i_n} (k) = i_k, \quad 1 \leq k \leq n. \]
On the other hand for \(\sigma \in S_n\), writing \(\sigma = \tau_k \circ \dots \circ \tau_1\), since \(\det\) is alternating, we have
\begin{align*} \det M(e_{\sigma_1}, \dots, e_{\sigma_n}) &= \det M (\sigma(e_1, \dots, e_n)) \\ &= \det M (\tau_k \circ \dots \tau_1 (e_1, \dots, e_n)) \\ &= - \det M (\tau_{k-1} \circ \dots \tau_1 (e_1, \dots, e_n)) \\ &= (-1)^2 \det M (\tau_{k-2} \circ \dots \tau_1 (e_1, \dots, e_n)) \\ &= \quad \vdots \\ &= (-1)^k \det M (e_1, \dots, e_n) \\ &= \operatorname{sgn} (\sigma) \end{align*}with the latter equality following from the definition of \(\operatorname{sgn}\sigma\) and from property 1 which implies \(\det M (e_1, \dots, e_n) = \det \operatorname{Id} = 1\).
Then by multilinearity,
\begin{align*} \det M(X_1, \dots, X_n) &= \det M\left(\sum_{j_1=1}^n X^{j_1}_1 e_{j_1}, \dots, \sum_{j_n=1}^n X^{j_n}_1 e_{j_n}\right) \\ &= \sum_{j_1=1}^n \cdots \sum_{j_n=1}^n X^{j_1}_1 \cdots X^{j_n}_n \det M\big(e_{j_1}, \dots, e_{j_n}\big) \\ &= \sum_{j_1=1}^n \cdots \sum_{j_n=1}^n X^{j_1}_1 \cdots X^{j_n}_n \operatorname{sgn} \sigma_{j_1, \dots, j_n} \\ \end{align*}where \(\sigma_{j_1, \dots, j_n}\) is the map \[ \sigma_{j_1, \dots, j_n} (i) = j_i. \]
This formula uniquely defines \(\det\).
Here is an important result. You can tell it's important from the number of corollaries presented after!
Let \(A\), \(B\) be \(n \times n\) matrices. Then \[ \det AB = \det A \det B. \]
Prove the lemma!
Let \(A\) be an invertible \(n \times n\) matrix. Then \(\det A \neq 0\) and \[ \det A^{-1} = \frac{1}{\det A}. \]
Taking the determinant of \(\operatorname{Id} = A^{-1} A\) we have
\begin{align*} 1 &= \det \operatorname{Id} = \det (A^{-1} A) \\ &= \det A^{-1} \det A \end{align*}Thus \(\det A \neq 0\) and dividing both sides by \(\det A\) we get \[ \det A^{-1} = \frac{1}{\det A} \]
An \(n \times n\) matrix \(A\) is invertible if and only if \(\det A \neq 0\).
We just established that if \(A\) is invertible, then \(\det A \neq 0\).
Conversely, suppose \(A\) is not invertible, hence the columns, \(X_1, \dots, X_n\) of \(A\) are linearly dependent. Thus \[ c^1 X_1 + \dots + c^n X_n = 0 \] for some scalars \(c^1, \dots, c^n\) with at least one \(c_i \neq 0\). Without loss of generality, assume \(c^1 \neq 0\). Then \[ X_1 = -\frac{c_2}{c_1} X_2 + \dots + -\frac{c_n}{c_1} X_n. \] Then by multilinearity,
\begin{align*} \det A &= \det M(X_1, \dots, X_n) \\ &= \det M \left(-\frac{c_2}{c_1} X_2 + \dots + -\frac{c_n}{c_1} X_n, X_2, \dots, X_n\right) \\ &= - \sum_{j=2}^n \frac{c_j}{c_1} \det M(X_j, X_2, \dots, X_n) \\ &= 0. \end{align*}since each term in the sum at the end contains the repeated vector \(X_j\).
Let \(A\) be a matrix and \(P\) an invertible matrix. Then \[ \det P^{-1} A P = \det A. \]
We have
\begin{align*} \det P^{-1} A P &= \det P^{-1} \det A \det P \\ &= \frac{1}{\det P} \det A \det P \\ &= \det A. \end{align*}Let \(T : V \to V\) be a linear map on a finite dimensional vector space \(V\). Let \(\mathcal{B} = \{e_1, \dots, e_n\}\) be a basis for \(V\) and let \(A_{\mathcal{B}}\) be the matrix with components \((A_{\mathcal{B}})^i_j = \alpha^i (T(e_j))\). That is, \(A_{\mathcal{B}}\) is the matrix representation for \(T\) with respect to the basis \(\mathcal{B}\). Here \(\{\alpha^j\}\) is the dual basis to \(\mathcal{B}\).
The determinant of \(T\) is defined to be \[ \det T = \det A_{\mathcal{B}}. \]
Equivalently, \[ \det T = \det M(T(e_1), \dots, T(e_n)). \]
Note that the two definitions are equivalent, since the columns of \(A_{\mathcal{B}}\) are precisely \(T(e_i)\) hence \(A_{\mathcal{B}} = M(T(e_1), \dots, T(e_n))\).
The determinant of \(T\) is independent of the basis chosen.
Let \(B, C\) be bases with \(P\) the change of basis matrix. Let \(A_B, A_C\) denote the matrices representing \(T\) with respect to the bases \(B, C\) respectively. Then \(A_C = P^{-1} A_B P\). Thus \[ \det A_C = \det P^{-1} A_B P = \det A_B. \]
The lemma says that to define the determinant of a linear map \(V \to V\), we may pick any basis and compute the determinant of the corresponding matrix. The result will be same regardless of which basis we chose.
The Euclidean inner product is the function \(\mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}\), \[ \langle u, v\rangle = u_1 v_1 + \dots + u_n v_n \] where \(u = (u_1, \dots, u_n)\) and \(v = (v_1, \dots, v_n)\).
The Euclidean inner product satisfies the following properties
Prove the lemma. It all follows directly from the definition.
Some particular cases are
From the Euclidean inner product we can define the Euclidean norm, which is a measure of the size of a vector.
The Euclidean norm on \(\mathbb{R}^n\) is the function \(\mathbb{R}^n \to \mathbb{R}\), \[ \|v\| = \sqrt{\langle v, v \rangle} = \sqrt{v_1^2 + \cdots + v_n^2} \] where \(v = (v_1, \dots, v_n)\).
By positivity of the inner product, we have \[ \|v\|^2 = \langle v, v \rangle. \]
The Euclidean norm satisfies the following properties:
For any \(u, v \in \mathbb{R}^n\) and any \(c \in \mathbb{R}\) we have
This follows directly from the definition. If you haven't seen this before - check it! You'll only need the back of an envelope, a napkin, a drinks coaster, etc. to do it. Unlike Fermat's last theorem, the margin of a page should also suffice.
Perhaps the most important property of the inner product is the Cauchy Schwartz inequality.
For all \(u, v \in \mathbb{R}^n\), \[ \langle u, v \rangle \leq \|u\| \|v\|. \]
Equality holds if and only if one of \(u,v\) is a scalar multiple of the other.
Before proving the statement, let us note that for any non-zero vector \(u \neq 0\) we have the unit length vector \(\tfrac{u}{\|u\|}\) in the direction \(u\). This vector has unit length by positive homogeneity of the norm,
\begin{align*} \left\|\tfrac{u}{\|u\|}\right\| &= \left\|\tfrac{1}{\|u\|} u\right\| \\ &= \left|\frac{1}{\|u\|}\right| \|u\| \\ &= \frac{\|u\|}{\|u\|} = 1. \end{align*}If either \(u = 0\) or \(v = 0\), both sides of the equation are zero hence the inequality becomes \(0 \leq 0\) which is indeed true. Moreover, we have equality. In this case note that if say \(u \neq 0\), then \(v = 0 = 0 \cdot u\) hence \(v\) is a scalar multiple of \(u\).
Now assume that both \(u \neq 0\) and \(v \neq 0\). By positivity and bilinearity of the inner product, for any \(u, v\) we have
\begin{align*} 0 &\leq \langle u - v, u - v \rangle \\ &= \langle u, u\rangle - 2\langle u, v\rangle + \langle v, v\rangle \\ &= \|u\|^2 + \|v\|^2 - 2 \langle u, v \rangle. \end{align*}Rearranging we get \[ \langle u, v \rangle \leq \frac{1}{2} \big(\|u\|^2 + \|v\|^2\big). \] In particular, applying this inequality to the unit vectors in the directions \(u, v\) we obtain \[ \frac{1}{\|u\|\|v\|} \langle u, v \rangle = \big\langle \frac{u}{\|u\|}, \frac{v}{\|v\|} \big\rangle \leq \frac{1}{2} \big(1 + 1\big) = 1. \] The first equality is from bilinearity of the inner product. Multiplying through by \(\|u\|\|v\|\) then yields the result.
For the equality case, suppose that \(\langle u, v \rangle = \|u\| \|v\|\). Dividing both sides by \(\|u\|\|v\|\) we obtain \[ \big\langle \tfrac{u}{\|u\|} \tfrac{v}{\|v\|}\big\rangle = 1. \] Then
\begin{align*} \big\langle \tfrac{u}{\|u\|} - \tfrac{v}{\|v\|}, \tfrac{u}{\|u\|} - \tfrac{v}{\|v\|}\big \rangle &= \big\|\tfrac{u}{\|u\|}\|^2 + \big\|\tfrac{v}{\|v\|}\|^2 - 2 \big\langle \tfrac{u}{\|u\|}, \tfrac{v}{\|v\|}\big\rangle \\ &= 1 + 1 - 2 \big\langle \tfrac{u}{\|u\|}, \tfrac{v}{\|v\|}\big\rangle \\ &= 1 + 1 - 2 = 0. \end{align*}By positivity of the Euclidean inner product, we then have \[ \tfrac{u}{\|u\|} - \tfrac{v}{\|v\|} = 0. \] Thus \[ v = \frac{\|v\|}{\|u\|} u \] and hence \(v\) is a scalar multiple of \(u\).
The theorem actually immediately implies the (a priori) strong statement that \[ \big|\langle u, v \rangle\big| \leq \|u\| \|v\|. \] To see this, by replacing \(u\) by \(-u\) we obtain \[ -\langle u, v \rangle = \langle -u, v\rangle \leq \|-u\| \|v\| = \|u\|\|v\|. \] Since \(|\langle u, v\rangle| = \pm \langle u, v\rangle\) (depending on whether \(\langle u, v\rangle\) is positive or negative), we obtain the result.
Triangle Inequality: \(\|u + v\| \leq \|u\| + \|v\|\).
Using bilinearity of the inner product, \(\|w\|^2 = \langle w, w\rangle\), and Cauchy-Schwartz we have
\begin{align*} \|u + v\|^2 &= \langle u + v, u + v\rangle \\ &= \|u\|^2 + 2\langle u, v\rangle + \|v\|^2 \\ &\leq \|u\|^2 + 2\|u\|\|v\| + \|v\|^2 \\ &= \big(\|u\| + \|v\|\big)^2. \end{align*}Taking square roots and using the fact that the square root is an increasing function on \([0, \infty)\) we obtain the result.
Finally, Cauchy-Schwartz allows us to define the angle between vectors.
Let \(u, v \neq 0\). Then we define \(\theta\) by \[ \cos \theta = \big\langle \tfrac{u}{\|u\|}, \tfrac{v}{\|v\|}\big\rangle. \] Note that the quantity on the right hand side is between \(-1\) and \(-1\). Thus choosing a branch on which \(\cos\) is invertible (e.g. \(\theta \in (0, \pi)\), there is a unique \(\theta\) in this range satisfying the definition and hence \(\theta\) is well defined. In other words, on such a branch we define \[ \theta = \cos^{-1} \left(\big\langle \tfrac{u}{\|u\|}, \tfrac{v}{\|v\|}\big\rangle\right). \]
Throughout this section \(V\) will denote a vector space equipped with an inner product \(\langle \cdot, \cdot \rangle\).
Let \(u, v \in V\). We say that \(u, v\) are orthogonal (written \(u \perp v\) if \(\langle u, v \rangle = 0\).
For \(V = \mathbb{R}^2\) equipped with the Euclidean inner product, let \(u = (1, 0)\), \(v = (0, 1)\), \(w = (1, 1)\), \(z = (-1, 1)\).
Then \(u \perp v\), \(w \perp z\) but no other pair are orthogonal.
Let \(u \perp v\) with \(u,v \neq 0\). Then \(\{u, v\}\) is a linearly independent set.
Let \(a, b \in \mathbb{R}\) be such that \(a u + b v = 0\). Then
\begin{align*} 0 &= \langle a u + b v, u \rangle = a \langle u, u \rangle + b \langle v, u \rangle \\ &= a \|u\|^2. \end{align*}Since \(u \neq 0\) we have \(\|u\|^2 \neq 0\) hence \(a = 0\). Similarly, \(b = 0\). Thus \(\{u, v\}\) is linearly independent.
Let \(U \subseteq V\) be a subspace. Show that the subset \[ U^{\perp} = \{w \in V : \forall u \in U, \langle w, u\rangle = 0 \} \] is a subspace.
For any subspace \(U \subseteq V\) we have \(U \cap U^{\perp} = \{0\}\).
Let \(v \in U \cap U^{\perp}\). Then \(v \in U^{\perp}\) implies that \(\langle v, u\rangle = 0\) for every \(u \in U\). In particular, since we also have \(v \in U\) letting \(u = v\) we obtain \(\langle v, v \rangle = 0\). By positivity of the inner-product, \(v = 0\).
For any \(u,v \in U\) we have \[ v - \left\langle v, \tfrac{u}{\|u\|}\right\rangle \frac{u}{\|u\|} \perp u \] since
\begin{align*} \left\langle v - \left\langle v, \tfrac{u}{\|u\|}\right\rangle \tfrac{u}{\|u\|}, u \right \rangle &= \langle v, u \rangle - \left\langle \left\langle v, \tfrac{u}{\|u\|}\right\rangle \tfrac{u}{\|u\|}, u \right \rangle \\ &= \langle v, u \rangle - \left\langle v, \tfrac{u}{\|u\|}\right\rangle \left\langle \tfrac{u}{\|u\|}, u \right \rangle \\ &= \langle v, u \rangle - \frac{1}{\|u\|^2} \langle v, u\rangle \langle u, u \rangle \\ &= 0. \end{align*}Let \(u, v \in V\). The orthogonal projection of \(v\) onto \(u\) is the vector \[ \pi_u(v) = \left\langle v, \tfrac{u}{\|u\|}\right\rangle \frac{u}{\|u\|}. \]
The orthogonal projection of \(v\) onto the orthogonal complement of \(u\) is the vector \[ pi_u^{\perp}(v) = v - \pi_v(u). \]
By definition, \(\pi_u(v)\) is a scalar multiple of \(u\). By the calculation just above, \(\pi_u^{\perp}(v) = v - \pi_v(u) \perp u\). We may then write \[ v = \pi_u(v) + \pi_u^{\perp}(v), \] expressing \(v\) as a linear combination of \(u\) and a vector orthogonal to (hence linearly independent of) \(u\).
Show that \(\pi_u^{\perp}(v) = 0\) if and only if \(\pi_u(v) = v\) if and only if \(v = c u\) for some \(c \in \mathbb{R}\).
One way to think about the preceding discussion is to consider \(U\), the (one-dimensional) subspace spanned by \(\{u\}\). In this case, \(\pi_U(v) := \pi_u(v)\) is the orthogonal projection of \(v\) onto \(U\) and \(\pi_U^{\perp}(v) := \pi_u^{\perp}(v) \in U^{\perp}\) is the component of \(v\) orthogonal to \(U\). We can generalise this to arbitrary finite dimensional subspaces \(U\).
First we need to discuss orthonormal bases and the Gram-Schmidt process.
Let \(V\) be a finite dimensional vector space equipped with an inner-product. A basis \(\{e_1, \dots, e_n\}\) for \(V\) is called orthonormal if each \(e_i\) has unit length and the \(\{e_i\}\) are pairwise orthogonal; i.e. \(e_i \perp e_j\) for \(i \neq j\).
Concisely, a basis \(\{e_1, \dots, e_n\}\) is orthonormal if \(\langle e_i, e_j\rangle = \delta_{ij}\) where
\begin{equation*} \delta_{ij} := \begin{cases} 1, & i = j \\ 0, & i \neq j. \end{cases} \end{equation*}Gram-Schmidt : Let \(V\) be a finite dimensional vector space equipped with an inner product. Then \(V\) possesses an orthonormal basis.
The proof is by induction on dimension.
Let \(V\) be a \(1\) dimensional vector space with basis \(\{v_1\}\). Define \[ e_i = \frac{v_1}{\|v_1\|}. \] Then for any \(v \in V\) we may write \(v = c v_1 = \tfrac{c}{\|v_1\|} e_1\) so that \(e_1\) is a basis. By definition it is unit length and since there are no other basis elements it is vacuously orthogonal to all other basis elements. That is \(\{e_1\}\) is an orthonormal basis for \(V\).
To see how the induction will work, let us do the \(n=2\) case. Let \(\{v_1, v_2\}\) be a basis for \(V\). Define \(e_1\) as above and define \[ e_2 = \frac{\pi_{e_1}^{\perp}(v_2)}{\|\pi_{e_1}^{\perp}(v_2)\|}. \] Note here that \[ \pi_{e_1}^{\perp}(v_2) = v_2 - \langle v_2, e_1 \rangle e_1 \neq 0 \] since otherwise we would have a non-trivial linear combination of \(e_1, e_2\) equalling \(0\) which contradicting that \(e_1, e_2\) are linearly independent. Thus we have \(e_1, e_2\) unit length vectors with \(e_1 \perp e_2\). Since \(e_1, e_2\) are orthogonal, they are linearly independent hence a basis since \(V\) is two-dimensional. Thus we have our orthonormal basis.
Now for the induction step. Suppose the result is true for in \(n\) dimensions. Let \(V\) be \(n+1\) dimensional with basis \(\{v_1, \dots, v_{n+1}\}\). The subspace \(V' = \operatorname{span} \{v_1, \dots, v_n\}\) is \(n\)-dimensional, hence by the inductive hypothesis \(V'\) posses an orthonormal basis \(\{e_1, \dots, e_n\}\). Since \(v_{n+1} \notin V'\) (otherwise \(V\) would have dimension \(n\)), \(\pi_{V'}^{\perp}(v_{n+1}) \neq 0\) - check this! Define \[ e_{n+1} = \frac{\pi_{V'}(v_{n+1})}{\|\pi_{V'}(v_{n+1})\|}. \] Then \(\{e_1, \dots, e_{n+1}\}\) is our desired orthonormal basis.
Fill in the details of the proof. That is, show that \(v_{n+1} \notin V'\) and hence show that \(\pi_{V'}^{\perp}(v_{n+1}) \neq 0\). Finally verify that \(\{e_1, \dots, e_{n+1}\}\) is an orthonormal basis for \(V\).
Let \(V = \mathbb{R}^2\) be equipped with the Euclidean inner product and consider the basis \(v_1 = (1, 1)\) and \(v_2 = (2, 0)\). The Gram-Schmidt process proceeds as follows: \[ e_1 = \frac{1}{\sqrt{2}} (1, 1). \]
\begin{align*} \pi_{e_1}^{\perp}(v_2) &= (2, 0) - \left\langle (2, 0), \frac{1}{\sqrt{2}} (1, 1)\right\rangle \frac{1}{\sqrt{2}} (1, 1) \\ &= (2, 0) - 2 \frac{1}{2} (1, 1) \\ &= (1, -1). \end{align*}\[ e_2 = \frac{\pi_{e_1}^{\perp} (v_2)}{\|\pi_{e_1}^{\perp} (v_2)\|} = \frac{1}{\sqrt{2}} (1, -1). \]
Thus from the basis \(\{(1, 1), (2, 0)\}\) we obtain the orthonormal basis \(\left\{\tfrac{1}{\sqrt{2}} (1, 1), \tfrac{1}{\sqrt{2}} (1, -1)\right\}\).
Now we can construct orthogonal projections onto finite dimensional subspaces.
Let \(V\) be a vector space equipped with an inner product. Let \(U \subseteq V\) be a finite dimensional subspace and let \(v \in V\). Then there exists unique vectors \(\pi_U(v) \in U\) and \(\pi_U^{\perp}(v) \in U^{\perp}\) such that \[ v = \pi_U(v) + \pi_U^{\perp}(v). \]
We start with uniqueness first. Suppose that \[ v = v_1 + w_1 = v_2 + w_2 \] with \(v_i \in U\) and \(w_i \in U^{\perp}\). Then \[ v_1 - v_2 = w_2 - w_1. \] Now \(v_1, v_2 \in U\) which is a subspace hence \(v_1 - v_2 \in U\). Likewise, \(w_1, w_2 \in U^{\perp}\) which is a subspace hence \(w_2 - w_1 \in U^{\perp}\). Then \[ v_1 - v_2 = w_2 - w_1 \in U \cap U^{\perp} = \{0\}. \] Hence \(v_1 = v_2\) and \(w_1 = w_2\) proving uniqueness.
For existence, let \(\{e_1, \dots, e_k\}\) be an orthonormal basis for \(U\). Define
\begin{align*} \pi_U(v) &= \sum_{i=1}^k \pi_{e_i}(v) \\ &= \langle v, e_i\rangle e_i. \end{align*}Then \(\pi_U(v)\) is a linear combination of the basis vectors \(e_i\) hence \(\pi_U(v) \in U\).
Define \(\pi_U^{\perp}(v) = v - \pi_U(v)\). Then for each \(j = 1, \dots, k\),
\begin{align*} \langle \pi_U^{\perp}(v), e_j \rangle & \left\langle v - \sum_{i=1}^k \langle v, e_i\rangle e_i, e_j \right\rangle \\ &= \langle v, e_j\rangle - \sum_{i=1}^k \langle v, e_i\rangle \langle e_i, e_j\rangle \\ &= \langle v, e_j\rangle - \langle v, e_j \rangle \\ &= 0. \end{align*}Therefore \(\pi_U^{\perp}(v) \perp e_j\) for each of the basis elements \(e_j\), hence \(\pi_U^{\perp}(v) \perp U\). To get the second last line, we used that \(\langle e_i, e_j \rangle\) equals \(0\) for \(i \neq j\) and equals \(1\) for \(i = j\). Thus in the sum, all terms vanish except for the term with \(i = j\).
Thus we have \(\pi_U(v) \in U\) and \(\pi_U^{\perp}(v) = v - \pi_U(v) \in U^{\perp}\). By definition of \(\pi_U^{\perp}(v)\), \(v = \pi_U(v) + \pi_U^{\perp}(v)\) giving existence.
We don't require \(V\) to be finite dimensional but we did require that the subspace \(U\) is finite dimensional. Notice that if \(V\) is not finite dimensional, the proof holds as written, but we can also observe that the span of \(U \cup \{v\}\) is finite dimensional and we could just work entirely within this span. That is (although we didn't need to) by replacing \(V\) with this span, we could have assumed that \(V\) was finite dimensional.
So if \(V\) is not finite dimensional, everything works as described. A natural question then is to ask what happens if \(U\) is not finite dimensional. This is outside the scope of our discussion here, but let us just note that the result is not true in general if \(U\) is infinite dimensional. Additional assumptions are required to define orthogonal projections; the standard assumption being that of completeness, i.e. \(V\) is a Hilbert space.
Show that
Show also that if \(V\) is finite dimensional (so that both of \(U, U^{\perp}\) are finite dimensional), then \(\pi_{U^{\perp}} = \pi_U^{\perp}\).