The Inverse Function Theorem is a cornerstone theorem in nonlinear analysis. It is essential for the development of sub-manifolds. There are several equivalent formulations such as the Implicit Function Theorem, the Immersion Theorem and the Submersion theorem. We will discuss each of these and see that they are in fact equivalent to each other. We won't focus too much on the proof itself however as this is result best dealt with in an analysis course.
Recall the following definition:
A function \(f : U \subseteq \mathbb{R}^n \to V \subseteq \mathbb{R}^n\) with \(U, V\) open is called a diffeomorphism if it is \(C^{\infty}\) with \(C^{\infty}\) inverse.
Recall also that a function \(f : U \to V\) is invertible if and only if it is bijective, in which case the inverse function \(f^{-1} : V \to U\) is defined by \[ f^{-1}(q) = p, \text{where \(p\) is the unique solution of \(f(p) = q\).} \]
Show that
Recall the \(1\)-d inverse function theorem:
Let \(f : \mathbb{R} \to \mathbb{R}\) be a smooth function with \(f'(x_0) \ne 0\). Then there exists an interval \(I\) containing \(x_0\) and an interval \(J\) containing \(f(x_0)\) so that \(f : I \to J\) is a diffeomorphism.
Generalising to arbitrary dimensions we have:
Inverse Function Theorem
Let \(f : \mathbb{R}^n \to \mathbb{R}^n\) a smooth function such that \(df_{x_0}\) is invertible at \(x_0\). Then there is an open set \(U\) containing \(x_0\) and an open set \(V\) containing \(f(x_0)\) such that \(f|_U : U \to V\) is a diffeomorphism. Moreover \[ df^{-1}_{f(x_0)} = (df_{x_0})^{-1} \]
Note that if \(f\) is a diffeomorphism, then \(f^{-1} \circ f (x) = x\). That is, \(f^{-1} \circ f = \operatorname{Id}_x\). Since \(d\operatorname{Id}_x = \operatorname{Id}_n\), by the chain rule we have \[ \operatorname{Id}_n = d\operatorname{Id}_x = d(f^{-1} \circ f)_{x_0} = df^{-1}_{f(x_0)} \cdot df_{x_0}. \] That is \(df_{x_0}\) is invertible and \[ (df_{x_0})^{-1} = df^{-1}_{f(x_0)}. \] That is, \(df^{-1}_{y_0}\) at \(y_0 = f(x_0)\) is necessarily equal to \((df_{x_0})^{-1}\) at \(x_0\). In one dimension, \(df = f'\) and \(d(f^{-1}) = 1/f'\).
The basic idea is that if \(df\) is invertible, then \(f\) is invertible to first order. Writing \[ f(x) = f(x_0) + df_{x_0} \cdot (x - x_0) + o(\|x-x_0\|). \] Let us ignore the \(o(|x-x_0|)\) term (after all, it's insignificant compared with everything else for \(x\) near \(x_0\)!) and assume \[ f(x) = f(x_0) + df_{x_0} \cdot (x - x_0). \] Then since \(df_{x_0}\) is invertible, we can rearrange to solve for \(x\) to get \[ x = x_0 + df_{x_0}^{-1} (f(x) - f(x_0)). \] Writing \(y = f(x)\) and \(y_0 = f(x_0)\) we obtain the inverse, \(f^{-1}(y) = x\): \[ f^{-1}(y) = f^{-1}(y_0) + df_{x_0}^{-1} \cdot (y - y_0). \]
The task then is to work out how to deal with the presense of the \(o(\|x-x_0\|)\) term. Dealing with this is a cornerstone result of analysis and we'll not go into details here, though below we discuss it briefly. It is worth noting that the approximation \(f(x) \simeq f(x_0) + df_{x_0} \cdot (x - x_0)\) is only guaranteed to be reasonable provided the \(o(\|x-x_0\|)\) error term is small, which requires \(x\) close to \(x_0\) which at least is suggestive as to why the theorem is only local in general.
For now, here is an example application of the Inverse Function Theorem.
Consider
\begin{equation*} \begin{cases} x - y^2 &= a \\ x^2 + y + y^3 &= b \end{cases} \end{equation*}For \((a, b) = (0, 0)\): \((x, y) = (0, 0)\) is a solution.
Question : For what \((a, b)\) is the system solvable?
To answer the question, let \(F(x, y) = (x - y^2, x^2 + y + y^3)\). Then
\begin{equation*} dF = \begin{pmatrix} 1 & 2y \\ 2x & 1 + 3y^2 \end{pmatrix} \end{equation*}We have \(dF_{(0, 0)} = \operatorname{Id}\) hence by the IFT there is a neighbourhood of \((x, y) = (0, 0)\) for which \(F\) maps diffeomorphically onto a neighbourhood of \((a, b) = (0, 0)\). Therefore, for \((a, b)\) in the latter neighbourhood of \((0, 0)\), the former neighbourhood of \((0, 0)\) contains a unique solution of \(F(x, y) = (a, b)\).
Note that given \((a, b))\), there is not generally a unique solution. In fact, even for \((a, b) = (0, 0)\) there is not a unique solution, since if \(y\) is a real root of \(y^3 + y^2 + 1\), then \(F(y^2, y) = (0, 0)\). Such a root always exists since \(y^3 + y^2 + 1\) is an odd-degree polynomial.
Using the isomorphism, \(\mathbb{R}^n \oplus \mathbb{R}^k \simeq \mathbb{R}^{n+k}\) we may write a point in \(\mathbb{R}^{n+k}\) as \((x, y)\) with \(x \in \mathbb{R}^n\) and \(y \in \mathbb{R}^k\). Then for a function \(F = F(x, y) : \mathbb{R}^{n+k} \to \mathbb{R}^k\) we also split the differential \(dF = d_x F \oplus d_y F\) into \(x,y\) parts:
\begin{equation*} dF = \begin{pmatrix} \underbrace{ \begin{matrix} \partial_{x_1} F^1 & \dots & \partial_{x_n} F^1 \\ \vdots & \ddots & \vdots \\ \partial_{x_1} F^k & \dots & \partial_{x_n} F^k \end{matrix} }_{d_x F} & \underbrace{ \begin{matrix} \partial_{y_1} F^1 & \dots & \partial_{y_k} F^1 \\ \vdots & \ddots & \vdots \\ \partial_{y_1} F^k & \dots & \partial_{y_k} F^k \end{matrix} }_{d_y F} \end{pmatrix} \end{equation*}Here \(d_xF\) denotes the derivatives of \(F\) with respect to the \(x = (x_1, \dots, x_n)\) variables, which is the \(k \times n\) block on the left, and \(d_y F\) denotes the derivatives of \(F\) with respect to the \(y = (y_1, \dots, y_k)\) variables, which is the \(k \times k\) block on the right. Keep in mind that each of \(\partial_{x_i} F^a\) and \(\partial_{y_j} F^a\) are functions of both \(x\) and \(y\).
Let \(F : \mathbb{R}^{n+k} \to \mathbb{R}^k\) be smooth with \((x_0, y_0)\) such that \(d_y F|_{(x_0, y_0)}\) is invertible. Then there is an open neighbourhood \(U\) of \(x_0\) and neighbourhood \(V\) of \(y_0\) along with a unique smooth function \(g : U \to V\) such that on the open set \(U \times V \subseteq \mathbb{R}^n \oplus \mathbb{R}^k\), \((x, g(x))\) is the unique solution to \(F(x, y) = F(x_0, y_0)\). That is \[ F(x, g(x)) = F(x_0, y_0), \] and if \((x, y) \in U \times V\) is such that \(F(x, y) = F(x_0, y_0)\), then \(y = g(x)\).
Since \(d_y F\) is invertible if and only if the columns of \(d_y F\) are linearly independent, \(dF\) has rank at least \(k\). But the rank of \(dF\) is equal to \(\dim \operatorname{range} dF\) which, being a subspace of \(\mathbb{R}^k\) is at most \(k\). Thus the rank of \(dF\) is exactly \(k\).
The conclusion of theorem is that the level set \(F(x, y) = F(x_0, y_0)\) is locally the graph \(\{y = g(x)\}\) in a neighbourhood of \((x_0, y_0)\).
The Implicit Function Theorem is equivalent to the Inverse Function Theorem. Here we show how to derive the Implicit Function Theorem from the Inverse Function Theorem.
Define \[ \bar{F}(x, y) = (x, F(x, y)) \in \mathbb{R}^{n+k} \] Then
\begin{equation*} d\bar{F} = \begin{pmatrix} \operatorname{Id}_n & 0 \\ \ast & d_y F \end{pmatrix} \end{equation*}is invertible at \((x_0, y_0)\) since the assumption is that \(d_y F\) is invertible at \((x_0, y_0)\). Hence by the inverse function theorem, \(\bar{F}\) is locally invertible.
Write \(\bar{F}^{-1} = (H, G)\), from which we claim that necessarily \(H(x, y) = x\). Since \(\bar{F}(x, y) = (x, F(x, y))\), by the definition of inverse functions,
\begin{equation*} \begin{split} (x, y) &= \bar{F} \circ \bar{F}^{-1} (x, y) \\ &= \bar{F} (H(x, y), G(x, y)) \\ &= (H(x, y), F(G(x, y))). \end{split} \end{equation*}Comparing the first component of the left and right hand sides we see that \(x = H(x, y)\) as claimed. Comparing the second component, we get \(y = F(G(x, y))\). Thus \[ \bar{F}^{-1}(x, y) = (x, G(x, y)) \] for a smooth function \(G : \mathbb{R}^{n+k} \to \mathbb{R}^k\) satisfying \(F(G(x, y)) = y\).
Now let \(c = F(x_0,y_0)\) and define \[ g(x) = G(x, c) = G(x, F(x_0, y_0)), \] from which it follows that \[ F(x, g(x)) = F(x, G(x, c)) = c = F(x_0, y_0). \]
Finally, if \((x, y)\) are in the domain where \(\bar{F}\) is invertible, and such that \(F(x, y) = F(x_0, y_0)\), then since \(F(x, g(x)) = F(x_0, y_0) = F(x, y)\) we obtain \[ \bar{F}(x, y) = (x, F(x, y)) = (x, F(x, g(x))) = \bar{F}(x, g(x)). \] Since \(\bar{F}\) is invertible, it is in particular injective hence \[ (x, y) = (x, g(x)). \] That is \(y = g(x)\).
Assuming the Implicit Function Theorem is true, prove the Inverse Function Theorem. That is, try to reverse the proof!
One can also reorder the variables. That is, in the statement of the Implicit Function Theorem, we required that \(d_y F\) was invertible. That is, the last \(k\)-columns of \(dF\) are linearly independent. The theorem is still true if some other collection of \(k\) columns are linearly independent.
Let \(F : \mathbb{R}^{n+k} \to \mathbb{R}^k\) be a smooth function such that \(dF\) has rank \(k\) at \(p_0 \in \mathbb{R}^{n+k}\). Equivalently, \(dF\) is surjective at \(p_0\).
By permuting the indices, use the Implicit Function Theorem to show that for \(p\) in a neighbourhood of \(p_0\), we may parametrise the level set \(F(p) = F(p_0)\) as the graph of a smooth function \(g : \mathbb{R}^n \to \mathbb{R}^k\).
Hint : First note that \(dF_{p_0}\) has rank \(k\) if and only if \(d_{p_0} F\) has \(k\) linearly independent columns. Let \(j_1, \dots, j_k\) denote those columns and let \(i_1, \dots, i_n\) denote the remaining columns. Define \[ \Phi : \mathbb{R}^{n+k} \to \mathbb{R}^{n+k} \] by
\begin{align*} \Phi(x_a) &= x_{i_a}, a = 1, \dots, n \\ \Phi(y_b) &= x_{j_b}, b = 1, \dots, k. \end{align*}Thus \(\Phi\) permutes the columns.
Verify that \(F \circ \Phi\) satisfies the hypotheses of the Implicit Function Theorem as stated above, hence we may apply it to \(\Phi \circ F\). By returning to the original ordering (i.e applying \(\Phi^{-1}\), conclude that there is a function \(g : U \subseteq \mathbb{R}^n \to \mathbb{R}^k\) such that near \(p_0\), the level set \(F(p) = F(p_0)\) is locally the graph \((x_{j_1}, \dots, x_{j_k}) = g(x_{i_1}, \dots, x_{i_n})\).
Let \(F(x, y) = x^2 + y^2\)
Here \(n=k=1\)
\begin{equation*} dF = \begin{pmatrix} 2x & 2y \end{pmatrix} \end{equation*}For \(y \neq 0\) (equivalently \(x \neq \pm 1\)), \[ F(x, \pm \sqrt{1-x^2}) = 1. \] For \(x \neq 0\) (equivalently \(y \neq \pm 1\)), \[ F(\pm \sqrt{1-y^2}, y) = 1. \]
Thus for \((x, y) \in \mathbb{S}^1 = \{(x, y) : F(x, y) = 1\}\), if \(y > 0\), then \(-1 < x < 1\) and \((x, y) = (x, \sqrt{1-x^2})\). If \(y < 0\), then \(-1 < x < 1\) and \((x, y) = (x, -\sqrt{1-x^2})\). Similarly, if \(x > 0\) or \(x < 0\).
Generalise the previous example to show that \(\mathbb{S}^n\) is locally a graph.
Show that if \(M \subseteq \mathbb{R}^n\) is locally a graph, then \(M\) is an embedded sub-manifold.
Here by locally a graph, we mean \(M = \bigcup_{\alpha} \operatorname{graph} g_{\alpha}\) for some collection of smooth functions \(g_{\alpha} : U_{\alpha} \subseteq \mathbb{R}^m \to \mathbb{R}^{n-m}\) with \(U_{\alpha}\) open and some fixed \(m\) independent of \(\alpha\).
Let \(F : U \subseteq_{\text{open}} \mathbb{R}^{n+k} \to \mathbb{R}^k\) be a smooth function. Suppose that for some \(c \in \mathbb{R}^k\), \(dF_p\) has rank \(k\) for every \(p\) in the level set \(F^{\ast} \{c\} : = \{q \in U : F(q) = c\}\). Show that \(F^{\ast} \{c\}\) is an embedded sub-manifold.
The Orthogonal Group
Define the Orthogonal Group, \[ O (n) = \{T : \mathbb{R}^n \to \mathbb{R}^n \, | \, \forall X \in \mathbb{R}^n \, \langle T(X), T(X) \rangle = \langle X, X\rangle\}. \]
Thus \(O(n)\) is the set of linear transformations of \(\mathbb{R}^n\) that preserve the inner-product.
Let \(M_n\) denote the set of \(n \times n\) matrices. Let \(A \in M_n\) be the matrix representing \(T\) with respect to the standard basis. Then \(T\) is orthogonal if and only if \(A^T A = \operatorname{Id} = AA^T\). Thus we make the identification, \[ O(n) = \{A \in M_n : A^T A = \operatorname{Id} = AA^T\}. \] We think of \(M_n\) as \(\mathbb{R}^{n^2}\) with components \(A^i_j\), \(1 \leq i,j \leq n\). That is \(M_n\) is a vector space under component-wise addition and scalar multiplication.
Define \(F : M_n \to M_n\) by \[ F(A) = AA^T. \] In fact, \(F\) maps into the set of symmetric matrices, \[ \operatorname{Sym}_n = \{A \in M_n : A = A^T\}. \] To see this, from \(F(A) = AA^T\) we obtain
\begin{align*} F(A)^T &= (AA^T)^T = (A^T)^T A^T \\ &= AA^T = F(A). \end{align*}Note that \(\operatorname{Sym}_n\) is a vector subspace of \(M_n\): if \(A, B \in \operatorname{Sym}_n\), and \(c, d \in \mathbb{R}\), then \[ (cA + dB)^T = cA^T + dB^T = cA + d B, \] hence \(cA + dB \in \operatorname{Sym}_n\). Let \(N\) denote the dimension of \(\operatorname{Sym}_n\) so that \(\operatorname{Sym}_n\) is isomorphic to \(\mathbb{R}^N\). In fact, \(N = \tfrac{n(n-1)}{2}\) but we don't need that explicitly here.
Thus we have in fact that \(F : M_n \to \operatorname{Sym}_n\) which we can write as \[ F : \mathbb{R}^{n^2} \to \mathbb{R}^N. \]
Now, \(F\) is a polynomial in the entries of \(A\) hence is \(C^{\infty}\), and \[ O(n) = F^{\ast} \{\operatorname{Id}\}. \] We aim to show that \(dF_A\) has rank \(N\) for \(A \in O(n)\) hence that \(O(n)\) is an embedded sub-manifold of dimension \(N = \tfrac{n(n-1)}{2}\). We will do this by showing that \(dF_A\) is surjective.
For \(A \in O(n)\) and \(B \in M_n\), we have
\begin{align*} dF_A (B) &= \partial_t|_{t=0} F(A + tB) \\ &= \partial_t|_{t=0} (A + tB)(A + tB)^T \\ &= \partial_t|_{t=0} \left(AA^T + tAB^T + tBA^T + t^2 BB^T\right) \\ &= AB^T + BA^T \\ &= (AB^T) + (AB^T)^T. \end{align*}To complete the claim, we need to show that given \(C \in \mathbb{R}^N \simeq \operatorname{Sym}_n\) there exists a \(B \in M_n\) such that \[ dF_A(B) = C. \] That is we need to solve \[ AB^T + (AB^T)^T = C \] for \(B\). For this, let \[ B = \frac{1}{2} CA. \] Finally, then
\begin{align*} dF_A (B) &= AB^T + (AB^T)^T \\ &= A\frac{1}{2}(CA)^T + \left(A\frac{1}{2}(CA)^T\right)^T \\ &= \frac{1}{2} AA^T C^T + \frac{1}{2} \left(AA^T C^T\right)^T \\ &= \frac{1}{2} C^T + \frac{1}{2} C \\ &= C. \end{align*}The second last equality uses \(AA^T = \operatorname{Id} = A^T A\) and the last equality follows since \(C \in \operatorname{Sym}_n\) hence \(C = C^T\).
Show that \[ \operatorname{dim} \operatorname{Sym}_n = \frac{n(n-1)}{2}. \]
Hint : A symmetric matrix satisfies \(A^T = A\); that is, \(A^i_j = A^j_i\). Hence \(A\) is determined uniquely by the diagonal and the strict upper triangle above the diagonal. Count the number of such entries!
Let us discuss why the Inverse Function Theorem is true. We won't go into the details here, but just offer an overview of how it is proven. This requires knowledge of the Banach fixed point theorem. If you are not familiar with this theorem, then you should just skip this section.
One way to prove the IFT is to take a sequence of approximations:
\begin{align*} x_1 &= x_0 + df_{x_0}^{-1} (f(x) - f(x_0)) \\ x_2 &= x_1 + df_{x_0}^{-1} (f(x_1) - f(x_0)) \\ & \vdots \\ x_n &= x_{n-1} + df_{x_0}^{-1} (f(x_{n-1}) - f(x_0)) \\ & \vdots \end{align*}Then \(f^{-1}(y) = \lim_{n\to\infty} x_n\).
Define the Newton map (so called because it's based on Newton's method for approximating roots of equations), \[ T_y(x) = x + df_{x_0}^{-1} (f(x) - f(x_0)). \] Then our sequence is \(x_n = T_y(x_{n-1})\). That is, we keep iterating \(T_y\) and in the limit we obtain the solution. Of course, proving this is the solution requires a proof and that takes a good deal of work!
A standard way to approach the proof is to construct a suitable contraction map (i.e. a map that strictly shrinks distances). Our goal in constructing the map, is that given \(y\) we need a unique solution of \(f(x) = y\). Define \[ T_y (x) = x - df_{x_0}^{-1} (f(x) - y). \] Then we show that for suitable \(r > 0\), \(T_y\) is a contraction map \(\overline{\mathbb{B}}_r(x_0) \to \overline{\mathbb{B}}_r(x_0)\) and a cornerstone result in analysis (namely the Banach Fixed Point Theorem) implies that \(T_y\) posses a unique fixed point \(x^{\ast}_y \in \overline{\mathbb{B}}_r(x_0)\). That is, there is a unique point \(x^{\ast}_y \in \overline{\mathbb{B}}_r(x_0)\) such that \(T_y(x^{\ast}_y) = x^{\ast}_y\). Observe then that
\begin{equation*} \begin{split} T_y(x^{\ast}_y) = x^{\ast}_y &\Leftrightarrow df_{x_0}^{-1} (f(x^{\ast}_y) - y) = 0 \\ &\Leftrightarrow f(x^{\ast}_y) = y. \end{split} \end{equation*}The last equivalence follows from the assumption that \(df_{x_0}\) is invertible. Thus \(f(x^{\ast}_y) = y\) if and only if \(T_y\) has a fixed point \(x^{\ast}_y\). By showing this fixed point is unique we then may unambiguously define \[ f^{-1}(y) = x^{\ast}_y. \]