Objectives: Topics
-
Inverse of a matrix, its algebraic properties, and its relation to solving systems of linear equations.
-
Elementary matrices and their role in calculating the matrix inverse.
Print preview
Section 2.2 Matrix Inverses
Handout 2.2 Matrix Inverses
Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.―Spielberg and Crichton, Jurassic Park, 1993 film
The algorithm we introduce in this section could be used to compute the inverse of an \(n\times n\) matrix. At the end of the lecture we’ll discuss some of the problems with our algorithm and why it can be difficult to compute a matrix inverse.
Objectives: Goals
-
Apply the formal definition of an inverse, and its algebraic properties, to solve and analyze linear systems.
-
Compute the inverse of an \(n\times n\) matrix, and use it to solve linear systems.
-
Construct elementary matrices.
Motivating Question: Is there a matrix \(A\) such that
\begin{equation*}
\begin{bmatrix}
2 \amp -1 \amp 0\\
-1 \amp 2 \amp -1\\
0 \amp -1 \amp 2
\end{bmatrix}A = I_3\; ?
\end{equation*}
Definition 2.16 Invertible (Non-singular) Matrix.
An \(n \times n\) matrix \(A\) is invertible (or non-singular) if there exists a matrix \(C\in\mathbb{R}^{n\times n}\) such that
\begin{equation*}
AC = CA = I_n.
\end{equation*}
If such a matrix exists, we write \(C=A^{-1}\text{.}\)
Theorem 2.17 Inverse Formula for a \(2\times 2\) Matrix.
The \(2\times 2\) matrix \(\begin{bmatrix} a \amp b\\ c \amp d \end{bmatrix}\) is invertible if and only if \(ad-bc\neq 0\text{.}\) In that case,
\begin{equation*}
\begin{bmatrix}
a \amp b\\
c \amp d
\end{bmatrix}^{-1}
=
\frac{1}{ad-bc}
\begin{bmatrix}
d \amp -b\\
-c \amp a
\end{bmatrix}.
\end{equation*}
Hint.
In the solution, I give a proof of this theorem. You may find it interesting to try it for yourself before looking at the full proof. The idea is to start with a matrix \(A = \begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix}\) for fixed \(a,b,c,d \in \mathbb{R}\) with \(ad - bc \neq 0\text{.}\) Now, let \(A^{-1} = \begin{bmatrix} x_1 \amp x_2 \\ x_3 \amp x_4 \end{bmatrix}\) for unknown \(x_1,x_2,x_3,x_4\in \mathbb{R}\text{.}\) Set \(AA^{-1} = I_2\) and determine what the unknowns \(x_1,x_2,x_3,x_4\) are in terms of the known \(a,b,c,d\text{.}\) You should end up solving a system of linear equations involving 4 equations in the 4 unknowns.
Solution.
Here is a proof of the above formula. (Before reading, you may want to try it yourself. Look at the hint above.)
Let \(A = \begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix}\) for fixed \(a,b,c,d \in \mathbb{R}\) with \(ad - bc \neq 0\text{.}\) We are looking for a \(2 \times 2\) matrix \(A^{-1}\) so that \(AA^{-1} = I_2\text{,}\) if such a matrix exists. Since \(A^{-1}\) is a \(2 \times 2\) matrix, it has four unknown entries. Let’s call them \(x_1, x_2, x_3, x_4 \in \mathbb{R}\text{,}\) so we have \(A^{-1} = \begin{bmatrix} x_1 \amp x_2 \\ x_3 \amp x_4 \end{bmatrix}\text{.}\) It follows that
\begin{equation*}
AA^{-1} = \begin{bmatrix}a \amp b \\ c \amp d \end{bmatrix}\begin{bmatrix} x_1 \amp x_2 \\ x_3 \amp x_4 \end{bmatrix} = \begin{bmatrix} ax_1 + bx_3 \amp ax_2 + bx_4 \\ cx_1 + dx_3 \amp cx_2 + dx_4\end{bmatrix}\text{.}
\end{equation*}
Since we want \(AA^{-1} = I_2\text{,}\) we can set the corresponding entries equal to arrive at the following system of linear equation:
\begin{equation*}
\left\{\begin{array}{ccccccccc} ax_1 \amp \amp \amp + \amp bx_3 \amp \amp \amp = \amp 1 \\ \amp \amp ax_2 \amp \amp \amp + \amp bx_4 \amp = \amp 0 \\ cx_1 \amp \amp \amp + \amp dx_3 \amp \amp \amp = \amp 0 \\ \amp \amp cx_2 \amp \amp \amp + \amp dx_4 \amp = \amp 1 \end{array}\right .
\end{equation*}
Note that we cannot have both \(a\) and \(c\) be zero, or else \(ad - bc = 0\text{.}\) Therefore, we can assume that at least one of them is nonzero, so let’s suppose \(a \neq 0\text{.}\) Converting this into an augmented matrix and performing some row operations, we obtain
\begin{equation*}
\begin{bmatrix}a \amp 0 \amp b \amp 0 \amp \rvert \amp 1 \\ 0 \amp a \amp 0 \amp b \amp \rvert \amp 0 \\ c \amp 0 \amp d \amp 0 \amp \rvert \amp 0 \\ 0 \amp c \amp 0 \amp d \amp \rvert \amp 1 \end{bmatrix} \underset{R_4 - \frac{c}{a}R_2 \rightarrow R_4}{\overset{R_3 - \frac{c}{a}R_1 \rightarrow R_1}{\Huge \sim}} \begin{bmatrix} a \amp 0 \amp b \amp 0 \amp \rvert \amp 1 \\ 0 \amp a \amp 0 \amp b \amp \rvert \amp 0 \\ 0 \amp 0 \amp d - \frac{bc}{a} \amp 0 \amp \rvert \amp -\frac{c}{a} \\ 0 \amp 0 \amp 0 \amp d - \frac{bc}{a} \amp \rvert \amp 1 \end{bmatrix}
\end{equation*}
Solving for \(x_4\text{,}\) we see that
\begin{equation*}
x_4 = \frac{1}{d-\frac{bc}{a}} = \frac{1}{\frac{ad - bc}{a}} = \frac{a}{ad - bc}\text{.}
\end{equation*}
Solving for \(x_3\text{,}\) we see that
\begin{equation*}
x_3 = \frac{-\frac{c}{a}}{d - \frac{bc}{a}} = \frac{-\frac{c}{a}}{\frac{ad-bc}{a}} = -\frac{c}{a}\left(\frac{a}{ad-bc}\right) = \frac{-c}{ad-bc}
\end{equation*}
Solving for \(x_2\text{,}\) we see that
\begin{equation*}
x_2 = \frac{-bx_4}{a} = \frac{-b\left(\frac{a}{ad-bc}\right)}{a} = \frac{-b}{ad-bc}
\end{equation*}
Lastly, solving for \(x_1\text{,}\) we see that
\begin{equation*}
x_1 = \frac{1 - bx_3}{a} = \frac{1 - b\left(\frac{-c}{ad-bc}\right)}{a} = \frac{\frac{ad-bc}{ad-bc} + \frac{bc}{ad-bc}}{a} = \frac{\frac{ad}{ad-bc}}{a} = \frac{d}{ad-bc}\text{.}
\end{equation*}
It follows that
\begin{equation*}
A^{-1} = \begin{bmatrix} x_1 \amp x_2 \\ x_3 \amp x_4 \end{bmatrix} = \begin{bmatrix} \frac{d}{ad-bc} \amp \frac{-b}{ad-bc} \\ \frac{-c}{ad-bc} \amp \frac{a}{ad-bc} \end{bmatrix} = \frac{1}{ad-bc}\begin{bmatrix} d \amp -b \\ -c \amp a \end{bmatrix}\text{.}
\end{equation*}
Example 2.18.
Find the inverse of the matrix below. Check that it truly is the inverse.
\begin{equation*}
A = \begin{bmatrix}
2 \amp 5\\
-3 \amp -7
\end{bmatrix}.
\end{equation*}
Solution.
First, note that \(ad - bc = 2(-7) - (5)(-3) = -14 + 15 = 1 \neq 0\text{,}\) so \(A\) is invertible. Using the formula, we have that
\begin{equation*}
A^{-1} = \frac{1}{ad - bc}\begin{bmatrix}d \amp -b \\ -c \amp a \end{bmatrix} = \begin{bmatrix} -7 \amp -5 \\ 3 \amp 2 \end{bmatrix}\text{.}
\end{equation*}
We can check this by verifying that \(AA^{-1} = I_2\) or by verifying \(A^{-1}A = I_2\text{.}\) Note that
\begin{equation*}
AA^{-1} = \begin{bmatrix} 2 \amp 5 \\ -3 \amp -7 \end{bmatrix}\begin{bmatrix} -7 \amp -5 \\ 3 \amp 2 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix} = I_2
\end{equation*}
Theorem 2.19 Inverse and Unique Solutions.
A matrix \(A\in\mathbb{R}^{n\times n}\) has an inverse if and only if for every \(\vec b\in\mathbb{R}^n\text{,}\) the system \(A\vec x=\vec b\) has a unique solution. In this case,
\begin{equation*}
\vec x = A^{-1}\vec b.
\end{equation*}
Solution.
Here is a proof of this theorem. If \(A\) is invertible, then we can multiply both sides of the matrix equation \(A\vec x = \vec b\) by \(A^{-1}\) on the left to obtain the following:
\(\begin{eqnarray*}A^{-1}(A\vec x) = A^{-1}(\vec b) \amp \implies \amp (A^{-1}A)\vec x = A^{-1}\vec b\\ \amp \implies \amp I_n\vec x = A^{-1}\vec b \\ \amp \implies \amp \vec x = A^{-1}\vec b\end{eqnarray*}\)
Example 2.20.
Solve the linear system.
\begin{equation*}
\begin{array}{ccc}
3x_1 + 4x_2 \amp= \amp 7 \\
5x_1 + 6x_2 \amp = \amp 7
\end{array}
\end{equation*}
Solution.
We can write the system as the following matrix equation
\begin{equation*}
\begin{bmatrix} 3 \amp 4 \\ 5 \amp 6 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 7 \\ 7 \end{bmatrix}\text{.}
\end{equation*}
Since \(3(6) - 4(5) = 18 - 20 = -2 \neq 0\text{,}\) the matrix is invertible. Using Theorem 2.17, we have
\begin{equation*}
\begin{bmatrix} 3 \amp 4 \\ 5 \amp 6 \end{bmatrix}^{-1} = \frac{1}{-2}\begin{bmatrix} 6 \amp -4 \\ -5 \amp 3 \end{bmatrix}
\end{equation*}
Therefore, we find that the solution to the system is
\begin{equation*}
\vec x = \frac{1}{-2}\begin{bmatrix}6 \amp -4 \\ -5 \amp 3 \end{bmatrix}\begin{bmatrix} 7 \\ 7 \end{bmatrix} = \frac{1}{-2}\begin{bmatrix} 42 - 35 \\ -28 + 21 \end{bmatrix} = \begin{bmatrix} -7 \\ 7 \end{bmatrix}\text{.}
\end{equation*}
Like always, it’s a good idea to check that \(\vec x\) is a solution to the system.
Let \(A\) and \(B\) be invertible \(n\times n\) matrices. Then
-
\(\displaystyle (A^{-1})^{-1} = A\)
-
\((AB)^{-1} = B^{-1}A^{-1}\) (non-commutative!)
-
\(\displaystyle (A^T)^{-1} = (A^{-1})^T\)
Example 2.21.
True or false: \((ABC)^{-1} = C^{-1}B^{-1}A^{-1}\text{.}\)
Solution.
Note that
\begin{equation*}
(ABC)^{-1} = (A (BC))^{-1} = (BC)^{-1}A^{-1} = (C^{-1}B^{-1})A^{-1} = C^{-1}B^{-1}A^{-1}\text{.}
\end{equation*}
Therefore, the statement is true.
If \(A\in\mathbb{R}^{n\times n}\) and \(n>2\text{,}\) we can compute \(A^{-1}\) with the following algorithm:
-
Row reduce the augmented matrix \(\begin{bmatrix} A \ \rvert \ I_n\end{bmatrix}\text{.}\)
-
If the reduction has the form \(\begin{bmatrix}I_n \ \rvert \ B\end{bmatrix}\text{,}\) then \(A\) is invertible and \(B=A^{-1}\text{.}\) Otherwise, \(A\) is not invertible.
Example 2.22.
Compute the inverse of
\begin{equation*}
A=
\begin{bmatrix}
0 \amp 1 \amp 2\\
1 \amp 0 \amp 3\\
0 \amp 0 \amp 1
\end{bmatrix}.
\end{equation*}
Solution.
Note the following:
\(\begin{eqnarray*}\begin{bmatrix} A \amp \rvert \amp I_3\end{bmatrix} \amp = \amp \begin{bmatrix} 0 \amp 1 \amp 2 \amp \rvert \amp 1 \amp 0 \amp 0 \\ 1 \amp 0 \amp 3 \amp \rvert \amp 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \amp \rvert \amp 0 \amp 0 \amp 1 \end{bmatrix}\\
\amp \sim \amp \begin{bmatrix} 1 \amp 0 \amp 3 \amp \rvert \amp 0 \amp 1 \amp 0 \\ 0 \amp 1 \amp 2 \amp \rvert \amp 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \amp \rvert \amp 0 \amp 0 \amp 1 \end{bmatrix} \\
\amp \sim \amp \begin{bmatrix} 1 \amp 0 \amp 0 \amp \rvert \amp 0 \amp 1 \amp -3 \\ 0 \amp 1 \amp 0 \amp \rvert \amp 1 \amp 0 \amp -2 \\ 0 \amp 0 \amp 1 \amp \rvert \amp 0 \amp 0 \amp 1 \end{bmatrix} \\
\amp = \amp \begin{bmatrix} I_3 \amp \rvert \amp A^{-1}\end{bmatrix}\end{eqnarray*}\)
Therefore, we have \(A^{-1} = \begin{bmatrix} 0 \amp 1 \amp -3 \\ 1 \amp 0 \amp -2 \\ 0 \amp 0 \amp 1 \end{bmatrix}\text{.}\) Let’s check:
\begin{equation*}
AA^{-1} = \begin{bmatrix}0 \amp 1 \amp 2 \\ 1 \amp 0 \amp 3 \\ 0 \amp 0 \amp 1 \end{bmatrix}\begin{bmatrix}0 \amp 1 \amp -3 \\ 1 \amp 0 \amp -2 \\ 0 \amp 0 \amp 1 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{bmatrix} = I_3\text{.}
\end{equation*}
Why does the previous algorithm work? We can think of our algorithm as simultaneously solving \(n\) linear systems:
\begin{equation*}
\begin{array}{ccc}
A\vec x_1 \amp = \amp \vec e_1\\
A\vec x_2 \amp = \amp \vec e_2\\
\amp\vdots\\
A\vec x_n \amp = \amp \vec e_n
\end{array}
\end{equation*}
Each column of \(A^{-1}\) is \(A^{-1}\vec e_i=\vec x_i\text{.}\)
We can also justify the algorithm by using elementary matrices.
Definition 2.23 Elementary Matrix.
An elementary matrix \(E\) is one that differs from \(I_n\) by exactly one elementary row operation.
Recall the elementary row operations:
-
Swap rows
-
Multiply a row by a non-zero scalar
-
Add a multiple of one row to another
Each operation can be represented by matrix multiplication using an elementary matrix.
Solution.
Let \(A = \begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix}\text{.}\) For (a), note that multiplication by \(E_1 = \begin{bmatrix} 0 \amp 1 \\ 1 \amp 0 \end{bmatrix}\) on the left swaps the rows of \(A\text{:}\)
\begin{equation*}
E_1A = \begin{bmatrix} 0 \amp 1 \\ 1 \amp 0 \end{bmatrix}\begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix} = \begin{bmatrix} 3 \amp 4 \\ 1 \amp 2 \end{bmatrix}\text{.}
\end{equation*}
For (b), consider the matrix \(E_2 = \begin{bmatrix} 1 \amp 0 \\ 0 \amp -2 \end{bmatrix}\) which was chosen by performing the row operation \(-2R_2 \rightarrow R_2\) to the \(2 \times 2\) identity matrix. Note that multiplication by \(E_2\) to the left of \(A\) will accomplish the same row operation to \(A\text{:}\)
\begin{equation*}
E_2A = \begin{bmatrix} 1 \amp 0 \\ 0 \amp -2 \end{bmatrix}\begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix} = \begin{bmatrix} 1 \amp 2 \\ -6 \amp -8 \end{bmatrix}\text{.}
\end{equation*}
For (c), consider the matrix \(E_3 = \begin{bmatrix} 1 \amp 0 \\ 3 \amp 1 \end{bmatrix}\) which was chosen by performing the row operation \(R_2 + 3R_1 \rightarrow R_2\) to the \(2 \times 2\) identity matrix. Note that multiplication by \(E_3\) to the left of \(A\) will accomplish the same row operation to \(A\text{:}\)
\begin{equation*}
E_3A = \begin{bmatrix} 1 \amp 0 \\ 3 \amp 1 \end{bmatrix}\begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix} = \begin{bmatrix} 1 \amp 2 \\ 6 \amp 10 \end{bmatrix}\text{.}
\end{equation*}
Example 2.24.
Suppose
\begin{equation*}
E
\begin{bmatrix}
1 \amp 1 \amp 1\\
-2 \amp 1 \amp 0\\
0 \amp 0 \amp 1
\end{bmatrix}
=
\begin{bmatrix}
1 \amp 1 \amp 1\\
0 \amp 3 \amp 2\\
0 \amp 0 \amp 1
\end{bmatrix}.
\end{equation*}
By inspection, what is \(E\text{?}\) How does it compare to \(I_3\text{?}\)
Solution.
Note that
\begin{equation*}
\begin{bmatrix} 1 \amp 1 \amp 1 \\ -2 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{bmatrix} \overset{R_2 + 2R_1 \rightarrow R_2}{\Huge \sim} \begin{bmatrix} 1 \amp 1 \amp 1 \\ 0 \amp 3 \amp 2 \\ 0 \amp 0 \amp 1 \end{bmatrix}\text{.}
\end{equation*}
Therefore, we have that
\begin{equation*}
\begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{bmatrix} \overset{R_2 + 2R_1 \rightarrow R_2}{\Huge \sim} \begin{bmatrix} 1 \amp 0 \amp 0 \\ 2 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{bmatrix} = E\text{.}
\end{equation*}
Returning to why the algorithm works, applying a sequence of row operations to \(A\) to obtain \(I_n\) can be written as:
\begin{equation*}
(E_k \cdots E_3E_2E_1)A = I_n.
\end{equation*}
Thus, \(E_k \cdots E_3E_2E_1\) is the inverse matrix we seek.
Theorem 2.25 Invertibility and Row Equivalence.
A matrix \(A\) is invertible if and only if it is row equivalent to the identity matrix. In this case, any sequence of elementary row operations that transforms \(A\) into \(I\text{,}\) when applied to \(I\text{,}\) generates \(A^{-1}\text{.}\)
Some final thoughts:
-
We could use \(A^{-1}\) to solve a linear system \(A\vec x=\vec b\text{.}\) We would compute \(A^{-1}\) and then solve by multiplying appropriately.
-
As many textbooks point out, \(A^{-1}\) is seldom used in practice: computing it can take a long time and may be prone to numerical error.
-
So why did we learn how to compute \(A^{-1}\text{?}\) Later in the course, we use elementary matrices and properties of \(A^{-1}\) to derive results.
-
A recurring theme: just because we can do something a certain way, doesn’t mean we should.
