Objectives: Topics
-
Matrix notation for systems of equations.
-
The matrix product \(A \vec{x}\text{.}\)
Print preview
Section 1.4 The Matrix Equation \(A \vec{x} = \vec{b}\)
Handout 1.4 The Matrix Equation \(A \vec{x} = \vec{b}\)
In this section we introduce another way of expressing a linear system that we will use throughout this course.
Objectives: Goals
-
Compute matrix–vector products.
-
Express linear systems as vector equations and matrix equations.
-
Characterize linear systems and sets of vectors using span, linear combinations, and pivots.
Below is some math notation that will be used frequently throughout the course.
| symbol | meaning |
| \(\in\) | belongs to |
| \(\mathbb{R}^{n}\) | the set of vectors with \(n\) real-valued elements |
| \(\mathbb{R}^{m \times n}\) | the set of real-valued matrices with \(m\) rows and \(n\) columns |
Example: the notation \(\vec{x} \in \mathbb{R}^{5}\) means that \(\vec{x}\) is a vector with five real-valued elements.
Definition 1.33 Matrix Vector Product as a Linear Combination.
Let \(A\) be an \(m \times n\) matrix with columns \(\vec{a}_1,\dotsc,\vec{a}_n\) and let \(\vec{x} \in \mathbb{R}^{n}\text{.}\) The matrix vector product \(A\vec{x}\) is the following linear combination of the columns of \(A\text{:}\)
\begin{equation*}
A \vec{x}
=
\begin{bmatrix}
\vert \ \amp \ \vert \ \amp \ \cdots \ \amp \ \vert \\
\vec{a}_1 \ \amp \ \vec{a}_2 \ \amp \ \cdots \ \amp \ \vec{a}_n \\
\vert \ \amp \ \vert \ \amp \ \cdots \ \amp \ \vert
\end{bmatrix}
\begin{bmatrix}
x_1 \\ x_2 \\ \vdots \\ x_n
\end{bmatrix}
=
x_1 \vec{a}_1 + x_2 \vec{a}_2 + \cdots + x_n \vec{a}_n.
\end{equation*}
In particular, \(A\vec{x}\) lies in the span of the columns of \(A\text{.}\)
Example 1.34.
Write the product below as a linear combination of column vectors.
\(\begin{bmatrix}
1 \ \amp \ 0 \ \amp \ -1 \\
0 \ \amp \ -3 \ \amp \ 3
\end{bmatrix}
\begin{bmatrix}
4 \\ 3 \\ 7
\end{bmatrix}
\;=\; \)
Solution.
\(4\begin{bmatrix}1 \\ 0 \end{bmatrix} + 3 \begin{bmatrix} 0 \\ -3 \end{bmatrix} + 7\begin{bmatrix} -1 \\ 3 \end{bmatrix} = \begin{bmatrix} 4 \\ 0 \end{bmatrix} + \begin{bmatrix} 0 \\ -9 \end{bmatrix} = \begin{bmatrix} -7 \\ 21 \end{bmatrix} = \begin{bmatrix} -3 \\ 12 \end{bmatrix}\)
Example 1.35.
Simplify the product below.
\(\begin{bmatrix}
1 \ \amp \ 0 \ \amp \ 2 \ \amp \ 0 \ \amp \ 3 \\
0 \ \amp \ 1 \ \amp \ 0 \ \amp \ 2 \ \amp \ 0
\end{bmatrix}
\begin{bmatrix}
x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5
\end{bmatrix}
\;=\;\)
Solution.
\(x_1\begin{bmatrix} 1 \\ 0 \end{bmatrix} + x_2 \begin{bmatrix} 0 \\ 1 \end{bmatrix} + x_3\begin{bmatrix} 2 \\ 0 \end{bmatrix} + x_4\begin{bmatrix} 0 \\ 2 \end{bmatrix} + x_5\begin{bmatrix} 3 \\ 0 \end{bmatrix} = \begin{bmatrix} x_1 + 2x_3 + 3x_5 \\ x_2 + 2x_4 \end{bmatrix}\)
Theorem 1.36 Equivalent Formulations of a Linear System.
Let \(A\) be an \(m \times n\) matrix with columns \(\vec{a}_1,\dotsc,\vec{a}_n\text{,}\) let \(\vec{x} \in \mathbb{R}^{n}\text{,}\) and let \(\vec{b} \in \mathbb{R}^{m}\text{.}\) The solutions to
\begin{equation*}
A \vec{x} = \vec{b}
\end{equation*}
are the same as the solutions to the vector equation
\begin{equation*}
x_1 \vec{a}_1 + \cdots + x_n \vec{a}_n \;=\; \vec{b},
\end{equation*}
which are the same as the solutions to the linear system with augmented matrix
\begin{equation*}
\begin{bmatrix}
\vec{a}_1 \ \amp \ \vec{a}_2 \ \amp \ \cdots \ \amp \ \vec{a}_n \ \amp \vert \amp \ \vec{b}
\end{bmatrix}.
\end{equation*}
Theorem 1.37 The Existence of Solutions for the Matrix Equation.
The equation \(A \vec{x} = \vec{b}\) has a solution if and only if \(\vec{b}\) is a linear combination of the columns of \(A\text{.}\)
Theorem 1.38 Consistency for all \(\vec{b}\) and pivots.
The equation \(A \vec{x} = \vec{b}\) is consistent for every \(\vec{b}\) if and only if every row of \(A\) has a pivot.
Example 1.39.
For what vectors \(\vec{b} = \begin{bmatrix} b_1 \\[2pt] b_2 \\[2pt] b_3 \end{bmatrix}\) does the equation have a solution?
\begin{equation*}
\begin{bmatrix}
1 \ \amp \ 3 \ \amp \ 4 \\
2 \ \amp \ 8 \ \amp \ 4 \\
0 \ \amp \ 1 \ \amp \ -2
\end{bmatrix}
\vec{x} \;=\; \vec{b}
\end{equation*}
Solution.
By Theorem 1.37, the system \(A\vec x = \vec b\) if and only if \(\vec b\) is a linear combination of the columns of \(A\text{.}\) Therefore, we have that
\begin{equation*}
\vec b \in \operatorname{Span}\left\{\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}, \begin{bmatrix} 3 \\ 8 \\ 1 \end{bmatrix}, \begin{bmatrix} 4 \\ 4 \\ -2 \end{bmatrix}\right\}
\end{equation*}
Unfortunately, this description of \(\vec b\) is not too informative as we do not know exactly how "big" that span is. (We will understand more about how "big" a span is when we learn about dimension in Chapter 2.)
Let’s augment \(A\) with \(\vec b\) and row reduce to determine if we encounter any problems. Note the following:
\(\begin{eqnarray*} \begin{bmatrix} 1 \amp 3 \amp 4 \amp \rvert \amp b_1 \\ 2 \amp 8 \amp 4 \amp \rvert \amp b_2 \\ 0 \amp 1 \amp -2 \amp \rvert \amp b_3 \end{bmatrix} \amp \sim \amp \begin{bmatrix} 1 \amp 3 \amp 4 \amp \rvert \amp b_1 \\ 0 \amp 2 \amp -4 \amp \rvert \amp b_2 - 2b_1 \\ 0 \amp 1 \amp -2 \amp \rvert \amp b_3 \end{bmatrix} \\
\amp \sim \amp \begin{bmatrix} 1 \amp 3 \amp 4 \amp \rvert \amp b_1 \\ 0 \amp 1 \amp -2 \amp \rvert \amp b_3 \\ 0 \amp 2 \amp -4 \amp \rvert \amp b_2 - 2b_1 \end{bmatrix} \\
\amp \sim \amp \begin{bmatrix} 1 \amp 3 \amp 4 \amp \rvert \amp b_1 \\ 0 \amp 1 \amp -2 \amp \rvert \amp b_3 \\ 0 \amp 0 \amp 0 \amp \rvert \amp b_2 - 2b_1 - 2b_3 \end{bmatrix}\end{eqnarray*}\)
By Theorem 1.20, the system is consistent if and only if \(b_2 - 2b_1 - 2b_3 = 0\text{.}\) This is the defining characteristic for which \(\vec b\) lead to \(A\vec x = \vec b\) having a solution.
We could stop now, as we have answered the question, but notice we can work with the equation \(b_2 - 2b_1 - 2b_3 = 0\) to get a more explicit description for \(\vec b\text{.}\) Solving for \(b_1\text{,}\) we see that \(b_1 = \frac{1}{2}b_2 - b_3\text{.}\) We can think of this single equation in three variables as a system of linear equations. With this view, we see that variable \(b_1\) is a pivot variable (which is why we solved for it), and variables \(b_2\) and \(b_3\) are free variables. By free variables, we literally mean that \(b_2\) and \(b_3\) can take on any values in \(\mathbb{R}\text{.}\) To emphasize this, I usually set my free variables equal to other variables, typically \(t, r, p, q, \dots\text{.}\) So, we have \(b_2 = r \in \mathbb{R}\) and \(b_3 = t \in \mathbb{R}\text{.}\) (Again, the renaming of the free variables is not necessary, but I like to do it. You are free to keep the free variables named \(b_2\) and \(b_3\text{.}\))
Now, we can solve for our pivot variable in terms of the free variables. So we get \(b_1 = \frac{1}{2}r - 2t\text{.}\) Putting this all together, we see that the vectors \(\vec b \in \mathbb{R}^3\) which make the system \(A\vec x = \vec b\) consistent are precisely the following:
\begin{equation*}
\vec b = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix} = \begin{bmatrix} \frac{1}{2}r - t \\ r \\ t \end{bmatrix} = \begin{bmatrix} \frac{1}{2}r \\ r \\ 0 \end{bmatrix} + \begin{bmatrix} -t \\ 0 \\ t \end{bmatrix} = r\begin{bmatrix} \frac{1}{2} \\ 1 \\ 0 \end{bmatrix} + t\begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}
\end{equation*}
This form of our answer is called the parametric vector form which will be covered in the next section.
Note that at the beginning, we saw that \(\vec b\) had to live in the span of the columns of \(A\text{.}\) Now, we have learned a more descriptive way of describing \(\vec b\) in terms of span; that is, we have that
\begin{equation*}
\vec b \in \operatorname{Span}\left\{\begin{bmatrix} \frac{1}{2} \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}\right\}
\end{equation*}
In fact, we have that
\begin{equation*}
\operatorname{Span}\left\{\begin{bmatrix} \frac{1}{2} \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}\right\} = \operatorname{Span}\left\{\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}, \begin{bmatrix} 3 \\ 8 \\ 1 \end{bmatrix}, \begin{bmatrix} 4 \\ 4 \\ -2 \end{bmatrix}\right\}
\end{equation*}
This means that the original span was actually only 2-dimensional and not 3-dimensional as it at first appeared.
Example 1.40.
For what values of \(h\) is the system \(A\vec{x} = \vec{b}\) consistent for every possible choice of vector \(\vec{b}\text{?}\)
\begin{equation*}
A \;=\; \begin{bmatrix}
1 \ \amp \ h \ \amp \ 2 \\
0 \ \amp \ 6 \ \amp \ 2 \\
-3 \ \amp \ 0 \ \amp \ -4
\end{bmatrix}
\end{equation*}
Solution.
By Theorem 1.38, we see that \(A\) needs to have a pivot in every row. Note that
\(\begin{equation} \begin{bmatrix} 1 \amp h \amp 2 \\ 0 \amp 6 \amp 2 \\ -3 \amp 0 \amp -4 \end{bmatrix} \sim \begin{bmatrix} 1 \amp h \amp 2 \\ 0 \amp 6 \amp 2 \\ 0 \amp 3h \amp 2 \end{bmatrix}
\sim \begin{bmatrix} 1 \amp h \amp 2 \\ 0 \amp 1 \amp \frac{1}{3} \\ 0 \amp 3h \amp 2 \end{bmatrix}
\sim \begin{bmatrix} 1 \amp h \amp 2 \\ 0 \amp 1 \amp \frac{1}{3} \\ 0 \amp 0 \amp 2-h \end{bmatrix} \end{equation}\)
It follows that \(A\) will have a pivot in every row if and only if \(2-h \neq 0\text{.}\) Therefore, the values of \(h\) which make the system \(A\vec x = \vec b\) consistent for every possible \(\vec b\) are \(h \neq 2\text{.}\)
We now have four equivalent ways of expressing linear systems.
A system of equations:
\begin{equation*}
\begin{aligned}
2x_1 + 3x_2 \ \amp=\ 7 \\
x_1 - x_2 \ \amp=\ 5
\end{aligned}
\end{equation*}
An augmented matrix:
\begin{equation*}
\left[
\begin{array}{rr|r}
2 \ \amp \ 3 \ \amp \ 7 \\
1 \ \amp \ -1 \ \amp \ 5
\end{array}
\right]
\end{equation*}
A vector equation:
\begin{equation*}
x_1 \begin{bmatrix} 2 \\[2pt] 1 \end{bmatrix}
\;+\;
x_2 \begin{bmatrix} 3 \\[2pt] -1 \end{bmatrix}
\;=\;
\begin{bmatrix} 7 \\[2pt] 5 \end{bmatrix}
\end{equation*}
A matrix equation:
\begin{equation*}
\begin{bmatrix}
2 \ \amp \ 3 \\
1 \ \amp \ -1
\end{bmatrix}
\begin{bmatrix}
x_1 \\ x_2
\end{bmatrix}
\;=\;
\begin{bmatrix}
7 \\ 5
\end{bmatrix}
\end{equation*}
Each representation gives us a different way to think about linear systems.
