Objectives: Topics
-
Identity and zero matrices
-
Matrix algebra (sums and products, scalar multiplies, matrix powers)
-
Transpose of a matrix
Print preview
Section 2.1 Matrix Operations
Handout 2.1 Matrix Operations
Objectives: Goals
-
Apply matrix algebra, the matrix transpose, and the zero and identity matrices, to solve and analyze matrix equations.
Definition 2.1 Zero Matrix.
A zero matrix is any matrix whose every entry is zero.
\begin{equation*}
0_{2\times 3}=
\begin{bmatrix}
0 \amp 0 \amp 0\\
0 \amp 0 \amp 0
\end{bmatrix},
\qquad
0_{2\times 1}=
\begin{bmatrix}
0\\
0
\end{bmatrix}.
\end{equation*}
Definition 2.2 Identity Matrix.
The \(n\times n\) identity matrix has ones on the main diagonal and zeros elsewhere.
\begin{equation*}
I_2=
\begin{bmatrix}
1 \amp 0\\
0 \amp 1
\end{bmatrix},
\qquad
I_3=
\begin{bmatrix}
1 \amp 0 \amp 0\\
0 \amp 1 \amp 0\\
0 \amp 0 \amp 1
\end{bmatrix}.
\end{equation*}
Note: Any matrix with dimensions \(n\times n\) is square. Zero matrices need not be square; identity matrices must be square.
Suppose \(A\in\mathbb{R}^{m\times n}\text{,}\) and \(a_{i,j}\) is the element of \(A\) in row \(i\) and column \(j\text{.}\)
-
If \(A\) and \(B\) are \(m\times n\) matrices, then the elements of \(A+B\) are \(a_{i,j}+b_{i,j}\text{.}\)
-
If \(c\in\mathbb{R}\text{,}\) then the elements of \(cA\) are \(c\,a_{i,j}\text{.}\)
Example 2.3.
If
\begin{equation*}
\begin{bmatrix}
1 \amp 2 \amp 3\\
4 \amp 5 \amp 6
\end{bmatrix}
+ c
\begin{bmatrix}
7 \amp 4 \amp 7\\
0 \amp 0 \amp k
\end{bmatrix}
=
\begin{bmatrix}
15 \amp 10 \amp 17\\
4 \amp 5 \amp 16
\end{bmatrix},
\end{equation*}
what are the values of \(c\) and \(k\text{?}\)
Solution.
Note that we can use rules (a) and (b) above to combine the left expression and get the following equation:
\begin{equation*}
\begin{bmatrix} 1 + 7c \amp 2 + 4c \amp 3 + 7c \\ 4 \amp 5 \amp 6 + kc \end{bmatrix} = \begin{bmatrix} 15 \amp 10 \amp 17 \\ 4 \amp 5 \amp 16 \end{bmatrix}
\end{equation*}
Setting corresponding entries equal to one another, we see that \(c = 2\) and \(k = 5\text{.}\)
Scalar multiples and matrix addition have the expected properties. If \(r,s\in\mathbb{R}\) are scalars and \(A,B,C\) are \(m\times n\) matrices, then:
-
\(\displaystyle A + 0_{m\times n} = A\)
-
\(\displaystyle (A+B)+C = A+(B+C)\)
-
\(\displaystyle r(A+B)=rA+rB\)
-
\(\displaystyle (r+s)A = rA + sA\)
-
\(\displaystyle r(sA) = (rs)A\)
Definition 2.4 Matrix Product via Columns.
Let \(A\) be an \(m\times n\) matrix and \(B\) be an \(n\times p\) matrix. The product \(AB\) is an \(m\times p\) matrix, given by
\begin{equation*}
AB
=
A
\begin{bmatrix}
\vec b_1 \amp \cdots \amp \vec b_p
\end{bmatrix}
=
\begin{bmatrix}
A\vec b_1 \amp \cdots \amp A\vec b_p
\end{bmatrix}.
\end{equation*}
Note: The dimensions of \(A\) and \(B\) determine whether \(AB\) is defined, and what its dimensions will be.
A visual guide to when \(AB\) is defined based on inner dimensions.
Theorem 2.5 Row-Column Rule for Matrix Multiplication.
If \(A\in\mathbb{R}^{m\times n}\) has rows \(\vec a_i\) and \(B\in\mathbb{R}^{n\times p}\) has columns \(\vec b_j\text{,}\) then each element of \(C=AB\) is
\begin{equation*}
c_{ij}=\vec a_i\cdot \vec b_j.
\end{equation*}
Example 2.6.
Compute the following using the row-column method.
\begin{equation*}
C=AB=
\begin{bmatrix}
2 \amp 0\\
1 \amp -1
\end{bmatrix}
\begin{bmatrix}
3 \amp 0 \amp 1\\
4 \amp 5 \amp 6
\end{bmatrix}.
\end{equation*}
Solution.
Note that
\(\begin{eqnarray*} AB \amp = \amp \begin{bmatrix} 2 \amp 0 \\ 1 \amp -1 \end{bmatrix}\begin{bmatrix} 3 \amp 0 \amp 1 \\ 4 \amp 5 \amp 6 \end{bmatrix}\\ \amp = \amp \begin{bmatrix} 2 \cdot 3 + 0 \cdot 4 \amp 2 \cdot 0 + 0 \cdot 5 \amp 2 \cdot 1 + 0 \cdot 6 \\ 1 \cdot 3 + (-1) \cdot 4 \amp 1 \cdot 0 + (-1) \cdot 5 \amp 1 \cdot 1 + (-1) \cdot 6 \end{bmatrix}\\ \amp = \amp \begin{bmatrix} 6 \amp 0 \amp 2 \\ -1 \amp -5 \amp -5 \end{bmatrix}\end{eqnarray*}\)
Note that this method is faster than using the original definition of matrix multiplication given in Definition 2.4. For example, to compute the first column of \(AB \) using the original definition, we would compute the following:
\begin{equation*}
\begin{bmatrix}2 \amp 0 \\ 1 \amp -1 \end{bmatrix}\begin{bmatrix} 3 \\ 4 \end{bmatrix} = 3\begin{bmatrix} 2 \\ 1\end{bmatrix} + 4 \begin{bmatrix} 0 \\ -1 \end{bmatrix} = \begin{bmatrix} 6 \\ -1 \end{bmatrix}
\end{equation*}
Theorem 2.7 Properties and Warnings.
Let \(A,B,C\) be matrices of sizes needed for the products to be defined, and suppose \(A\) is \(m\times n\text{.}\)
-
Associative: \((AB)C = A(BC)\)
-
Left distributive: \(A(B+C)=AB+AC\)
-
Right distributive: \((A+B)C = AC + BC\)
-
Identity: \(I_m A = A I_n\)
Warnings:
-
Non-commutative: In general, \(AB\neq BA\text{.}\)
-
Non-cancellation: \(AB=AC\) does not imply \(B=C\text{.}\)
-
Zero divisors: \(AB=0\) does not imply \(A=0\) or \(B=0\text{.}\)
The associative property is \((AB)C = A(BC)\text{.}\) If \(C=\vec x\text{,}\) then:
\begin{equation*}
(AB)\vec x = A(B\vec x).
\end{equation*}
Schematically:
Diagram showing two ways of getting from \(\vec x\) to \(AB\vec x\text{,}\) either by \(\vec x \mapsto B\vec x \mapsto AB\vec x\) or by direct multiplication by \(\vec x \mapsto AB\vec x\text{.}\)
The product \(AB\vec x\) can be obtained either by multiplying by \(AB\text{,}\) or by multiplying by \(B\) and then by \(A\text{.}\) This corresponds to composition of linear transformations.
Example 2.8.
Let
\begin{equation*}
A=
\begin{bmatrix}
1 \amp 1\\
0 \amp 0
\end{bmatrix}.
\end{equation*}
Give an example of a \(2\times 2\) matrix \(B\) such that \(AB\neq BA\text{.}\)
Solution.
Honestly, pretty much any matrix you choose at random will work. For example, let \(B = \begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix}\) and note that
\begin{equation*}
AB = \begin{bmatrix} 1 \amp 1 \\ 0 \amp 0 \end{bmatrix}\begin{bmatrix}1 \amp 2 \\ 3 \amp 4 \end{bmatrix} = \begin{bmatrix} 1 \cdot 1 + 1 \cdot 3 \amp 1 \cdot 2 + 1 \cdot 4 \\ 0 \cdot 1 + 0 \cdot 3 \amp 0 \cdot 2 + 0 \cdot 4 \end{bmatrix} = \begin{bmatrix} 4 \amp 6 \\ 0 \amp 0 \end{bmatrix}\text{,}
\end{equation*}
while
\begin{equation*}
BA = \begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix}\begin{bmatrix} 1 \amp 1 \\ 0 \amp 0 \end{bmatrix} = \begin{bmatrix} 1 \cdot 1 + 2 \cdot 0 \amp 1 \cdot 1 + 2 \cdot 0 \\ 3 \cdot 1 + 4 \cdot 0 \amp 3 \cdot 1 + 4 \cdot 0 \end{bmatrix} = \begin{bmatrix} 1 \amp 1 \\ 3 \amp 3 \end{bmatrix}\text{.}
\end{equation*}
Therefore, \(AB \neq BA \text{.}\)
Definition 2.9 Transpose.
The transpose \(A^T\) is the matrix whose columns are the rows of \(A\text{.}\)
Example 2.10.
Compute:
\begin{equation*}
A = \begin{bmatrix}
1 \amp 2 \amp 3 \amp 4 \amp 5\\
0 \amp 1 \amp 0 \amp 2 \amp 0
\end{bmatrix}^T
\end{equation*}
Solution.
We have that \(A^T = \begin{bmatrix} 1 \amp 0 \\ 2 \amp 1 \\ 3 \amp 0 \\ 4 \amp 2 \\ 5 \amp 0 \end{bmatrix}\)
Theorem 2.11 Properties of Matrix Transpose.
-
\((A^T)^T =\) \(A\)
-
\((A+B)^T =\) \(A^T + B^T\)
-
\((rA)^T =\) \(rA^T\)
-
\((AB)^T =\) \(B^TA^T\)
Example 2.12.
Define
\begin{equation*}
A=
\begin{bmatrix}
1 \amp 0\\
0 \amp 0
\end{bmatrix},
\qquad
B=
\begin{bmatrix}
1 \amp 0 \amp 0\\
0 \amp 0 \amp 8
\end{bmatrix},
\qquad
C=
\begin{bmatrix}
1 \amp 0 \amp 0\\
0 \amp 2 \amp 0\\
0 \amp 0 \amp 2
\end{bmatrix}.
\end{equation*}
Which of these operations are defined, and what are the dimensions of the result?
(a)
\(A + 3C\)
Solution.
Note that \(A \) is a \(2 \times 2\) matrix while \(3C\) is a \(3 \times 3\) matrix. Therefore, the sum \(A + 3C\) is not defined.
(b)
\(C(AB)^T\)
Solution.
First, we can use Property (d) from Theorem 2.11 to write \(C(AB)^T = CB^TA^T\text{.}\) Since \(B \) is a \(2 \times 3\) matrix, we have that \(B^T \) is a \(3 \times 2\) matrix. Since \(A \) is a \(2 \times 2\) matrix, we have that \(A^T \) is also a \(2 \times 2\) matrix. Thus, the product \(CB^TA^T\) is the product of a \(3 \times 3\) matrix with a \(3 \times 2 \) matrix with a \(2 \times 2\) matrix. Since all inner dimensions match, this product is defined.
Let’s calculate the product. One thing to note, even though matrix multiplication is not commutative, it is associative (see Theorem 2.7), so we can either simplify the product by either computing \((CB^T)A\) (mutliplying \(C\) and \(B^T\) first) or by computing \(C(B^TA^T)\) (multiplying \(B^T\) and \(A^T\) first). I choose whichever looks easier. It follows that
\begin{equation*}
CB^TA^T = \begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 2 \end{bmatrix}\begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \\ 0 \amp 8 \end{bmatrix}\begin{bmatrix}1 \amp 0 \\ 0 \amp 0 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \\ 0 \amp 16 \end{bmatrix}\begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \\ 0 \amp 0 \end{bmatrix}
\end{equation*}
(c)
\(A + ABCB^T\)
Solution.
Note that \(A \) is a \(2 \times 2\) matrix, \(B \) is a \(2 \times 3\) matrix, \(C \) is a \(3 \times 3\) matrix, and \(B^T\) is a \(3 \times 2\) matrix. Therefore, all of the inner dimensions of the product \(ABCB^T\) match and results in a \(2 \times 2\) matrix. It follows that \(A + ABCB^T\) is defined.
We will simply this expression by making use of the associative property and simplifying \(ABCB^T\) by first multiplying \(AB\) and \(CB^T\text{,}\) then multiplying the resulting matrices. Note \(\begin{eqnarray*} A + ABCB^T \amp = \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \end{bmatrix} + \left(\begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \end{bmatrix}\begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 8 \end{bmatrix}\right) \left(\begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 2 \end{bmatrix}\begin{bmatrix}1 \amp 0 \\ 0 \amp 0 \\ 0 \amp 8 \end{bmatrix}\right) \\ \amp = \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \end{bmatrix} + \begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \end{bmatrix}\begin{bmatrix}1 \amp 0 \\ 0 \amp 0 \\ 0 \amp 16 \end{bmatrix} \\ \amp = \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \end{bmatrix} + \begin{bmatrix} 1 \amp 0 \\ 0 \amp 0 \end{bmatrix} \\ \amp = \amp \begin{bmatrix} 2 \amp 0 \\ 0 \amp 0 \end{bmatrix}\end{eqnarray*}\)
For any \(n\times n\) matrix and positive integer \(k\text{,}\) \(A^k\) is the product of \(k\) copies of \(A\text{.}\)
\begin{equation*}
A^k = AA\cdots A.
\end{equation*}
Example 2.13.
Compute \(A^2\) for
\begin{equation*}
A = \begin{bmatrix} 3 \amp -2 \\ 5 \amp 1 \end{bmatrix}
\end{equation*}
Solution.
\begin{equation*}
A^2 = \begin{bmatrix} 3 \amp -2 \\ 5 \amp 1 \end{bmatrix}\begin{bmatrix}3 \amp -2 \\ 5 \amp 1 \end{bmatrix} = \begin{bmatrix} -1 \amp -8 \\ 20 \amp -9 \end{bmatrix}
\end{equation*}
Example 2.14.
Compute \(B^8\) for
\begin{equation*}
B=
\begin{bmatrix}
1 \amp 0 \amp 0\\
0 \amp 2 \amp 0\\
0 \amp 0 \amp 2
\end{bmatrix}.
\end{equation*}
Solution.
As we saw in Example 2.13, you cannot distribute the exponent to the individual entries in the matrix. For diagonal matrices, you actually can. Note that
\begin{equation*}
B^2 = \begin{bmatrix}1 \amp 0 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 2 \end{bmatrix}\begin{bmatrix}1 \amp 0 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 2 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 4 \amp 0 \\ 0 \amp 0 \amp 4 \end{bmatrix} = \begin{bmatrix} 1^2 \amp 0 \amp 0 \\ 0 \amp 2^2 \amp 0 \\ 0 \amp 0 \amp 2^2 \end{bmatrix}\text{.}
\end{equation*}
Therefore, we have that
\begin{equation*}
B^8 = \begin{bmatrix} 1^8 \amp 0 \amp 0 \\ 0 \amp 2^8 \amp 0 \\ 0 \amp 0 \amp 2^8 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 256 \amp 0 \\ 0 \amp 0 \amp 256 \end{bmatrix}\text{.}
\end{equation*}
We will see an application of this to diagonalizable matrices in Section 5.3.
Example 2.15.
True or False. If false, explain briefly.
(a)
Let \(n \in \mathbb{R}\text{.}\) Then for any \(n \times n\) matrix \(\text{,}\) we have \((I_n + A)(I_n - A) = I_n - A^2\text{.}\)
Solution.
Recall from Theorem 2.7 that the Distributive Property applies to matrix mutliplication and addition. Therefore, we start with the distributive property to simply the product.
\begin{equation*}
(I_n + A)(I_n - A) = I_nI_n + I_n(-A) + AI_n + A(-A)
\end{equation*}
By the multiplicative property of the identity matrix, we have that \(I_nI_n = I_n\) and \(I_n(-A) = -A\) and \(AI_n = A\text{.}\) Recall from our properties of matrix multipliation that we are allowed to commute scalars with matrix multiplication, just not matrices with matrix multiplication. Therefore, we can factor out the \(-1\) and rewrite \(A(-A) = -AA = -A^2\text{.}\) It follows that
\begin{equation*}
(I_n + A)(I_n - A) = I_n - A + A - A^2 = I_n - A^2
\end{equation*}
since \(-A + A\) is the zero matrix, which doesn’t change anything when you add it to a term. Therefore, we see that this statement is TRUE.
(b)
Let \(n \in \mathbb{R}\text{.}\) For any \(n \times n\) matrices \(A\) and \(B\text{,}\) we have \((A+B)^2 = A^2 + B^2 + 2AB\text{.}\)
Solution.
Proceeding in a similar manner to the problem above, we see that
\begin{equation*}
(A+B)^2 = (A+B)(A+B) = AA + AB + BA + BB\text{.}
\end{equation*}
We can rewrite \(AA = A^2\) and \(BB = B^2\text{.}\) Also, since matrix addition is commutative, we can move the terms \(AB\) and \(BA\) to the end. This leaves us with
\begin{equation*}
(A+B)^2 = A^2 + B^2 + AB + BA\text{.}
\end{equation*}
Note that if \(AB = BA\text{,}\) then we could combine the middle terms into \(2AB\text{.}\) However, from the warning in Theorem 2.7, we see that matrix multiplication is often not commutative. Therefore, we cannot replace \(AB + BA\) with \(2AB\text{.}\) This leads us to believe the statement is false.
We can prove it is false by giving a counterexample. Probably, pretty much anything we try will work since matrix commutation is very rare. Let \(A = \begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix}\) and \(B = \begin{bmatrix} 4 \amp 3 \\ 2 \amp 1 \end{bmatrix}\text{.}\) Note that
\(\begin{eqnarray*}(A+B)^2 = \left(\begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix} + \begin{bmatrix} 4 \amp 3 \\ 2 \amp 1 \end{bmatrix}\right)^2 = \begin{bmatrix} 5 \amp 5 \\ 5 \amp 5 \end{bmatrix}^2 \amp = \amp \begin{bmatrix} 5 \amp 5 \\ 5 \amp 5 \end{bmatrix}\begin{bmatrix} 5 \amp 5 \\ 5 \amp 5 \end{bmatrix}\\ \amp = \amp \begin{bmatrix} 50 \amp 50 \\ 50 \amp 50 \end{bmatrix};\end{eqnarray*}\)
however,
\(\begin{eqnarray*}A^2 + B^2 + 2AB \amp = \amp \begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix}^2 + \begin{bmatrix} 4 \amp 3 \\ 2 \amp 1 \end{bmatrix}^2 + 2\begin{bmatrix}1 \amp 2 \\ 3\amp 4 \end{bmatrix}\begin{bmatrix} 4 \amp 3 \\ 2 \amp 1 \end{bmatrix} \\ \amp = \amp \begin{bmatrix} 1 \amp 2 \\ 3 \amp 4 \end{bmatrix}\begin{bmatrix}1 \amp 2 \\ 3 \amp 4\end{bmatrix} + \begin{bmatrix} 4 \amp 3 \\ 2 \amp 1 \end{bmatrix}\begin{bmatrix} 4 \amp 3 \\ 2 \amp 1 \end{bmatrix} + 2\begin{bmatrix} 8 \amp 5 \\ 20 \amp 13 \end{bmatrix} \\ \amp = \amp \begin{bmatrix} 7 \amp 10 \\ 15 \amp 22 \end{bmatrix} + \begin{bmatrix} 22 \amp 15 \\ 10 \amp 7 \end{bmatrix} + \begin{bmatrix} 16 \amp 10 \\ 40 \amp 26 \end{bmatrix} \\ \amp = \amp \begin{bmatrix} 45 \amp 35 \\ 65 \amp 55 \end{bmatrix}\end{eqnarray*}\)
Therefore, since \((A+B)^2 \neq A^2 + B^2 + 2AB\) for these particular matrices, the statement must be FALSE.
