Objectives: Topics
-
Symmetric matrices
-
Orthogonal diagonalization
-
Spectral decomposition
Print preview
Section 7.1 Diagonalization of Symmetric Matrices
Handout 7.1 Diagonalization of Symmetric Matrices
Objectives: Goals
-
Construct an orthogonal diagonalization \(A=PDP^T\) of a symmetric matrix.
-
Construct a spectral decomposition of a matrix.
Definition 7.12. Symmetric Matrix.
Example 7.13.
\begin{equation*}
A=[a],\quad \quad
B=\begin{bmatrix} a \amp b\\ b \amp c\end{bmatrix},\quad \quad
C=\begin{bmatrix} a \amp b\\ 0 \amp 0\end{bmatrix},
\end{equation*}
\begin{equation*}
D=\begin{bmatrix}4 \amp 2\\ 0 \amp 0\\ 0 \amp 0\end{bmatrix}, \quad \quad
E=\begin{bmatrix}
4 \amp 2 \amp 0 \amp 1\\
2 \amp 1 \amp 7 \amp 4\\
0 \amp 7 \amp 6 \amp 0\\
1 \amp 4 \amp 0 \amp 3
\end{bmatrix}.
\end{equation*}
Note 7.14.
For any matrix \(A\) with columns \(\vec a_1,\dots,\vec a_n\text{,}\) the matrix \(A^TA\) is symmetric and has entries given by dot products of the columns of \(A\text{.}\)
\begin{equation*}
A^TA =
\begin{bmatrix}
\vec a_1^T\vec a_1 \amp \cdots \amp \vec a_1^T\vec a_n\\
\vdots \amp \ddots \amp \vdots\\
\vec a_n^T\vec a_1 \amp \cdots \amp \vec a_n^T\vec a_n
\end{bmatrix}.
\end{equation*}
Theorem 7.15. Orthogonality of Eigenspaces.
If \(A\) is symmetric and \(\vec v_1,\vec v_2\) are eigenvectors corresponding to distinct eigenvalues, then \(\vec v_1\) and \(\vec v_2\) are orthogonal.
More generally, eigenspaces associated with distinct eigenvalues are orthogonal subspaces.
Example 7.16.
\begin{equation*}
A=\begin{bmatrix}
0 \amp 0 \amp 1\\
0 \amp 1 \amp 0\\
1 \amp 0 \amp 0
\end{bmatrix},
\qquad \lambda=-1,\,1.
\end{equation*}
Note 7.17.
Theorem 7.18. Spectral Theorem.
-
All eigenvalues of \(A\) are real.
-
The dimension of each eigenspace equals its algebraic multiplicity.
-
Eigenspaces corresponding to distinct eigenvalues are orthogonal.
-
\(A\) can be orthogonally diagonalized as \(A=PDP^T\text{,}\) where \(P\) is orthogonal.
Theorem 7.19. Spectral Decomposition.
Suppose \(A \) can be orthogonally diagonalized as
\begin{equation*}
A=PDP^T = \begin{bmatrix} \vec u_1 \amp \cdots \amp \vec u_n\end{bmatrix} \begin{bmatrix} \lambda_1 \amp \cdots \amp 0 \\ \vdots \amp \ddots \amp \vdots \\ 0 \amp \cdots \amp \lambda_n \end{bmatrix}\begin{bmatrix}\vec u_1^T \\ \vdots \\ \vec u_n^T \end{bmatrix}\text{.}
\end{equation*}
Then \(A\) has the decomposition
\begin{equation*}
A=\sum_{i=1}^n \lambda_i \vec u_i \vec u_i^T = \lambda_1 \vec u_1\vec u_1^T + \cdots + \lambda_n \vec u_n\vec u_n^T\text{.}
\end{equation*}
Note that each term \(\lambda_i\vec u_i\vec u_i^T\) is a rank 1 matrix.
Example 7.20.
Construct a spectral decomposition of \(A\text{,}\) given its orthogonal diagonalization:
\begin{equation*}
A=\begin{bmatrix}3 \amp 1\\ 1 \amp 3\end{bmatrix}
=
\begin{bmatrix}
\tfrac1{\sqrt2} \amp -\tfrac1{\sqrt2}\\
\tfrac1{\sqrt2} \amp \tfrac1{\sqrt2}
\end{bmatrix}
\begin{bmatrix}4 \amp 0\\ 0 \amp 2\end{bmatrix}
\begin{bmatrix}
\tfrac1{\sqrt2} \amp \tfrac1{\sqrt2}\\
-\tfrac1{\sqrt2} \amp \tfrac1{\sqrt2}
\end{bmatrix}.
\end{equation*}
Note 7.21.
Note in the solution to Example 7.20 that if \(\lambda_1 = 4\) is the larger of the two eigenvalues and \(\vec u_1\) is a normalized eigenvector corresponding to \(\lambda_1\text{,}\) then \(A \approx \lambda_1\vec u_1\vec u_1^T\) is a close approximation of the matrix \(A\text{.}\) In fact, it is the closest approximation of \(A\) using a rank 1 matrix.
In general, if we label the eigenvalues of an \(n \times n\) symmetric matrix \(A\) in non-increasing order by magnitude, so \(|\lambda_1| \geq |\lambda_2| \geq \cdots \geq |\lambda_n|\text{,}\) then each matrix
\begin{equation*}
\sum_{i=1}^j \lambda_i\vec u_i\vec u_i^T = \lambda_1\vec u_1\vec u_1^T + \cdots + \lambda_j\vec u_j \vec u_j^T
\end{equation*}
is the closest\(^*\) rank \(j\) approximation of \(A\text{,}\) which gets more accurate as \(j\) approaches \(n\text{.}\)
\(^*\text{:}\) The distance between two matrices is measured by subtracting the two matrices and calculating the square root of the sums of the squares of the entries in the resulting matrix. This calculation is the same calculation used to calculate the distance between two vectors. By closest, we mean that there is no rank \(j\) matrix that is closer. There could be other rank \(j\) matrices that are as close.
Example 7.22.
Construct a spectral decompositon of
\begin{equation*}
A = \begin{bmatrix} 0 \amp 0 \amp 1 \\ 0 \amp 1 \amp 0 \\ 1 \amp 0 \amp 0 \end{bmatrix}\text{.}
\end{equation*}
