Objectives: Topics
-
Orthogonal projections and their properties
-
Best approximations
Print preview
Section 6.3 Orthogonal Projections
Handout 6.3 Orthogonal Projections
Objectives: Goals
-
Apply orthogonality and projections to:
-
compute projections and distances,
-
express vectors using orthogonal decompositions,
-
construct best approximations.
-
Motivating Question
For the matrix \(A\) and vector \(\vec b\text{,}\) which vector \(\widehat b \in \Col(A)\) is closest to \(\vec b\text{?}\)
\begin{equation*}
A=\begin{bmatrix}
1 \amp 2\\
3 \amp 0\\
-4 \amp -2
\end{bmatrix},
\qquad
\vec b=\begin{bmatrix}1\\1\\1\end{bmatrix}.
\end{equation*}
Theorem 6.36. Orthogonal Decomposition.
If \(W\) is a subspace of \(\mathbb{R}^n\text{,}\) then each \(\vec y\in\mathbb{R}^n\) has a unique decomposition:
\begin{equation*}
\vec y = \widehat y + w^\perp
\end{equation*}
where \(\widehat y \in W\) and \(w^\perp \in W^\perp\text{.}\) In fact, if \(\{\vec u_1,\dots,\vec u_p\}\) is any orthogonal basis for \(W\text{,}\) then
\begin{align*}
\widehat y \amp = \proj_W \vec y \\
\amp = \proj_{\vec u_1}\vec y + \cdots + \proj_{\vec u_p}\vec y\\
\amp = \frac{\vec y\cdot\vec u_1}{\vec u_1\cdot\vec u_1}\vec u_1 + \cdots + \frac{\vec y\cdot\vec u_p}{\vec u_p\cdot\vec u_p}\vec u_p
\end{align*}
The vector \(\widehat y\) is called the orthogonal projection of \(\vec y\) onto \(W\) and can be written \(\widehat y = \proj_{W}\vec y\text{.}\)
We have that \(w^\perp = \vec y - \widehat y\text{.}\)
Example 6.37.
Let \(\{\vec u_1,\dots,\vec u_5\}\) be an orthonormal basis of \(\mathbb{R}^5\) and let \(W=\operatorname{Span}\{\vec u_1,\vec u_2\}\text{.}\)
For any \(\vec y\in\mathbb{R}^5\text{,}\) write the orthogonal decomposition: \(\vec y=\widehat y + w^\perp\text{,}\) with \(\widehat y\in W\) and \(w^\perp\in W^\perp\text{.}\)
Example 6.38.
Given
\begin{equation*}
\vec y=\begin{bmatrix}-1\\2\\6\end{bmatrix},\quad
\vec u_1=\begin{bmatrix}3\\-1\\2\end{bmatrix},\quad
\vec u_2=\begin{bmatrix}1\\-1\\-2\end{bmatrix},
\end{equation*}
construct the decomposition \(\vec y = \widehat y + w^\perp\text{,}\) where \(\widehat y\) is the orthogonal projection of \(\vec y\) onto \(W=\operatorname{Span}\{\vec u_1,\vec u_2\}\text{.}\)
Theorem 6.39. Best Approximation Theorem.
Let \(W\subseteq\mathbb{R}^n\) and \(\vec y\in\mathbb{R}^n\text{.}\) If \(\widehat y\) is the orthogonal projection of \(\vec y\) onto \(W\text{;}\) that is \(\widehat y = \proj_W\vec y\text{,}\) then for every \(\vec w\in W\) with \(\vec w\ne \widehat y\text{,}\)
\begin{equation*}
\lVert\vec y-\widehat y\rVert < \lVert\vec y-\vec w\rVert.
\end{equation*}
Definition 6.40.
Let \(W \subset \mathbb{R}^n\) and \(\vec y \in \mathbb{R}^n\text{.}\) The distance of a vector \(\vec y\) from a subspace \(W\), denoted \(\dist(\vec y, W)\text{,}\) is the minimum distance between \(\vec y\) and a vector \(\vec w \in W\text{;}\) that is,
\begin{equation*}
\dist(\vec y, W) = \operatorname{min}\left\{\dist(\vec y, \vec w) : w \in W\right\}\text{.}
\end{equation*}
By Theorem 6.39, we see that the vector in \(W\) which is closest to \(\vec y\) is \(\widehat y = \proj_W\vec y\text{.}\) Therefore,
\begin{equation*}
\dist(\vec y, W) = \dist(\vec y, \widehat y)\text{.}
\end{equation*}
Example 6.41.
Using the same vectors as Example 6.38:
\begin{equation*}
\vec y=\begin{bmatrix}-1\\2\\6\end{bmatrix},\quad
\vec u_1=\begin{bmatrix}3\\-1\\2\end{bmatrix},\quad
\vec u_2=\begin{bmatrix}1\\-1\\-2\end{bmatrix},
\end{equation*}
Example 6.42.
Find the distance from \(\vec y\) to the subspace \(W = \Span\{\vec v_1, \vec v_2, \vec v_3\}\text{,}\) where
\begin{equation*}
\vec y = \begin{bmatrix} 11 \\ 1 \\ -2 \\ 6 \end{bmatrix}, \qquad \vec v_1 = \begin{bmatrix} 1 \\ 1 \\ 0 \\ -1 \end{bmatrix}, \qquad \vec v_2 = \begin{bmatrix} 1 \\ 0 \\ 1 \\ 1 \end{bmatrix}, \qquad \vec v_3 = \begin{bmatrix} 0 \\ -1 \\ 1 \\ -1\end{bmatrix}\text{.}
\end{equation*}
Example 6.43.
Indicate whether each statement is true or false. If true, explain why. If false, give a counterexample.
(a)
If \(\vec x\) is orthogonal to \(\vec v\) and \(\vec w\text{,}\) then \(\vec x\) is orthogonal to \(\vec v - \vec w\text{.}\)
(b)
(c)
If \(\vec y = \vec u_1 + \vec v_1\text{,}\) where \(\vec u_1 \in W\) and \(\vec v_1 \in W^\perp\text{,}\) then \(\vec u_1\) is the orthogonal projection of \(\vec y\) onto \(W\text{.}\)
