Objectives: Topics
-
Dot product of vectors
-
Magnitude of vectors and distances in \(\mathbb{R}^n\)
-
Orthogonal vectors and complements
-
Angles between vectors
Print preview
Section 6.1 Dot Product, Length, and Orthogonality
Handout 6.1 Dot Product, Length, and Orthogonality
Objectives: Goals
-
Compute (a) dot products, (b) magnitudes, (c) distances in \(\mathbb{R}^n\text{,}\) and (d) angles between vectors.
-
Apply theorems involving orthogonal complements, row/column space, and their relationships to linear systems.
Motivating Question
For a matrix \(A\text{,}\) which vectors are orthogonal to all the rows of \(A\text{?}\) To the columns of \(A\text{?}\)
Definition 6.1 Dot Product.
The dot product of two vectors \(\vec u,\vec v\in\mathbb{R}^n\) is:
\begin{equation*}
\vec u\cdot\vec v = \vec u^T\vec v = \begin{bmatrix} u_1 \amp u_2 \amp \cdots \amp u_n \end{bmatrix}\begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} = u_1v_1+u_2v_2+\cdots+u_n v_n.
\end{equation*}
Example 6.2.
For what values of \(k\) is \(\vec u\cdot\vec v=0\text{?}\)
\begin{equation*}
\vec u=\begin{bmatrix}-1\\3\\k\\2\end{bmatrix},\qquad
\vec v=\begin{bmatrix}4\\2\\1\\-3\end{bmatrix}.
\end{equation*}
Solution.
Note that
\begin{equation*}
\vec u \cdot \vec v = -1(4) + 3(2) + k(1) + 2(-3) = -4 + 6 + k - 6 = k - 4\text{.}
\end{equation*}
It follows that \(\vec u \cdot \vec v = 0\) if and only if \(k = 4\text{.}\)
Theorem 6.3 Basic Identities of the Dot Product.
Let \(\vec u,\vec v,\vec w\in\mathbb{R}^n\) and \(c\in\mathbb{R}\text{.}\)
-
(Symmetry) \(\vec u\cdot\vec w = \vec w\cdot\vec u\text{.}\)
-
(Linearity) \((\vec v+\vec w)\cdot\vec u = \vec v\cdot\vec u + \vec w\cdot\vec u\text{.}\)
-
(Scalars) \((c\vec u)\cdot\vec w = c(\vec u\cdot\vec w) = \vec u \cdot (c\vec w)\text{.}\)
-
(Positivity) \(\vec u\cdot\vec u\ge 0\text{,}\) and it equals 0 if and only if \(\vec u=\vec 0\text{.}\)
Definition 6.4 Length (Magnitude).
The length, or magnitude, of \(\vec u\in\mathbb{R}^n\) is
\begin{equation*}
\|\vec u\|=\sqrt{\vec u\cdot\vec u} = \sqrt{u_1^2+u_2^2+\cdots+u_n^2}.
\end{equation*}
Example 6.5.
Compute \(\lVert\overrightarrow{OP}\rVert\) where \(P=(1,3,2)\text{.}\)
Solution.
Note that
\begin{equation*}
\overrightarrow{OP} = \begin{bmatrix} 1 \\ 3 \\ 2 \end{bmatrix}\text{.}
\end{equation*}
It follows that
\begin{equation*}
\|\overrightarrow{OP}\| = \sqrt{1^2 + 3^2 + 2^2} = \sqrt{14}\text{.}
\end{equation*}
Example 6.6.
Let \(\lVert\vec u\rVert=5\text{,}\) \(\lVert\vec v\rVert=\sqrt{3}\text{,}\) and \(\vec u\cdot\vec v=-1\text{.}\) Compute \(\lVert\vec u+\vec v\rVert\text{.}\)
Solution.
Using Definition 6.4, see that
\begin{equation*}
\|\vec u + \vec v \| = \sqrt{(\vec u + \vec v)(\vec u + \vec v)} = \sqrt{(\vec u \cdot \vec u) + (\vec u \cdot \vec v) + (\vec v \cdot \vec u) + (\vec v \cdot \vec v)}\text{.}
\end{equation*}
Note that squaring both sides of the equation \(\|\vec u \| = \sqrt{\vec u \cdot \vec u}\) yields \(\|\vec u\|^2 = \vec u \cdot \vec u\text{.}\) Also, note that the dot product is commutative. Thus, we have that
\begin{equation*}
\|\vec u + \vec v\| = \sqrt{\|\vec u\|^2 + 2\vec u \cdot \vec v + \|\vec v\|^2} = \sqrt{5^2 + 2(-1) + \sqrt{3}^2} = \sqrt{26}\text{.}
\end{equation*}
For any vector \(\vec v\) and scalar \(c\text{,}\)
\begin{equation*}
\| c\vec v\| = \lvert c\rvert\|\vec v\|.
\end{equation*}
Definition 6.7 Unit Vector.
A vector of length 1 is called a unit vector.
Examples of unit vectors:
\begin{equation*}
\vec e_1=\begin{bmatrix}1\\0\end{bmatrix},\quad
\vec y=\frac{1}{\sqrt{5}}\begin{bmatrix}1\\2\end{bmatrix},\quad
\vec v=\frac{1}{3}\begin{bmatrix}2\\1\\2\end{bmatrix}.
\end{equation*}
Definition 6.8 Distance.
For \(\vec u,\vec v\in\mathbb{R}^n\text{,}\) the distance between them is
\begin{equation*}
\operatorname{dist}(\vec u,\vec v)=\lVert\vec u-\vec v\rVert.
\end{equation*}
Example 6.9.
Compute the distance between \(\vec u=\begin{bmatrix}7\\1\end{bmatrix}\) and \(\vec v=\begin{bmatrix}3\\2\end{bmatrix}\text{.}\)
Solution.
Note that
\begin{equation*}
\operatorname{dist}(\vec u, \vec v) = \|\vec u - \vec v\| = \left\|\begin{bmatrix} 4 \\ -1 \end{bmatrix}\right\| = \sqrt{4^2 + (-1)^2} = \sqrt{17}.
\end{equation*}
Definition 6.10 Orthogonal Vectors.
Two vectors \(\vec u,\vec v\) are orthogonal if \(\vec u\cdot\vec v=0\text{.}\)
This is equivalent to
\begin{equation*}
\| \vec u+\vec v\|^2
= \|\vec u\|^2 + \|\vec v\|^2.
\end{equation*}
Solution.
Proof: The equivalent statement, known as the Pythagorean Theorem, is true since
\begin{equation*}
\|\vec u + \vec v\|^2 = (\vec u + \vec v)(\vec u + \vec v) = \|\vec u\|^2 + \|\vec v\|^2 + 2\vec u \cdot \vec v = \|\vec u\|^2 + \|\vec v\|^2
\end{equation*}
since \(\vec u \cdot \vec v = 0\) because the vectors are orthogonal.
The zero vector is orthogonal to every vector in \(\mathbb{R}^n\text{.}\)
Example 6.11.
Sketch the set of all vectors orthogonal to \(\vec v=\begin{bmatrix}3\\2\end{bmatrix}\text{.}\)
Solution.
Let \(\vec x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}\text{.}\) Note that
\begin{equation*}
\vec v \cdot \vec x = 0 \quad \implies \quad 3x_1 + 2x_2 = 0\text{.}
\end{equation*}
Therefore, geometrically, we can describe the set of all vectors which are orthogonal to \(\vec v\) as the line \(3x_1 + 2x_2 = 0\text{.}\) Algebraically, the set of vectors which are orthogonal to \(\vec v\) is given by
\begin{equation*}
\Nul(\begin{bmatrix} 3 \\ 2 \end{bmatrix}) = \Span\left\{\begin{bmatrix} 2 \\ -3 \end{bmatrix}\right\}\text{.}
\end{equation*}
It is no coincidence that this set is a subspace. (See Definition 6.12.)
Definition 6.12 Orthogonal Complement.
If \(W\) is a subspace of \(\mathbb{R}^n\text{,}\) a vector \(\vec z\) is orthogonal to \(W\) if \(\vec z\cdot\vec w=0\) for every \(\vec w\in W\text{.}\) The set of all such vectors is the orthogonal complement:
\begin{equation*}
W^\perp=\{\vec z\in\mathbb{R}^n:\vec z\cdot\vec w=0 \;\text{for all}\; \vec w\in W\}.
\end{equation*}
The set \(W^\perp\) is also a subspace of \(\mathbb{R}^n\text{.}\) We have that
\begin{equation*}
\dim(W) + \dim(W^\perp) = n\text{.}
\end{equation*}
Furthermore, for any subspace \(W\text{,}\) we have that \(\left(W^\perp\right)^\perp = W\text{.}\)
Example 6.13.
Suppose \(A=\begin{bmatrix}1 \amp 3\\ 2 \amp 6\end{bmatrix}\text{.}\) Find \(\Nul(A)^\perp\text{.}\)
Solution.
Note that
\begin{equation*}
A = \begin{bmatrix} 1 \amp 3 \\ 2 \amp 6 \end{bmatrix} \sim \begin{bmatrix} 1 \amp 3 \\ 0 \amp 0 \end{bmatrix}\text{.}
\end{equation*}
It follows that
\begin{equation*}
\Nul(A) = \Span\left\{\begin{bmatrix} -3 \\ 1 \end{bmatrix}\right\}\text{.}
\end{equation*}
Note that \(\vec x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \in \Nul(A)^\perp\) if and only if \(\vec x\) is orthogonal to every vector in \(\Nul(A)\text{.}\) Since every vector in \(\Nul(A)\) is a multiple of \(\vec v = \begin{bmatrix} -3 \\ 1 \end{bmatrix}\text{,}\) we have that \(\vec x\) is orthogonal to every vector of \(\Nul(A)\) as long as it is orthogonal to \(\vec v\text{.}\) Note that
\begin{equation*}
\vec x \cdot \vec v = 0 \quad \implies \quad -3x_1 + x_2 = 0 \text{.}
\end{equation*}
It follows that \(\vec x = t\begin{bmatrix} 1 \\ 3 \end{bmatrix}\) for \(t \in \mathbb{R}\text{.}\) Thus,
\begin{equation*}
\Nul(A)^\perp = \Span\left\{\begin{bmatrix} 1 \\ 3 \end{bmatrix}\right\}\text{.}
\end{equation*}
Example 6.14.
Let \(L\) be the line spanned by \(\vec v=\begin{bmatrix}1\\-1\\2\end{bmatrix}\text{.}\) Since \(L\) is 1-dimensional, its orthogonal complement \(L^\perp\) is a plane. Find an equation for the plane \(L^\perp\text{.}\)
Solution.
First, note that by Definition 6.12, we have that \(\dim(L^\perp) = 3 - \dim(L) = 2\text{.}\) Thus, the space \(L^\perp\) really is a plane in \(\mathbb{R}^3\text{.}\) Let \(\vec x = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \in \mathbb{R}^3\text{.}\) Note that
\begin{equation*}
\vec v \cdot \vec x = 0 \quad \implies \quad x_1 - x_2 + 2x_3 = 0\text{.}
\end{equation*}
Thus, we have that
\begin{equation*}
L^\perp = \Nul\left(\begin{bmatrix} 1 \amp -1 \amp 2 \end{bmatrix}\right) = \Span\left\{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ 0 \\ -1 \end{bmatrix}\right\}\text{.}
\end{equation*}
Definition 6.15 Row Space.
The row space \(\operatorname{Row}(A)\) is the subspace spanned by the rows of a matrix \(A\text{.}\)
Facts:
-
\(\dim(\operatorname{Row}(A))=\dim(\operatorname{Col}(A))\text{.}\)
-
A basis for \(\operatorname{Row}(A)\) is given by the pivot rows of \(A\text{.}\)
-
In general, \(\operatorname{Row}(A)\) and \(\operatorname{Col}(A)\) are unrelated spaces.
Example 6.16.
Describe \(\Nul(A) \subseteq \mathbb{R}^n\) in terms of an orthogonal subspace.
-
\(\vec x\in\Nul(A)\) if and only if \(\underline{\hspace{2in}}\text{.}\)
Answer.
\(A\vec x = \vec 0\) -
This means \(\vec x\) is \(\underline{\hspace{2in}}\) to each row of \(A\text{.}\)
Answer.
orthogonal -
Therefore \(\operatorname{Row}(A)\) is \(\underline{\hspace{3in}}\) of \(\Nul(A)\text{.}\)
Answer.
the orthogonal complement -
\(\dim(\operatorname{Row}(A))+\dim(\Nul(A))=\underline{\hspace{1in}}\text{.}\)
Answer.
\(n\)
Theorem 6.17 The Four Subspaces.
For any \(A\in\mathbb{R}^{m\times n}\text{:}\)
-
\(\Nul(A)\) is the orthogonal complement of \(\operatorname{Row}(A)\text{:}\)\begin{equation*} \Row(A)^\perp = \Nul(A) \quad \text{and} \quad \Nul(A)^\perp = \Row(A) \end{equation*}
-
\(\Nul(A^T)\) is the orthogonal complement of \(\operatorname{Col}(A)\text{:}\)\begin{equation*} \Col(A)^\perp = \Nul(A^T) \quad \text{and} \quad \Nul(A^T)^\perp = \Col(A) \end{equation*}
This geometric structure is often shown using overlapping regions representing the four fundamental subspaces.
Theorem 6.18 Dot Product and Angle.
The dot product satisfies
\begin{equation*}
\vec a\cdot\vec b=\lVert\vec a\rVert\,\lVert\vec b\rVert\cos\theta.
\end{equation*}
Thus, if \(\vec a\cdot\vec b=0\text{,}\) then either one vector is the zero vector or the vectors are orthogonal.
Solution.
Proof: We will prove this theorem for vectors in \(\mathbb{R}^2\text{.}\) A similar argument can be made for two vectors in \(\mathbb{R}^n\) by considering them in the plane (or line) which they span.
First, note that if either \(\vec a = \vec 0\) or \(\vec b = \vec 0\text{,}\) then the equation is true since both the left side and right side are 0. Now suppose \(\vec a \neq \vec 0\) and \(\vec b \neq \vec 0\text{.}\) Then, we know that \(\|\vec a\| \neq 0\) and \(\|\vec b\| \neq 0\text{.}\) Let’s consider the unit vectors \(\vec u_1 = \frac{\vec a}{\|\vec a\|}\) and \(\vec u_2 = \frac{\vec b}{\|\vec b\|}\text{.}\) Since both \(\vec u_1\) and \(\vec u_2\) are unit vectors in \(\mathbb{R}^2\text{,}\) they both lie on the unit circle. Note that rotating both vectors by the same angle will not change the angle between them. So, let’s rotate the vectors so that \(\vec u_1\) lies on the positive \(x\)-axis. (This is similar to choosing a new reference frame in physics). Then, \(\vec u_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}\) and \(\vec u_2 = \begin{bmatrix} \cos \theta \\ \sin \theta \end{bmatrix}\text{,}\) where \(\theta\) is the angle between \(\vec u_1\) and \(\vec u_2\text{.}\) In this reference frame, we have that
\begin{equation*}
\vec u_1 \cdot \vec u_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \cdot \begin{bmatrix} \cos \theta \\ \sin \theta \end{bmatrix} = \cos\theta\text{.}
\end{equation*}
By replacing \(\vec u_1\) and \(\vec u_2\) with their definitions in terms of \(\vec a\) and \(\vec b\text{,}\) we see that
\begin{equation*}
\dfrac{\vec a}{\|\vec a\|} \cdot \dfrac{\vec b}{\|\vec b\|} = \cos \theta \quad \implies \quad \vec a \cdot \vec b = \|\vec a\|\|\vec b\|\cos \theta\text{.}
\end{equation*}
Example 6.19.
Find the angle between \(\vec v = \begin{bmatrix} 3 \\ 4 \end{bmatrix}\) and \(\vec u = \begin{bmatrix} \dfrac{3 - 4\sqrt{3}}{10} \\ \dfrac{4 + 3\sqrt{3}}{10} \end{bmatrix}\text{.}\) Note that \(\vec u\) is a unit vector.
Solution.
Note that
\begin{align*}
\vec v \cdot \vec u \amp = 3\left(\dfrac{3 - 4\sqrt{3}}{10}\right) + 4\left(\dfrac{4 + 3\sqrt{3}}{10}\right)\\
\amp = \dfrac{9 - 12\sqrt{3} + 16 + 12\sqrt{3}}{10}\\
\amp = \dfrac{25}{10}\\
\amp = \dfrac{5}{2}\text{.}
\end{align*}
Also, note that \(\|\vec v\| = \sqrt{3^2 + 4^2} = 5\text{.}\) Since \(\vec u\) is a unit vector, we have that \(\|\vec u \| = 1\text{.}\) Therefore, we have
\begin{equation*}
\cos(\theta) = \dfrac{\vec v \cdot \vec u}{\|\vec v\|\|\vec u\|} = \dfrac{\frac{5}{2}}{5} = \dfrac{1}{2},
\end{equation*}
so
\begin{equation*}
\theta = \operatorname{cos}^{-1}\left(\frac{1}{2}\right) = \frac{\pi}{3} = 60^\circ\text{.}
\end{equation*}
Thus, there is a \(60^\circ\) angle between \(\vec v\) and \(\vec u\text{.}\)
