Objectives: Topics
-
Least Squares Lines
-
Linear and more complicated models
Print preview
Section 6.6 Applications to Linear Models
Handout 6.6 Applications to Linear Models
Objectives: Goals
-
Apply least-squares and mutiple regression to construct a linear model from a set of data points.
-
Apply least-squares to fit polynomials and other curves to data.
One of the most useful applications of finding least squares solutions is to determine curves of best fit for data.
The graph above gives an approximate linear relationship between \(x\) and \(y\text{.}\)
-
The black circles are data.
-
The blue line is the least squares line, also known as the line of best fit.
-
The lengths of the red lines are the errors between our line of best fit and the data.
-
The least squares line minimizes the sum of squares of the errors.
Example 6.60.
Suppose we want to construct a line of the form \(y=mx+b\) that best fits the data below.
(a)
Find a system of equations \(A\vec x = \vec b\) that can be used to solve for \(m\) and \(b\text{.}\)
(b)
Is the system found in (a) consistent?
(c)
Identify the best choices of \(m\) and \(b\) by finding the least-squares solution to \(A\vec x = \vec b\text{.}\)
We can use this technique to find line of best fit even when there are more than two variables.
Example 6.61.
(a)
Setup the least-squares problem which will produce the best linear model for this situation:
\begin{equation*}
y = \beta_0 + \beta_1 x_1 + \beta_2 x_2\text{.}
\end{equation*}
(b)
Solving the system above, we find that
\begin{equation*}
\widehat \beta = \begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \end{bmatrix} = \begin{bmatrix} 120 \\ 35 \\ 30\end{bmatrix}\text{.}
\end{equation*}
According to this model, how much should a house with 2000 ft\(^2\) and 3 bedrooms be worth?
We do not have to restrict ourselves to linear curves of best fit. All that is required to use Least Squares is that the unknowns we are trying to solve for, our coefficients, are linear. There is nothing wrong with allowing the variables to be nonlinear.
In general, we can consider least squares fitting for the form
\begin{equation*}
y = \beta_0 + \beta_1 f_1(x_1, \dots, x_m) + \beta_2 f_2(x_1, \dots, x_m) + \cdots + \beta_n f_n(x_1, \dots, x_m)\text{,}
\end{equation*}
where the functions \(f_i\) are known, but not necessarily linear. Keep in mind that this is a linear problem in the \(\beta_i\) variables.
Example 6.63.
For the remaining four examples, we will fit various curves to the following points:
\begin{equation*}
(-1,-3), (0,0), (1,3), (2,2)\text{.}
\end{equation*}
Once you find the curve, graph the points and the curve on Desmos and see how good the curve fits.
We will start by fitting a linear curve to the data:
\begin{equation*}
y = \beta_0 + \beta_1 x\text{.}
\end{equation*}
This one can be solved without the need of a calculator.
Example 6.64.
Continuing from Example 6.63, we will now fit a quadratic curve to the data:
\begin{equation*}
y = \beta_0 + \beta_1 x + \beta_2 x^2\text{.}
\end{equation*}
For the remaining problems, feel free to use a matrix equation solver after setting up the least squares problem as \(A^TA\widehat x = A^T\vec b\text{.}\)
Example 6.65.
Continuing from Example 6.63, we will now fit a cubic curve to the data:
\begin{equation*}
y = \beta_0 + \beta_1 x + \beta_2 x^2 + \beta_3 x^3\text{.}
\end{equation*}
What do you notice about your least-squares solution? Is this expected?
Example 6.66.
Continuing from Example 6.63, we will now fit a somewhat trigonometric curve to the data:
\begin{equation*}
y = \beta_0 x + \beta_1 \sin\left(\frac{\pi x}{2}\right)\text{.}
\end{equation*}
What do you notice about your least-squares solution?
