Suppose you are in a science class and you receive these instructions:
Find the temperature of the water (in degrees Celsius) at the times 1, 2, 3, 4, and 5 seconds after you have applied heat to the container. Conduct your experiment carefully. Graph each data point with time on the The data from the experiment looks like this when charted and graphed:
This is a common problem with experimental sciences because the data points that we measure seldom fall on a straight line. Therefore, scientists try to find an approximation. In this case, they would try to find the line that best fits the data in some sense. The first problem is to define "best fit." It is convenient to define an error as a distance from the actual value of You have just read a lot of new information, so let's illustrate the concepts with our example. We have the graph of the data above. Now we need to guess which line best fits our data. If we assume that the first two points are correct and choose the line that goes through them, we get the line
If we choose the line that goes through the points when The sum of the squares of the error is 18. That is a better fit, but can we do even better? Let's try the line that is half way between these two lines. The equation would be
A line in slope-intercept form looks like
In general, we cannot solve this system because the system is usually inconsistent because it is overdetermined. In other words, we have more equations than unknowns (the unknowns are the two variables,
The normal equations will give us the "best fit" line (or curve) every time according to the way we defined "best fit." The proof of this is at the end of this chapter. Let's try applying the normal equations to our system. First, we multiply so that we have a system that we can solve.
The sum of the squares of the error is 2.7. This is a great improvement over our guesses and we know that we cannot do any better. In general, if we have
, and The ellipse marks (written as or tell you to continue in the same pattern. What if we are told that our data is not supposed to fit a straight line, but instead falls in the shape of a parabola? Consider the following data from another experiment:
We can find the curve that best fits our data in a similar manner. The general equation for a parabola is
These coefficients indicate that the curve we want is Let's graph this curve and fill in our chart:
We find that the sum of the squared errors is Using our definition of least squares "best fit," you will not be able to find a parabola that fits the data better than this one. In general, to find the parabola that best fits the data, you use the normal equations
Notice that the normal equations used to find the best fit line and the best fit parabola have the same form. Do you think that we could expand this to higher degree polynomials? Yes, we can. In general, we use the normal equations X^{T}Xc = X^{T}y with
where x. Remember that the degree is the highest power of the variable in your equation. A line is a first degree polynomial and a parabola is a second degree polynomial. ^{m}If we can find the best fit curve for any degree polynomial, why don't we always use a higher degree polynomial and fit the data better? After all, if we have If you notice, we said that we usually fit a curve so that we can predict what would happen between our data points. Predicting an outcome between data points is called
If we extrapolate back several years, this young man was over two and a half feet tall when he was born. According to this model, he will never stop growing, so he will be 8 feet 4 inches tall by the time he is 30 and almost 14 feet tall by the time he is 60. Do you think that this is an accurate prediction? If the temperature at the airport on the 4th of July was in the 90's for two years in a row, would it be reasonable to predict that the temperature in January between those years was also in the 90's? No, it would not. We have two problems with this model. One problem is that we only have 2 data points. You can always find a line that fits the two points, but there is no reason to believe that the relationship between the day of the year and the temperature is a linear relationship. Also, we didn't take into account other factors that could affect our model such as the pattern of the seasons. These are problems that can arise when you model a situation. When we start modeling situations and using least squares to make predictions, we are entering the world of statistics. That means that we must think about what the data represents rather than just apply the normal equations. There are many interesting applications of statistics that you can explore in another course. However, using matrices, you already know one way to find a "best fit" curve for your data. |

## Tuesday, September 1, 2009

### Least Squares Approximation

Subscribe to:
Post Comments (Atom)

## No comments:

## Post a Comment