But at any rate, our discussion from orthonormal bases notes that because of

course U is orthonormal, grabbing any three columns of U,

especially the first three columns of U, is also going to be orthonormal.

And so our estimate of gamma, our gamma hat

is just going to be u 3 transposed times y.

Okay so what we find is that the way in which we get sort of

principal component regressors is simply by taking

the singular value decomposition of our centered X matrix,

taking the relevant columns from our score.

Our vector last singular values which if we think

about it in terms of principal components, as our scores and

then if we simply multiply them by y multiply the transpose of them times y.

We actually get the associated coefficients.

So this just goes to show how we can use these nice operations that

we get out of squares in this particular case.

So using the singular value decomposition to come up with the orthonormal

basis I think represents of the three most important bases,

concepts and statistics, certainly I would describe wavelengths,

transforms, and principal components basis as the three.

And I think you can see that in this case

it fits very nicely into the topic of regression.

And it also fits very nicely if we have a large

X matrix with a lot of columns that we want to summarize.

One caveat, I would suggest to be careful of.

Again, we get U.

We can think of U as these linear combinations of our columns of X.

If the units of x don't make sense to combine

then this procedure may not make a lot of sense to do.

So if the first column of x is a particular kind of units and

the second column of x is a different kind of units then the interpretability

of your scores may really suffer as a result.

So again, there's a lot of intricacies to do this.

And I think if you wanted to learn more about this,

a class on multivariate statistics would be the way to go.

But I just wanted to reinforce this point that when we have a design matrix that's

orthonormal, we work out with a really simple solution for the coefficients.

Okay, and next we'll go through a coding example where we go through some

of these sorts of examples.