So now we have set of vector, set of scalars,
we know how to multiply vectors by scalars and
we know how to put vectors together by addition, but we need something more.
We need something to measure and compare vectors Vectors.
This is provided by the inner product which is an additional
new operator that we defined for the vector space.
The inner product takes a couple of vectors and
returns a scalar, which is a measure of similarity between the two vectors.
If the inner product is zero,
this is a very important concept that we will examine in detail,
we say that the vectors are maximally different or in other words, orthogonal.
Like for scalar multiplication and addition, the inner product is defined
axiomatically and these are the properties that it will have to fulfill.
We want the inner product to be distributive with respect to
vector addition.
We want the inner product to be commutative with congregation.
This, of course, applies when our set of scalars is complex valued.
We want the inner product to be fistributed
with respect to scalar multiplication and
we conjugate the scalar if it affects the first operant in the inner product.
We want the self inner product to be greater than or equal to zero,
which means that the self inner product is necessarily a real number.
And finally, the self inner product can be zero only if the element is
the null element for the vector edition.
Let's look at some examples of inner products to foster our intuition.
In R2 the inner product between two vectors is defined as the product of
the first component of each pair,
plus the product of the second components of each pair.
Now this is not immediately intuitive, so let's take it apart a bit.
First of all if we look at the self inner product of a vector, we can see that it's
equal to x0 squared plus x1 squared, so the sum of the coordinate square.
Now if we look at the graphical representation of the vector we can see
that this by Pythagoras' theorem is equal to the length of the vector square.
We actually have a name for this, we call this the norm of a vector.
And so the norm of a vector is the square root of the self inner product,
and it indicates the length of the vector.
So, the self inner product is a very good measure of the size of a vector
in R2 and by extension in all vector spaces.
Let's go back to the general definition for the inner product.
We could use some simple trigonometry to show that
this formulation is actually equivalent to this.
So the inner product between two vectors
is equal to the norm of the first vector times the norm of the second vector
times the cosine of the angle formed by the two vectors.
Now if the two vectors have equal norm
their inner product will be a good measure of similarity between this two vectors,
because it will go from the norm square when the vector go inside and
therefore angle alpha is zero to zero when the vectors are at 90 degrees and
therefore are pointing in orthogonal directions.
So, orthogonality is indeed the maximal difference
between two vectors on the plane.
As another example, lets have a look at the inner product in L2 of -1, 1.
This is usually defined as the integral of the product of the two functions
that corresponds to the two vectors.
It's very easy to verify that this inner product fulfills the axioms.
Let's now use this inner product to compute the norm
of the vector x = sin of pie t.
This will be equal to the self inner product, and so
the integral from -1 to 1 of sin squared of pie t in dt.
This is the area that is shaded in the picture and it's equal to one.
Let's take another element of L2 -1,
1 the vector y = t, so the linear ramp between -1 and 1.
If we compute the norm as the self inner product of this vector, we have
the integral from -1 to 1 of t squared and dt which is equal to two thirds.
So, this vector doesn't have the unit norm, if we want to normalize this vector,
we can define an alternate version where we say y=
t divided by the square root if two thirds.
We divide the vector by the norm and this vector will now have the unit norm.
The reason why we do that is because now we can use the inner product to compare
in L2 of -1, 1, the function sine of pi t
with the function t divided by the square-root of two thirds.
So, here we have the first function,
here we have the second function, we compute the inner product between the two.
We have to compute this area here, the result of the inner product is 0.78.
Now remember that we're comparing unit norm of vectors so
the inner product can range between one, in the case of maximal similarity,
and zero in the case of orthogonality.
Here we have a value of 0.78 which indicates that the linear ramp and
the sine functions are actually pretty close.
We can take the inner product with maximally the similar functions
if we take our usual x equal to sin of pi t, which is an antisymmetric function.
We take the inner product of this function with a symmetric function, like for
instance, the triangle function, 1 minus absolute value of t.
If we take the integral of the product of these two functions,
we have to sum the green area with the red area.
These two areas have opposite sides, and so the integral and, therefore,
the inner product, will be equal to zero.
So, the inner product has successfully captured the fact that we will
never be able to express any symmetric function in terms of
an antisymmetric shape.orthogonality.
There are maximally different, they have nothing in common,
their inner product is zero.
Another famous case of orthogonality between functions
is given by sinusoids whose frequencies are harmonically related.
So we're still in L2 of -1, 1.
Let's pick as the first function, sin of 4 pi t, and
as the second function, sin of 5 pi t.
These two frequencies are multiples of the fundamental frequency for
the interval, which is pi.
If we compute the inner product between the two, even graphically,
we can see that we have to compute these areas here and sum them together.
Now for each red colored area you have a corresponding
green colored area that has opposite sign.
And so as you sum them together, you end up having a total inner product of zero.
