Loading...

2.2.a Vector space

Course video 10 of 108

Digital Signal Processing is the branch of engineering that, in the space of just a few decades, has enabled unprecedented levels of interpersonal communication and of on-demand entertainment. By reworking the principles of electronics, telecommunication and computer science into a unifying paradigm, DSP is a the heart of the digital revolution that brought us CDs, DVDs, MP3 players, mobile phones and countless other devices. The goal, for students of this course, will be to learn the fundamentals of Digital Signal Processing from the ground up. Starting from the basic definition of a discrete-time signal, we will work our way through Fourier analysis, filter design, sampling, interpolation and quantization to build a DSP toolset complete enough to analyze a practical communication system in detail. Hands-on examples and demonstration will be routinely used to close the gap between theory and practice. To make the best of this class, it is recommended that you are proficient in basic calculus and linear algebra; several programming examples will be provided in the form of Python notebooks but you can use your favorite programming language to test the algorithms described in the course.

Visualizar o programa do curso

Habilidades que você aprenderá

Signal Processing, Discrete Fourier Transform, Data Transmission, Ipython, Fourier Analysis, Convolution, Linear Algebra, Digital Signal Processing

Ministrado por

Paolo Prandoni
Lecturer
Martin Vetterli
Professor

Transcrição

Instead of introducing vector space inductively,

namely by listing the axioms that regulate the behavior of vectors in vector space,

let's start with some examples that you should be familiar with.

R2 and R3 are also known as the Euclidean plane and

Euclidean space, these are very familiar entities.

The extension of Euclidean space to an arbitrary number of dimension is

called RN that is the subject of linear algebra.

Another extension uses complex numbers instead of real numbers, so

we have CN, vectors that are made of tuples of N complex numbers,

again the subject of linear algebra.

Other vector spaces are perhaps less known, we have L2 of Z.

This is the space of square-summable infinite sequences,

we will see that this is useful for modeling infinite like signals.

And L2 of a, b is the space of square integrable functions over an interval.

Here, the key point is that vectors can be arbitrarily complex entities,

such as functions for instance.

And this will be very useful in unifying our approach to signal processing.

Some spaces, some vector spaces can be represented graphically this is the case

for instance of planar geometry.

The intuition is that if you take the plane and set up a cartesian reference

system like in this figure here, then you can identify any point on the plane,

with a pair of real valued coordinates like so.

This correspondence between points on the plane and

vectors in R2 allow us to interpret the properties of vectors in R2

in terms of geometric properties on the Euclidean plane.

This analogy is extremely useful, especially when extended also to three

dimensions, because it will allow us to use our geometric intuition to understand

the properties of arbitrarily complex vector spaces that don't allow for

an intuitive representation or a graphical representation.

In three dimensions,

we use triples of coordinates to locate uniquely a point in space, and

at four we have a complete correspondence between Euclidean space and R3.

So, here we have our point in space and it's 3D coordinates will be like so,

and again, geometrical intuition is pretty straightforward.

Let's look now at a function vector space, this is L2 of -1,1,

so the space of square integrable function on the integral -1,1.

Any element of this vector space is a function, so this is the vector notation,

and then explicitly it will be a function of a variable t that goes from -1 to 1.

An element for instance is the sine function we can take x equal to

sine pi t and the vector can be drawn by plotting the function over the interval.

Other vector spaces do not admit to graphical representation, for

instance RN for N > 3.

Well there is no way that we can block that in a way that we can comprehend.

And CN, so the tuples of complex numbers cannot be drawn for anything but

N equal to 1 which case we have the classical complex plane.

So from this examples,

we can see that vectors spaces can be a very diverse bunch.

And what all these vectors spaces have in common is that they obey to set of

an axioms that define the properties of the vectors and

what we can do with these vectors.

So the ingredients for a basic vector space are a set of vectors V,

and we say nothing about what is inside these vectors.

And a set of scalars that in our examples will always be the set of complex numbers.

And we need to be able to at least resize vectors, so multiply them by a scalar,

to change their size, and combine vectors together, sum them together.

The way we do this is by defining some axiomatic properties for the sum and

for the scalar multiplication.

So this is pretty dry, but we will go through these axioms, and

then we will understand that all the vector spaces that we

have seen in the examples actually do fulfill these axioms.

So we want the addition to be commutative, we want the addition to be distributive,

we want the scalar multiplication to be distributive with respect

to vector addition and we want scalar multiplication to be distributive

with respect to the addition in the field of scalars.

Scalar multiplication is associative as well, we want to have a null element

in the vector space, so that the sum of any vector plus the null element.

And because of commutitivity,

the sum of the null element plus the vector will give back the original vector.

And then we want that for every element in the vector space,

there exists an inverse element for the addition.

So that X plus -x as we indicate the inverse for

the addition will be equal to the null vector.

All right, so let's look at an example if we take RN the space of tuples of

capital N real numbers and we define the basic operations like so.

So we have scalar multiplication, we take a tuple and

we multiply each element in the tuple by a scalar alpha.

And then the sum is defined as a tuple where each element is the sum

of the corresponding elements of the two tuples.

So if we define the operations like so, then we can easily verify that these

operations fulfill the axioms and therefore RN is a valid vector space.

We can, as we said, use the geometric intuition in low dimensions to understand

what these operations do.

Consider the scalar multiplication vector in R2,

well the vector is identified by a pair of coordinates, X0 and X1.

And if we multiply each component in the pair by alpha, we are in fact,

lengthening the vector by a factor alpha without changing its direction.

Vector addition in two dimension corresponds to summing the first element

of each tuple together and this becomes a new first element of the sum, and

then we sum the second components together.

Graphically we take our first vector, we take the second vector,

and we place it at the end of the first vector without changing it's direction.

And the new vector will be the arrow that goes from the origin to the end of

this combination of vectors.

This is the famous Parallelogram Rule.

In L2 of -1,1, scalar multiplication is

intuitively defined as multiplying the function by a scalar alpha.

So here we have the vector sine of pi t, we multiply this vector by a scalar 1.5,

and what we get is a function with a wider amplitude.

Sum is defined intuitively as well in L2 of -1,1 as simply

the sum of the two functions over the interval.

So we take our first vector sine of pi t, a second vector 0.33 times sine

of 25 pi t we sum them together and we get this other element of the vector space.

So now we have set of vector, set of scalars,

we know how to multiply vectors by scalars and

we know how to put vectors together by addition, but we need something more.

We need something to measure and compare vectors Vectors.

This is provided by the inner product which is an additional

new operator that we defined for the vector space.

The inner product takes a couple of vectors and

returns a scalar, which is a measure of similarity between the two vectors.

If the inner product is zero,

this is a very important concept that we will examine in detail,

we say that the vectors are maximally different or in other words, orthogonal.

Like for scalar multiplication and addition, the inner product is defined

axiomatically and these are the properties that it will have to fulfill.

We want the inner product to be distributive with respect to

vector addition.

We want the inner product to be commutative with congregation.

This, of course, applies when our set of scalars is complex valued.

We want the inner product to be fistributed

with respect to scalar multiplication and

we conjugate the scalar if it affects the first operant in the inner product.

We want the self inner product to be greater than or equal to zero,

which means that the self inner product is necessarily a real number.

And finally, the self inner product can be zero only if the element is

the null element for the vector edition.

Let's look at some examples of inner products to foster our intuition.

In R2 the inner product between two vectors is defined as the product of

the first component of each pair,

plus the product of the second components of each pair.

Now this is not immediately intuitive, so let's take it apart a bit.

First of all if we look at the self inner product of a vector, we can see that it's

equal to x0 squared plus x1 squared, so the sum of the coordinate square.

Now if we look at the graphical representation of the vector we can see

that this by Pythagoras' theorem is equal to the length of the vector square.

We actually have a name for this, we call this the norm of a vector.

And so the norm of a vector is the square root of the self inner product,

and it indicates the length of the vector.

So, the self inner product is a very good measure of the size of a vector

in R2 and by extension in all vector spaces.

Let's go back to the general definition for the inner product.

We could use some simple trigonometry to show that

this formulation is actually equivalent to this.

So the inner product between two vectors

is equal to the norm of the first vector times the norm of the second vector

times the cosine of the angle formed by the two vectors.

Now if the two vectors have equal norm

their inner product will be a good measure of similarity between this two vectors,

because it will go from the norm square when the vector go inside and

therefore angle alpha is zero to zero when the vectors are at 90 degrees and

therefore are pointing in orthogonal directions.

So, orthogonality is indeed the maximal difference

between two vectors on the plane.

As another example, lets have a look at the inner product in L2 of -1, 1.

This is usually defined as the integral of the product of the two functions

that corresponds to the two vectors.

It's very easy to verify that this inner product fulfills the axioms.

Let's now use this inner product to compute the norm

of the vector x = sin of pie t.

This will be equal to the self inner product, and so

the integral from -1 to 1 of sin squared of pie t in dt.

This is the area that is shaded in the picture and it's equal to one.

Let's take another element of L2 -1,

1 the vector y = t, so the linear ramp between -1 and 1.

If we compute the norm as the self inner product of this vector, we have

the integral from -1 to 1 of t squared and dt which is equal to two thirds.

So, this vector doesn't have the unit norm, if we want to normalize this vector,

we can define an alternate version where we say y=

t divided by the square root if two thirds.

We divide the vector by the norm and this vector will now have the unit norm.

The reason why we do that is because now we can use the inner product to compare

in L2 of -1, 1, the function sine of pi t

with the function t divided by the square-root of two thirds.

So, here we have the first function,

here we have the second function, we compute the inner product between the two.

We have to compute this area here, the result of the inner product is 0.78.

Now remember that we're comparing unit norm of vectors so

the inner product can range between one, in the case of maximal similarity,

and zero in the case of orthogonality.

Here we have a value of 0.78 which indicates that the linear ramp and

the sine functions are actually pretty close.

We can take the inner product with maximally the similar functions

if we take our usual x equal to sin of pi t, which is an antisymmetric function.

We take the inner product of this function with a symmetric function, like for

instance, the triangle function, 1 minus absolute value of t.

If we take the integral of the product of these two functions,

we have to sum the green area with the red area.

These two areas have opposite sides, and so the integral and, therefore,

the inner product, will be equal to zero.

So, the inner product has successfully captured the fact that we will

never be able to express any symmetric function in terms of

an antisymmetric shape.orthogonality.

There are maximally different, they have nothing in common,

their inner product is zero.

Another famous case of orthogonality between functions

is given by sinusoids whose frequencies are harmonically related.

So we're still in L2 of -1, 1.

Let's pick as the first function, sin of 4 pi t, and

as the second function, sin of 5 pi t.

These two frequencies are multiples of the fundamental frequency for

the interval, which is pi.

If we compute the inner product between the two, even graphically,

we can see that we have to compute these areas here and sum them together.

Now for each red colored area you have a corresponding

green colored area that has opposite sign.

And so as you sum them together, you end up having a total inner product of zero.

In general, if we have a vector space equipped with an inner product,

we have a natural norm defined on the space,

which is the square root of the self inner product.

We can use this notion of norm to define a distance between vectors

as the norm of the difference between vectors.

Again, in R2 this is very easy to visualize,

take one vector, take the other vector.

Now the difference between two vectors is the vector that connects their

end points and the norm of this connecting vector

is a quantitative measure of distance between the original vectors.

Note that the distance is not like orthogonality,

as a matter of fact orthogonal vectors can have a very large distance,

the distance is certainly not zero.

In function vector spaces the norm of the difference between vectors is

usually known as the mean square of error.

It is indeed the integral for instance in the case of L2 of -1,

1 of the difference squared between two functions.

So here we have as an example of our first vector sine of pi 4t,

the second function is sine of 5 pi t,

their difference will be this weird function here.

And if we compute the area of this functions squared we have a value of two

which is the distance between this two harmonically related functions.

Sobre o Coursera

Cursos, especializações e graduações on-line, ministradas pelos melhores instrutores das melhores universidades e instituições de ensino.

Join a community of 40 million learners from around the world

Earn a skill-based course certificate to apply your knowledge

Gain confidence in your skills and further your career

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2019 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play