0:01

Hi, guys.

Â Rich here, one of your friendly Coursera TA's at the University of Washington.

Â And today I'm here to talk to you about math.

Â As the course progresses, I'll be going through a couple little miniature

Â mathematical tutorials regarding the topics that come up during the lectures

Â for those of you who could maybe use a refresher on the mathematics.

Â I won't be going through detailed proofs and derivations,

Â which I'm sure some of you are very thankful for.

Â Instead, I'll be focusing more on the intuition and how we can start to think

Â about different mathematical objects in the terms of computational neuroscience.

Â 1:14

So, some of you may have heard about vectors in a physics or engineering class,

Â for example.

Â Things that have a direction and a magnitude or whatever, whatever.

Â But today, while we won't be inconsistent with that understanding,

Â we are going to start to think about vectors in a more abstract way

Â that will be useful for us as computational neuroscientists.

Â 2:00

Nothing more, nothing less.

Â Just a list of numbers.

Â What's one example?

Â What are several examples?

Â Here's an example, (3,1).

Â That's a list of two numbers.

Â Here's another example (-10,8), another list of two numbers.

Â want to get crazy?

Â Let's add another number.

Â Let's say (3, 6, -5).

Â That's a list of three numbers.

Â Let's make a list of five numbers,

Â (-1, 2, 8, 6, -5).

Â 3:29

This one is a 5D vector.

Â Now, there's something very special about 2 and

Â 3D vectors that is not quite the case with higher dimensional vectors.

Â And that is that, for 2 and 3D vectors you can draw them as arrows or

Â points in the plane or in some three-dimensional space.

Â How would we draw the vector 3,1?

Â First we would start with our 2D plane.

Â And let's label the axis x1 and x2, and

Â we want to plot the vector 3,1 in our plane.

Â How do we do that?

Â 4:11

First we look up the x1 coordinates.

Â Well, that's just the first number, that's 3.

Â So we going to go over 1, 2, 3.

Â Now we look up the x2 coordinates.

Â That's 1.

Â So we've been over 3,

Â since the x2 coordinate is 1 we go up 1 and we put a point there.

Â Pretty crazy, right?

Â I know, I thought so too.

Â Sometimes, maybe in physics and engineering,

Â you'll see the vectors drawn as arrows.

Â But often in neuroscience you'll just see them as dots.

Â So for now we'll just leave it as a dot.

Â 4:49

Sound good?

Â So that's 3,1.

Â What if we wanted to plot the vector?

Â -2, 4.

Â Well we go over to over -2, and up 4.

Â 1, 2, 3, 4.

Â So we can plot multiple vectors in the plane as points.

Â You can do the same thing with 3D vectors if you're good at drawing 3D.

Â I am not, unfortunately.

Â So, I wont give you an example there.

Â However, when you get to four dimensional and

Â five dimensional vectors it becomes awfully hard to draw pictures.

Â Yet, all the mathematical wonders we can do with 2 and

Â 3D vectors we can do with 5D vectors as well.

Â So now we're going to talk about the basic operations you can do with vectors.

Â And the reason we introduce them with vectors

Â is because we want to understand their applications to functions as well.

Â What is a very simple operation?

Â This first simple operation is just called the sum operation.

Â Let's say we have the vector x equals, you can put numbers, but I'm just

Â going to leave them as variables for now so that they can mean anything you want.

Â And I'm going to have them be arbitrarily dimensioned.

Â So, what is the dimensionality of this vector?

Â This is just ND vector, a list of N numbers.

Â How do we take the sum?

Â Well, it's very simple.

Â All you do is add all of the elements together.

Â Not too hard, right?

Â So, with the sum operation.

Â What goes in?

Â A vector goes in, that was a list of numbers.

Â And what comes out?

Â Another vector?

Â No, a scalar, which is just a single number.

Â And the reason the output is a scalar is because each of the elements

Â in the vector x was a scalar.

Â So when we add a bunch of scalars together, we get another scalar.

Â All right, next and most important operation is the dot product.

Â This will come up over and over again in your life.

Â And if you understand it, you will be a better person.

Â So how does the dot product work?

Â First of all, how do we write the dot product?

Â So let's say vector x = (x1,

Â x2,..., xn,

Â and the vector y = y1,

Â y2, ..., yn.

Â Then the dot product is just written as x.y.

Â Pretty crazy.

Â In order to compute the dot product, all we do is multiply together

Â the elements of x and y, element ys and then add the whole thing up.

Â We'll start with x1 and y1, multiply those together.

Â And to that we add the product of x2y2, and to that we add the product of x3y3.

Â And we keep going and going and

Â going until we get to the end, where we add the product xnyn.

Â 8:15

So let's do an example.

Â Let's say x, we'll just use four d vectors for now.

Â Let's say x is minus 1,0, 5 and

Â 2 and y is equal to 0, 2 minus 3, 6.

Â What is x.y equal for?

Â How do we do it?

Â Let's start with the first element,

Â is minus 1 times 0, 0 times 2,

Â then 5 times minus 3, added to 2 times 6.

Â Did I do that right?

Â I think so.

Â So what's that look like?

Â 0 plus 0 minus 15 plus 12 equals minus 3.

Â Not too hard, right?

Â 9:33

The input is two vectors.

Â And what's the output?

Â The output, here let's draw arrows on these things so they make more sense.

Â So the input to the dot product is two vectors and the output is a scalar.

Â An important thing to not is that you can take the sum and

Â dot products regardless of the dimensionality

Â of the vector or vectors in the case of the dot product.

Â This will come in handy later.

Â Okay, but returning to the dot product, why is the dot product useful?

Â Some of you may enjoy multiplying pairs of numbers together and

Â adding them up on a Friday night.

Â But I promise there's actually a reason to do this too, besides mere boredom.

Â Let's return to our geometric understanding of vectors.

Â So this is x1, that's

Â x2 and let's take the dot product between two vectors,

Â where we draw them out and see what things look like geometrically.

Â 11:17

And what's x2?

Â So that's x1, x2 you go over 1 up 5, that's x2.

Â Oops, let's stay consistent and leave the 1 up there.

Â What are we actually doing when we are computing the dot product?

Â What the dot product does is it tells us how aligned two vectors are.

Â So, how it works, and this is actually one of the reasons I sometimes call it

Â a dot product projection, a term you will be hearing a lot throughout our lectures.

Â Is that we imagine a light shining.

Â So we are going to draw an axis perpendicular to x1 and

Â we are going to imagine a light, a big bright light,

Â maybe the sun, shining down on x2 and on x1.

Â 12:13

And this light will cast a shadow of x2 over x1,

Â or if you like it projects x2 onto x1.

Â Crazy, right?

Â Now we can ask, what is the product of the magnitude of x1 and

Â the component of x2 that is lined up with x1.

Â Or what is the product of the magnitude of this blue vector and

Â the magnitude of this red vector that was x1?

Â Well, it is the dot product.

Â 12:50

All we do is calculate the dot product.

Â So x1.x2 equals 3 times

Â minus 1 plus 3 times 5

Â which is equal to 12.

Â So, that means that the length of the blue vector, or the magnitude of the blue

Â vector, multiplied by the magnitude of the red vector x1 is equal to 12.

Â I'm not going to prove to you why that is the case, but

Â if you're interested it's a fun exercise for the reader.

Â But, anyhow,

Â that's a good way of thinking about what the dot product does geometrically.

Â It's kind of hard to picture for things like five or ten or

Â 20 dimensional vectors, but it works in generally the same.

Â It calculates the amount that the two vectors are lined up.

Â So, just as a quick thing, let's introduce, what color can we use?

Â Cyan.

Â Vector x3 which is equal to 4 minus 4.

Â Where would we draw that?

Â 1, 2, 3, 4, up 4 so that's x3.

Â 15:12

Now we are going to introduce functions.

Â And the idea is that what you have learned about vectors

Â will help you understand what you can do with functions.

Â So what's a function?

Â Just a refresher.

Â Here's an example of a function f of x equals 3x minus 2.

Â A function takes an input or argument x and

Â it spits out a function value f of x.

Â So if x were equal to 2, we would say f of 2 is equal to 3 times 2 is 6 minus 2 is 4.

Â Not too hard, right?

Â 15:50

And what does this function look like?

Â Well, first of all it's just a line and it's got a slope of three and

Â y-intercept of, -2.

Â So we start at -2 and for every one we go over we go up 3, about like that and

Â it extends forever in to the infinite beyond in both directions.

Â Okay, so that's one way to think of a function and

Â a way that a lot of you have probably encountered before.

Â Today, we are going to try to think of a function as a vector.

Â That might sound crazy, but

Â I promise there are crazier things in math than thinking of a function as a vector.

Â 16:31

So let's pick a new example and

Â I'll try to tell you what the heck I'm talking about.

Â Let's pick the example f(x), we'll call it f1(x),

Â just in case we need to introduce more functions, is equal to x/2.

Â And we're going to say that x is between 0 and 1, we'll make that inclusive.

Â 17:32

And how does that work?

Â Well, well, let's do this.

Â Let's pick a few points on our function.

Â Let's say we'll pick 0 because everybody loves 0.

Â We'll pick 0.1, 0.2, 0.3, 0.4, 0.5, 0.6 and so forth.

Â And now what we're going to do is we're going to write down the function

Â values at each of those numbers.

Â So this was 0, 0.1, 0.2, 0.3,

Â these are the x values, 0.4, and so on and so forth.

Â So what is at f(0)?

Â f1(0)?

Â That's just 0.

Â What is f1(0.1)?

Â That's 0.1 over 2, which is point 0.05.

Â What is half of 0.02?

Â Well that's just 0.02 over 2, 0.1.

Â So we can continue to fill this out and

Â hopefully this is starting to look a little familiar to you.

Â 18:43

Now we didn't have to choose 0.1 as our increment.

Â We could have chose 0.01 in which case

Â our vector would look like 0.005,

Â 0.01, 0.015 and so on and so forth.

Â And if we wanted to not only 0.01 as our increment for

Â x, but 0.001 or 0.0001.

Â And in fact, you can keep making your increments smaller and smaller and

Â smaller until you have an infinite list of numbers.

Â But, the key thing to remember is that

Â the function is still just a list of numbers.

Â And I claim this is true because if you had all the numbers for

Â every increment that would specify your function completely.

Â 19:39

Therefore, the function is a list of numbers or,

Â as we like to call it, a vector.

Â Is that crazy?

Â Yeah, it's pretty crazy.

Â And I'm not going to go into the math to actually prove that, but

Â hopefully the intuition makes some sense.

Â Okay.

Â Now that we can think of functions as vectors.

Â And if that doesn't quite sit comfortably with you spend a few minutes.

Â Pause the video and just really let yourself that this is true,

Â that functions are vectors.

Â Because they are nothing more than a really,

Â really, really detailed list of numbers.

Â 20:21

Okay, that makes sense.

Â So let's move on to operations, but

Â now performed not with vectors, but with functions.

Â And we actually only talked about two operations of vectors, and

Â those will be the two operations we talk about with functions.

Â So the first one in the vector world it was the sum.

Â And for those of you who have taken calculus, what happens to sums when we

Â move our elements infinitesimally closer together and

Â let there be an infinite number of them?

Â Well, that becomes an integral.

Â And the sum becomes an integral because that is how you would most naturally add

Â up an infinite list of numbers, which was our function.

Â 21:11

The integral, if we have some arbitrary function, half of x versus x,

Â and we'll again have it justified under the domain from 0 to 1.

Â The integral is just the area under the curve.

Â That's not too hard really.

Â There are lots of good calculus textbooks that tell you exactly how to compute

Â the area under the curve.

Â But what we're interested in is just the fact that it is this integral.

Â So here's another reason why the sum and the integral are very similar.

Â 23:03

Well, when we deal with infinities we go to calculus.

Â So that sum over all of the pair-wise products

Â becomes an integral over the pair-wise products.

Â So what does that look like?

Â The projection of one function onto another.

Â Let's say the projection of, and

Â sometimes we write this in the same way as a dot product.

Â We say f1 of x dotted with f2

Â of x is equal to the integral.

Â And let's just work from 0 to 1 now.

Â You can pick other bounds if you like, but for now since we've been using 0 and

Â 1 as our bounds, we'll stick with those.

Â The dot product of the two functions is the integral of the products

Â of the functions.

Â Crazy.

Â Does that make sense?

Â I hope so.

Â And the intuition is that this is just the natural generalization

Â of dot products to the world of functions.

Â Sums become integrals.

Â So for our sum operation, we were adding together a bunch of elements.

Â That becomes an integral.

Â In the dot products operation, we were adding together a bunch of these products.

Â But remember each product was just a number.

Â So it was a sum of a bunch of numbers, therefore,

Â in the function world we're just doing an integral of these products.

Â Hopefully that makes sense.

Â Okay, so let's do a quick example of how we would compute the dot product of two

Â functions, or the projection of one function onto another.

Â So, remember that is the integral.

Â Let's make it general, let's just say from a to b,

Â f1(x) times f2(x), dx.

Â So let's say that f1(x), we'll draw that in green,

Â is something like a line, so f1(x).

Â Always label your axes.

Â And let's say f2(x) is something that looks like that.

Â 25:55

That's not too bad, right?

Â I think that's okay.

Â So that's the product of f1(x) and f2(x).

Â And then the integral is just the area under that curve.

Â So the integral of f1(x) and f2(x) from

Â a to b, dx is just the shaded area.

Â Cool?

Â Cool.

Â So what is the dot product between two functions actually telling you?

Â 27:40

So let's say f1(x) is this parabola like thing,

Â and f2(x), some sort of sinusoid like that.

Â That's f2(x), and so this is a and b.

Â What is the dot product of f1(x) and f2(x)?

Â How do we figure this out?

Â Well, we multiply f1(x) and

Â f2(x) together.

Â So in this part, it's negative because f1(x) is positive, but f2(x) is negative.

Â And in this part, it's positive because f1(x) and f2(x) are both positive.

Â And now we take the integral.

Â So, if you were to take the integral of that blue curve,

Â which is the product of f1(x) and f2(x) what would you get?

Â And the answer is 0.

Â Why is the answer zero?

Â Well, the first lobe is under the x-axis, so that's a negative area.

Â And the second lobe is above the x-axis, so that's a positive area.

Â And since the functions were at least supposed to be kind of symmetric,

Â the area underneath the x-axis cancels out the area above the x-axis,

Â which means that the integral is 0.

Â So you would say that these two functions are orthogonal.

Â So we would write that f1(x) and

Â f2(x) are orthogonal.

Â And so all orthogonal means in this case is that they don't overlap at all.

Â Or, rather, their overlaps cancel each other out.

Â That's about it for right now.

Â I hope that made some sense.

Â I hope that made things not more confusing, but

Â at least a little less confusing.

Â The idea Idea of projecting one function on to another, will come up over and

Â over again, especially in convolutions in Linux systems.

Â So it's a good thing to make sure you understand at least basic idea of.

Â Okay, that's it, see you next time.

Â