How can robots perceive the world and their own movements so that they accomplish navigation and manipulation tasks? In this module, we will study how images and videos acquired by cameras mounted on robots are transformed into representations like features and optical flow. Such 2D representations allow us then to extract 3D information about where the camera is and in which direction the robot moves. You will come to understand how grasping objects is facilitated by the computation of 3D posing of objects and navigation can be accomplished by visual odometry and landmark-based localization.
Compute Projective Transformations
Loading...
Do curso por University of Pennsylvania
Robotics: Perception
388 classificações
Na lição
Projective Transformations
Now that we have a good camera model, we will explore the geometry of perspective projections in depth. We will find that this projection is the cause of the main challenge in perception, as we lose a dimension that we can no longer directly observe. In this module, we will learn about several properties of projective transformations in depth, such as vanishing points, which allow us to infer complex information beyond our basic camera model.
Conheça os instrutores
Kostas DaniilidisProfessor of Computer and Information Science
School of Engineering and Applied Science
Jianbo ShiProfessor of Computer and Information Science
School of Engineering and Applied Science
In this lecture we're going to talk about projective transformations and
how can we compute such transformations?
What is a projective transformation visually?
A projective transformation is any perspective projection of a plane.
You can see on the left the front of image of a book cover.
The way it appears, for example, in the original website.
On the right,
you see a pictures of the cover of the book Vision Science, by Stephen Palmer.
And, this is a projective transformation from the original front of image.
Any such picture which we take from a plane,
and corresponds to the original limits, mapped to the image plane,
is called a projective transformation.
For example, when we have a drone, flying over a planar pattern,
like the pattern you see with these codes in the picture.
Then we can compute the projective transformation from the robot position
to this pattern, and
then we can extract the pose of the robot from this planar pattern.
So such projective transformations are very useful.
Let us, and remember, again, what is a projective transformation?
When we have two projective plates, any invertible 3 by 3 matrix
transformation is a projective transformation.
And because we deal with homogeneous coordinates, we use the simple
of the tilde and write Tilde A times p which means that
point p is mapped through the projected transformation to the point p prime.
Tilde is always in the homogeneous coordinates,
it means that lambda p prime equals Ap.
The projective transformation is also known as a coordination or Homography.
The most important property of a projective transformation
is that it preserves incidence.
What is incidence?
Let's look at these three loop points on the floor.
These three loop points are collinear.
They're collinear on the original floor.
If you put the ruler there, they all lie on the same ruler.
If we take a picture from any viewpoint,
these three blue points will still remain on the same line on your image plane.
This is called preservation of collinearity.
Let's take now multiple lines, at least three lines.
And in the original floor, these three lines are parallel.
They're intersecting at infinity.
Then again, in the picture taking from any view point,
these three lines will intersect at the same point.
This is called preservation of concurrency.
So we have seen how, from the beginning,
we have seen how points map through a projective transformation, A.
A point p maps to a point Ap.
If we have a line, l, how is this line transformed?
Remember that in the projective plane,
we can have an inner product, l transposed p equals zero for the equation of a line.
If we replace this p with a inverse p primed,
then we get the equation of the line in the transform plane and
we see that the coefficients of the map line are a inverse transposed l.
How can we compute the projective transformation
between two projective planes?
One projective planes is the original floor built out of black and
white tiles, like a checkerboard.
And the other is a pixel plane, where we have just recorded
an image from this floor from some view point.
The projected transformation is in one to one transformation
between the two planes because it is invertible.
So if we have these three blue points, they transform to
unique points from the left to the right, and from the right to the left.
Which means that if we know the projective transformation then we can
take any button on the left and paint it on the right and vice versa.
Let us see how can we compute this projective transformation.
We are going to explain it very visually.
Let's say we had a point a, which corresponds to all the vanishing points,
which go to the right.
We will call them the vanishing point (1,0,0).
This corresponds to all the parallel lines that they're parallel to the x axis.
Let's have second vanishing point, again on the floor plane,
which corresponds, to all the lines vertical to the x axis,
which are the planes parallel to the y axis.
And we call this vanishing point, (0,1,0).
These two points, when I take the picture, are not anymore of infinite.
They correspond to the point on the right, in this yellow square, where this
parallel lines intersect, vanishing point one and vanishing point two.
Let's look at two more points.
One is a point where I define the origin of this ground floor,
which is the point (0,0,1).
And let us see if from these three points
we can compute the projected transformation.
We assume that we have the mapping A of the point (1,0,0).
We have the mapping b of the point (0,1,0) with two vanishing points.
And then we have the mapping of the origin (0,0,1), we call c.
Then we can write that a is alpha times a.
B is beta times b, and c is equal gamma times c.
This is very simple, because we just write the fact that all the homogeneous
vectors in the projective plane do not change when they multiply by a constant.
Now, this trivial fact can be written as a matrix.
We have a matrix alpha a, beta b, gamma c, which when multiplied
with an identity matrix gives a, b, c.
Why the identity matrix?
Because the first column of this matrix is
the vanishing point (1,0,0) mapping to the point a.
The second column is the vector 010, the second vanishing point.
And the third is the origin, 001.
So we have on the right all the points,
the three points we considered on the original plane.
And on the left-hand side we have their maps, and
we still see that we have three unknowns, alpha, beta, and gamma.
So these three points are not sufficient to compute the projective transformation.
We will add a fourth point, which is the point 1,1 in that
Euclidean plane of the ground floor or written homogeneous coordinates 1, 1, 1.
Now we map this point 1,
1, 1 by exactly the same transformation alpha a, beta b and
gamma c, into a new point, d.
Again because this is a projective transformation,
we have to multiply with a factor lambda.
So we have an equation lambda d is equal alpha a plus beta b
plus gamma c, and we'll try to solve for alpha, beta and gamma.
We see that we have four unknowns and
only three equation because these vectors are in a three but it doesn't matter.
Because if [COUGH] a, b, c are not co-linear, then for every solution alpha,
beta, gamma, the solution alpha divided by lambda beta divided by lambda and
gamma divided by lambda will always be the same transformation.
So we said lambda is equal to one, and
we obtain an equation which is the three columns, a, b,
c, our known images from the vanishing point (1, 0,
0) (0,1,0) and (0,0,1).
And on the right, the mapping of the point, (1,1,1).
And we solve for alpha, beta, gamma, a system of three equations and
three unknowns.
We can solve this because a,b,c are not co-linear,
and hence, the determinant of the a,b,c matrix is not zero.
After we solve it, the projective transformation that we find,
there is just alpha a, beta b and gamma c,
just using these three original points a, b and c.
What can we do with this transformation?
First we see that we didn't need any more points.
With four points it was more than enough to recover unambiguously
this projective transformation.
Having just this very simple pattern of a checkerboard and
using the computed transformation a from two vanishing points and two other points,
we can apply them in a very simple application called virtual billboarding.
This is what you see in any sports where the flags
are implanted on the ground or when you see the yellow line in the photo.
In this image of Stephen Colbert, we take the front of the image on the left and
with just this information we can then paint it
on the 3 by 3 aisle we have used for our project information.
Now, remember that in this original checker board,
we had vanishing points because we had parallel lines.
But what happens when we want to compute the projective transformation between
two images?
The image on the left with points a, b, c, d, and
the the image on the right, a primed, b primed, c primed and d primed.
In none of those cases, we have in this image is parallel lines.
We have images of parallel lines, but
there are really no vanishing points of infinity.
So neither a, b, c, d, nor a prime, b prime, c prime,
to d prime, has the 4100, 010 and so on.
What should we do in this case?
Again we introduce this virtual points, 1,0,0, 0,1,0.
And 0,0,1, what we call the four canonical points.
And we take the transformation from these four canonical points
to the left image and we call it t.
So a will be a multiple of t times 1, 0, 0.
B will be a multiple of t times 0, 1, 0 and so on.
We take their and we say that there is a transformation
primed between this canonical, not visible vanishing points,
1, 0,0 multiplied by t prime makes it primed.
0,1,0 multiplied by t prime makes b primed and so on.
So we have this whole formation from this virtual system
to the image on the left and the image on the right.
If we find both, the only thing we can do is to invert one of them,
and this way we can find us a ratio from a to a primed,
which that's the inverse.
This way, just with four point correspondence between
we can find information by using this auxiliary canonical coordinate system.
Explore nosso catálogo
Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.
O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.
© 2018 Coursera Inc. Todos os direitos reservados.
