In this lecture we're going to talk about projective transformations and how can we compute such transformations? What is a projective transformation visually? A projective transformation is any perspective projection of a plane. You can see on the left the front of image of a book cover. The way it appears, for example, in the original website. On the right, you see a pictures of the cover of the book Vision Science, by Stephen Palmer. And, this is a projective transformation from the original front of image. Any such picture which we take from a plane, and corresponds to the original limits, mapped to the image plane, is called a projective transformation. For example, when we have a drone, flying over a planar pattern, like the pattern you see with these codes in the picture. Then we can compute the projective transformation from the robot position to this pattern, and then we can extract the pose of the robot from this planar pattern. So such projective transformations are very useful. Let us, and remember, again, what is a projective transformation? When we have two projective plates, any invertible 3 by 3 matrix transformation is a projective transformation. And because we deal with homogeneous coordinates, we use the simple of the tilde and write Tilde A times p which means that point p is mapped through the projected transformation to the point p prime. Tilde is always in the homogeneous coordinates, it means that lambda p prime equals Ap. The projective transformation is also known as a coordination or Homography. The most important property of a projective transformation is that it preserves incidence. What is incidence? Let's look at these three loop points on the floor. These three loop points are collinear. They're collinear on the original floor. If you put the ruler there, they all lie on the same ruler. If we take a picture from any viewpoint, these three blue points will still remain on the same line on your image plane. This is called preservation of collinearity. Let's take now multiple lines, at least three lines. And in the original floor, these three lines are parallel. They're intersecting at infinity. Then again, in the picture taking from any view point, these three lines will intersect at the same point. This is called preservation of concurrency. So we have seen how, from the beginning, we have seen how points map through a projective transformation, A. A point p maps to a point Ap. If we have a line, l, how is this line transformed? Remember that in the projective plane, we can have an inner product, l transposed p equals zero for the equation of a line. If we replace this p with a inverse p primed, then we get the equation of the line in the transform plane and we see that the coefficients of the map line are a inverse transposed l. How can we compute the projective transformation between two projective planes? One projective planes is the original floor built out of black and white tiles, like a checkerboard. And the other is a pixel plane, where we have just recorded an image from this floor from some view point. The projected transformation is in one to one transformation between the two planes because it is invertible. So if we have these three blue points, they transform to unique points from the left to the right, and from the right to the left. Which means that if we know the projective transformation then we can take any button on the left and paint it on the right and vice versa. Let us see how can we compute this projective transformation. We are going to explain it very visually. Let's say we had a point a, which corresponds to all the vanishing points, which go to the right. We will call them the vanishing point (1,0,0). This corresponds to all the parallel lines that they're parallel to the x axis. Let's have second vanishing point, again on the floor plane, which corresponds, to all the lines vertical to the x axis, which are the planes parallel to the y axis. And we call this vanishing point, (0,1,0). These two points, when I take the picture, are not anymore of infinite. They correspond to the point on the right, in this yellow square, where this parallel lines intersect, vanishing point one and vanishing point two. Let's look at two more points. One is a point where I define the origin of this ground floor, which is the point (0,0,1). And let us see if from these three points we can compute the projected transformation. We assume that we have the mapping A of the point (1,0,0). We have the mapping b of the point (0,1,0) with two vanishing points. And then we have the mapping of the origin (0,0,1), we call c. Then we can write that a is alpha times a. B is beta times b, and c is equal gamma times c. This is very simple, because we just write the fact that all the homogeneous vectors in the projective plane do not change when they multiply by a constant. Now, this trivial fact can be written as a matrix. We have a matrix alpha a, beta b, gamma c, which when multiplied with an identity matrix gives a, b, c. Why the identity matrix? Because the first column of this matrix is the vanishing point (1,0,0) mapping to the point a. The second column is the vector 010, the second vanishing point. And the third is the origin, 001. So we have on the right all the points, the three points we considered on the original plane. And on the left-hand side we have their maps, and we still see that we have three unknowns, alpha, beta, and gamma. So these three points are not sufficient to compute the projective transformation. We will add a fourth point, which is the point 1,1 in that Euclidean plane of the ground floor or written homogeneous coordinates 1, 1, 1. Now we map this point 1, 1, 1 by exactly the same transformation alpha a, beta b and gamma c, into a new point, d. Again because this is a projective transformation, we have to multiply with a factor lambda. So we have an equation lambda d is equal alpha a plus beta b plus gamma c, and we'll try to solve for alpha, beta and gamma. We see that we have four unknowns and only three equation because these vectors are in a three but it doesn't matter. Because if [COUGH] a, b, c are not co-linear, then for every solution alpha, beta, gamma, the solution alpha divided by lambda beta divided by lambda and gamma divided by lambda will always be the same transformation. So we said lambda is equal to one, and we obtain an equation which is the three columns, a, b, c, our known images from the vanishing point (1, 0, 0) (0,1,0) and (0,0,1). And on the right, the mapping of the point, (1,1,1). And we solve for alpha, beta, gamma, a system of three equations and three unknowns. We can solve this because a,b,c are not co-linear, and hence, the determinant of the a,b,c matrix is not zero. After we solve it, the projective transformation that we find, there is just alpha a, beta b and gamma c, just using these three original points a, b and c. What can we do with this transformation? First we see that we didn't need any more points. With four points it was more than enough to recover unambiguously this projective transformation. Having just this very simple pattern of a checkerboard and using the computed transformation a from two vanishing points and two other points, we can apply them in a very simple application called virtual billboarding. This is what you see in any sports where the flags are implanted on the ground or when you see the yellow line in the photo. In this image of Stephen Colbert, we take the front of the image on the left and with just this information we can then paint it on the 3 by 3 aisle we have used for our project information. Now, remember that in this original checker board, we had vanishing points because we had parallel lines. But what happens when we want to compute the projective transformation between two images? The image on the left with points a, b, c, d, and the the image on the right, a primed, b primed, c primed and d primed. In none of those cases, we have in this image is parallel lines. We have images of parallel lines, but there are really no vanishing points of infinity. So neither a, b, c, d, nor a prime, b prime, c prime, to d prime, has the 4100, 010 and so on. What should we do in this case? Again we introduce this virtual points, 1,0,0, 0,1,0. And 0,0,1, what we call the four canonical points. And we take the transformation from these four canonical points to the left image and we call it t. So a will be a multiple of t times 1, 0, 0. B will be a multiple of t times 0, 1, 0 and so on. We take their and we say that there is a transformation primed between this canonical, not visible vanishing points, 1, 0,0 multiplied by t prime makes it primed. 0,1,0 multiplied by t prime makes b primed and so on. So we have this whole formation from this virtual system to the image on the left and the image on the right. If we find both, the only thing we can do is to invert one of them, and this way we can find us a ratio from a to a primed, which that's the inverse. This way, just with four point correspondence between we can find information by using this auxiliary canonical coordinate system.
