Hi everyone. This is Professor Yoongjin Yoon from KAIST. This is second session of second year for the basic math for the beginner of AI, and the part 1 is linear algebra. From last session we studied about the review of matrix and end up with a product of matrix. Product of matrix, we can express by using the summation of the case of law of A and p-th column of B like this. Here, what is the condition for formation of product BA, and what is the order BA if it can be formed? You can start [inaudible], by using the definition of the product of matrix. What is the condition for the formation or product of BA. Matrix B, is P by Q, and matrix A is M by N. From the definition, so number of Q and number of M should be same. The condition is what? Q equals M. Second one, what is the order of BA if it can be formulated? What is that? P by Q times M by N is what? P by N. The order of BA is P by N. Let's study this product of matrix with example. We have three by two matrix P and two by four matrix Q, let's say. P is 1, 2, 3, 4, 5, 6, and matrix Q is 5, 1, 2, 2, 3, 3, 1, 2 like this. Here can we form the PQ or not? We can form here P by Q. Why? The number of columns in P is two and, number of row in Q is two, so it's match each other, so we can form P by Q. However, we cannot form the Q by P because number of column in Q is four, and number of row in P equals three, so it's not matching each other, so we cannot form the QP. Please check what is the condition for the formulation of PQ. This product or matrix is very basic for learning the deep learning. As you see the example of how we use the deep learning for the deep neural network. We have a lot of hidden layer. Each hidden layer represent each hidden matrix. By multiplication, by product of those matrix, we can formulate those hidden layer AI engine. This is very basic and very important. We should know what is the condition when you try to do the product of matrix PQ like this. Here, PQ like this, so P times [inaudible]. For example, let me calculate the P times Q, the product of P times Q, on the right-hand side, there is element show 23. This is third row and second column, element in the third row and second column. How to calculate it? How to calculate, I use third row of P, and second column of Q to calculate this element in P times Q. How to do that? Just 5 times 1 plus 6 times 3, which becomes 23. Multiplication of matrix is not commutative, which means PQ and QP is different, it's not formation, which means even if AB and BA can be formed, AB may or may not be equal to BA, which means multiplication of matrix is not commutative. To satisfy commutative condition, A times B should equals to B times A. But matrix is not. Sometimes can be, but not always. Multiplication of matrix is not commutative. This is very important too. Order or multiplication or matrix is very different because if ordering is changed, the result is changed. Let me explain about identity matrix. Identity matrix is similar to the number one in the number. If you have N by N matrix, C_ij, such that C_11 equals C_22 equals C_33 equal C_NN equals 1, which means placed in the diagonal element. C_ij equals 0. For the i is not equal to j, so it means it's out of diagonal element called an identity matrix. But here identity matrix, you can see only defined in the square matrix. N by N matrix is square matrix. When you have a square matrix, if the element on the diagonal position equals 1 and the other elements are 0, then it means identity matrix. Here are several examples of identity matrix. For the 2 by 2 square matrix, 1 0 0 1 is identity matrix, and you see elements in the diagonal position 1 1 is 1, which is identity matrix, 3 by 3 matrix 1 0 0 0 1 0 0 0 1, this is identity matrix for the 3 by 3 matrix and 4 by 4 matrix too. What's special characteristics about identity matrix? Let's say identity matrix denoted as I and IB on identity matrix and AB on any matrix. If the product IA can be formed, then IA equals A. A similar characteristic with number 1. Similarly, if the product of A times I can be formed, then AI equals A. I times A equals A times I, so this identity matrix can be a commutative. Let me introduce about transpose of a matrix. If A is an M by N matrix, then the transpose of A is the N by M matrix. You can see the difference. A is M by N matrix, then transpose of A is the N by M matrix. I will tell you in the following how to do that. The i-th column of the transpose of A is the i-th row of A. The transpose of A is denoted by A^T. Do you remember when we studied about the support vector machine? There is an equation of the plane in the 2D, 3D, and the N dimensional plane, hyperplane, how we define W when W is a normal vector for the plane and WTX. In that, WT is transpose matrix T. Maybe I'll a little bit discuss, so let me introduce with an example. Let's say matrix A is 3 by 1 matrix, or the three-dimensional vector, A. Transpose of matrix A denoted as A^T, and also equals 1 2 3 ^T, like this. Then this 1 is the element in the first row and first column, so it becomes first row and first column is 1. Then the second element 2 is the second row and first column, so it becomes first row and second column, 2 3 like this. Another example is that, if you have matrix B, the order is 3 times 4, 3 times 4 matrix. Transpose of B is 1 2 3 4 5 6 7 8, like this. e have 12 elements and transpose, it becomes, you can put first row of matrix B become first column of transpose matrix of BT. So one, two, three, four, the first row becomes first column of transpose of B. Second row. Five, six, seven, eight becomes the second column of transpose of B. Last slot, row of the matrix B become the third column of transpose of B. You just change row and column to each other and you can make transpose matrix. Up to here is the short review of the matrix. Now let's start on the linear algebra now. Good starting point is to look at the system of linear algebraic equation. What is the linear algebraic equation? Many simultaneous linear algebraic equations form a system. How can you solve a system of linear algebraic equation? Those are the topics we are going to study about. First on what is a linear algebraic equation? Here we can see the terminology, linear algebraic equation. An example of a linear algebraic equation in one unknown x is what? For example, 2x plus one equals zero. So in the linear algebraic education, or the terms, 2x plus one should be linear here; 2x, one, zero is linear. There is no quadratic explanation. The solution of the above linear algebraic equation is what? X equals minus one over two. A linear algebraic equation in two unknowns, x and y, is an equation of the form like this. For example, 3x minus y equals nine, right? So we have two unknowns, x and y. These is the example of two unknowns, x, y, in a linear algebraic equation. How many solutions here, if we let y equals zero, x equals three? Three, zero can be a solution of above equation. You can find many other case too. For example, if you put the y as a three and x equals what? Become four like that. What is the linear algebraic equation? A linear algebraic equation in N unknowns, which I just introduced. Single unknown and two unknowns. If we want to express with our N unknowns, for example, unknown one is x_1, x_2 to the x_N is an equation of the form, you can form like this, c_1x_1 plus c_2x_2 plus of two the plus c_Nx_N equals d_N. If you have x_1 square x_1 cubed, whatever here, there is a two or wherever some non-linear term, then this is not the linear algebraic equation. When you encounter this N unknowns linear algebraic equation, why bother? Because there will be too many solutions and we don't know how to solve this one. From the next session, we are going to study about how can you solve this very complicated Ns unknown linear algebraic equation system with the help of some computer simulation. Also we will study about the application of the linear algebraic equation. Thank you very much.
