[MUSIC] In the previous video, we defined the notion of the optimal control problem. We defined of how can we solve the optimal control problem using the dynamic program principle? In this video, we will introduce the non-cooperative differential games and introduce method to solve it or, in our case, define the Nash equilibrium in the game. Let's go back to our example of advertising costs. In this case, we suppose that there is not one company on the market but there are two companies on the market. The revenue of each company depends on the market share of the company, but the problem is that the market capacity is limited. So when the first company tries to increase its market share then the market share of the second company becomes smaller. The only tool that they can use in order to increase their market share or compete on the market is to use the advertisement. So the question is, of what will be the advertising costs or what will be the optimal advertising costs for both players over both companies on this market? Or how can we model their behavior on the market? In order to answer this question we need to define the noncooperative differential games. First thing we need to do is to define the set of players large N which is 1, 2 etc., n. So the number of the players in the game, we need to define the payoff function of player i which is defined in the same way as we define the functional (2) in our previous video. But in here, the functions g and q also depend on the number of the player. So for each player, the functions g and q is different. The state of the game is also defined by the set of differential equations or motion equations. But it is important that in differential game model, the right hand side of the motion equation and the functional itself depends not on one control function u(t,x). But on the set of control functions, or the functions, or the strategies of all the players, so u1(t,x), u2(t,x) etc., un(t,x). So in the beginning of the game, players choose their strategies or they choose of how they are going to behave for each time instant t and for each state of the game x(t) or for each market share. But we will go to that later. The description of the noncooperative differential games for the advertising cost problem is presented on the slide. So the set of players N includes two players, the first and the second company. The payoff function for each player or for each company is defined as the revenue of the company. Of course, it depends on the market share of the company i, xi(t) and on the advertising costs of the company i or ui(t,x). The market share of the first company, x1(t), we'll denote as x(t). And since the market capacity is limited, then we say that the x2(t) or the market share of the second company is actually equal to 1- x(t). So there are only two companies on the market and the sum of their shares is equal to the whole capacity of the market. Then we need to define the motion equation or the system of differential equations that would define the state of the game. And you can see it on the slide below. The right-hand side of the system of differential equations depends on the market share of the company 1, on the market share of company 2 at the current state. And on the advertising costs for the first and the second companies. So the question is, of how can we model the behavior of these two companies or of these players in this game? How can we forecast their behavior? In order to do that, we can use a classical approach or use the Nash equilibrium. The Nash equilibrium is a strategy profile such that for each player i, the individual deviation is not beneficial. So his payoff in Nash equilibrium is more or equal to his payoff when all players but him used the strategies from the Nash equilibrium. And he uses some arbitrary strategy or some arbitrary function ui(t,x). But it is important to notice that when one player, for example player i, deviates from the Nash equilibrium, then trajectory of the system or the function x(t) changes. And below on the slide you can see of how can we calculate this function, for a case when one of the players deviates from the strategy profile? But how can we find Nash equilibrium in this game? We can use a special form of a Bellman equation or a system of Bellman equations. If there exist continuously differentiable functions Vi(t,x), satisfying the following system of Bellman equations. Then the strategies that are obtained by maximizing the right hand side of each of these equations will form a Nash equilibrium. The functions Vi(t,x) or the Bellman functions now are the payoffs of players in Nash equilibrium. So, the Vi(t,x) is a payoff of player i in Nash equilibrium. It is important to notice that in the equation for the player i, the maximization of the right-hand side is done using by the function Vi(t,x), so using his strategy. Given that the other players use strategies from Nash equilibrium. So we define a strategy which is the best to reply on the strategies from the Nash equilibrium, which is indeed a Nash equilibrium. But how can we solve the system of Bellman equations or how can we find the Nash equilibrium for a particular game model? For example, for the advertising costs model. On the slide you can see a system of differential equations with two equations for the player 1 and for the player 2. And the question is how can we solve it? We need to do it in the same way by guessing the form of the Bellman function. And here we will try to find the solution of the Bellman equation, if the form of Bellman function V1(t,x) is exponent^(-r(t))*[A1(t)x + B1(t)]. And the Bellman function for the second player V2(t,x) is equal to exponent^(-r(t))[A2(t)*(1-x) + B2(t)]. And then we can define the strategies that correspond to the Nash equilibrium. But the strategies also depend on the unknown functions A1(t) and A2(t). And the Bellman function also depends on the B1(t) and B2(t). So how can we find these functions? We can also use the same approach that we used for the optimal control problem. We can derive the system of differential equations for the functions A1(t), A2(t), B1(t), and B2(t). By solving these system of differential equations, if the solution exists. Then by solving it, we can substitute these functions into the optimal controls or the equilibrium strategies, then to the Bellman functions. Then using the equilibrium strategies, we can substitute them into the motion equations and derive the trajectory which corresponds to the Nash equilibrium. In our case, we'll do that numerically and the results are presented on the slide. On the right-hand side, you can see the strategies from the Nash equilibrium for the player 1 and for the player 2 along the equilibrium trajectory. So along the trajectory, which is a solution of the motion equations, given that we use a Nash equilibrium. On the left hand side, you can see the corresponding equilibrium trajectory. On this side, you can see a list of references where you could find more information on how to define the noncooperative differential game and how to solve it or how to find the Nash equilibrium using the system of Bellman equations.