0:00
Okay. Now, let's talk about the most interesting application of the framework
that we considered in the last week of our course on Reinforcement Learning.
We mentioned back that the same framework that was
developed there can also be used for modeling not only a policy over single trader,
but also to learn market dynamics.
And in your course project for that course,
I offer it to you to try to calibrate the simple model of
market dynamics that is obtained from this analysis.
And now, I would like to discuss this approach in more details.
So let me start with the notion of an agent in the market.
When people discuss market dynamics in terms of agent-based models,
they typically mean one of the two possible settings.
The first setting views an agent as a so-called representative investor
whose objective is to optimize
a given investment portfolio given some objective function.
A representative investor is some kind of an average investor that has
a certain utility function that is optimized in the process of trading by the agent.
And in this view of the world,
the market is clearly external to the agent.
This approach is quite popular in financial literature.
Another approach to agent-based models considers many interaction agents.
Such models were developed in physics and computer science communities.
These models seek to explain some general stylized facts about your markets,
but what if we go back to
the portfolio optimization problem in a single agent formulation and take
an inverse optimization view of this problem as in Black-Litterman model.
In this approach, the optimal portfolio is already known.
Its market portfolio itself.
But then a good question would be who's an agent that
dynamically maintains or he balances such market optimal portfolio?
And we could probably identify such agent with a some sort of a collective mode of
all individual traders that worked in the market that are guided
in their decisions by a commonly agreed set of predictors that we call Zt,
in our previous course,
that may include news,
other market indicators, or marketing.
These is variables describe in the current state of the Limit Order Book etc.
And therefore, the first difference of our framework from
a conventional utility-based model would be that
our agent is a sum of all investors rather than their average,
that is a representative investor.
And because such agent would aggregate actions of
a partly homogeneous and partly heterogeneous crowd of individual investors,
it cannot be a fully rational agent but rather
should be represented as an agent with bounded rationality.
3:44
implement all trades in the market,
then their joined actions actually mount
to dynamically adjusting the market optimal portfolio itself.
And therefore, such agent,
can in some sense,
be identified with a bounded rational component of the market itself.
So, this approach can be implemented and we have a working paper on that and it leads to
a certain type of market dynamics that can be explored and compared with the data.
And the course project that you had actually was about playing with
such a model of market dynamics that is inspired by reinforcement learning.
Now in this week,
we will take another look at these dynamics taking,
this time, a bit more general approach that is
not based directly on reinforcement learning.
And as we will see later,
the more general approach will let us generalize such dynamics and then see
how it can be used to model a plethora of different phenomena in the markets,
and not only to predict market prices.
But to do all this,
I would like to rewind a bit,
almost to the square one,
in quantitative finance and talk first about
one of the most important financial models, namely,
the model known as the Geometric Brownian Motion or the GBM model,
which is also known as the Log-normal model.
This model underlies thousands of other models
and probably millions of different applications,
and this is what makes it specially important to look at and to look at its foundations.
So let's start doing that in our next video.
Igor Halperin
