Invisible Hand

Loading...

Do curso por New York University Tandon School of Engineering

Overview of Advanced Methods of Reinforcement Learning in Finance

20 classificações

New York University Tandon School of Engineering

Overview of Advanced Methods of Reinforcement Learning in Finance

20 classificações

Curso 4 de 4 em Specialization Machine Learning and Reinforcement Learning in Finance

In the last course of our specialization, Overview of Advanced Methods of Reinforcement Learning in Finance, we will take a deeper look into topics discussed in our third course, Reinforcement Learning in Finance. In particular, we will talk about links between Reinforcement Learning, option pricing and physics, implications of Inverse Reinforcement Learning for modeling market impact and price dynamics, and perception-action cycles in Reinforcement Learning. Finally, we will overview trending and potential applications of Reinforcement Learning for high frequency trading, cryptocurrencies, peer-to-peer lending, and more.

Na lição

Reinforcement Learning for Optimal Trading and Market Modeling

From Portfolio Optimization to Market Model9:24

Invisible Hand5:32

GBM and Its Problems9:23

The GBM Model: An Unbounded Growth Without Defaults9:09

Dynamics with Saturation: The Verhulst Model7:02

The Singularity is Near9:07

What are Defaults?11:53

Quantum Equilibrium-Disequilibrium11:19

Conheça os instrutores

Igor Halperin

Okay. Now, let's talk about the most interesting application of the framework

that we considered in the last week of our course on Reinforcement Learning.

We mentioned back that the same framework that was

developed there can also be used for modeling not only a policy over single trader,

but also to learn market dynamics.

And in your course project for that course,

I offer it to you to try to calibrate the simple model of

market dynamics that is obtained from this analysis.

And now, I would like to discuss this approach in more details.

So let me start with the notion of an agent in the market.

When people discuss market dynamics in terms of agent-based models,

they typically mean one of the two possible settings.

The first setting views an agent as a so-called representative investor

whose objective is to optimize

a given investment portfolio given some objective function.

A representative investor is some kind of an average investor that has

a certain utility function that is optimized in the process of trading by the agent.

And in this view of the world,

the market is clearly external to the agent.

This approach is quite popular in financial literature.

Another approach to agent-based models considers many interaction agents.

Such models were developed in physics and computer science communities.

These models seek to explain some general stylized facts about your markets,

but what if we go back to

the portfolio optimization problem in a single agent formulation and take

an inverse optimization view of this problem as in Black-Litterman model.

In this approach, the optimal portfolio is already known.

Its market portfolio itself.

But then a good question would be who's an agent that

dynamically maintains or he balances such market optimal portfolio?

And we could probably identify such agent with a some sort of a collective mode of

all individual traders that worked in the market that are guided

in their decisions by a commonly agreed set of predictors that we call Zt,

in our previous course,

that may include news,

other market indicators, or marketing.

These is variables describe in the current state of the Limit Order Book etc.

And therefore, the first difference of our framework from

a conventional utility-based model would be that

our agent is a sum of all investors rather than their average,

that is a representative investor.

And because such agent would aggregate actions of

a partly homogeneous and partly heterogeneous crowd of individual investors,

it cannot be a fully rational agent but rather

should be represented as an agent with bounded rationality.

Bounded rationality is the second key difference

of such approach from a classical agent-based approach.

Therefore, because jointly all individual traders

implement all trades in the market,

then their joined actions actually mount

to dynamically adjusting the market optimal portfolio itself.

And therefore, such agent,

can in some sense,

be identified with a bounded rational component of the market itself.

So, this approach can be implemented and we have a working paper on that and it leads to

a certain type of market dynamics that can be explored and compared with the data.

And the course project that you had actually was about playing with

such a model of market dynamics that is inspired by reinforcement learning.

Now in this week,

we will take another look at these dynamics taking,

this time, a bit more general approach that is

not based directly on reinforcement learning.

And as we will see later,

the more general approach will let us generalize such dynamics and then see

how it can be used to model a plethora of different phenomena in the markets,

and not only to predict market prices.

But to do all this,

I would like to rewind a bit,

almost to the square one,

in quantitative finance and talk first about

one of the most important financial models, namely,

the model known as the Geometric Brownian Motion or the GBM model,

which is also known as the Log-normal model.

This model underlies thousands of other models

and probably millions of different applications,

and this is what makes it specially important to look at and to look at its foundations.

So let's start doing that in our next video.

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2019 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play