In this course, you will learn about several algorithms that can learn near optimal policies based on trial and error interaction with the environment---learning from the agent’s own experience. Learning from actual experience is striking because it requires no prior knowledge of the environment’s dynamics, yet can still attain optimal behavior. We will cover intuitively simple but powerful Monte Carlo methods, and temporal difference learning methods including Q-learning. We will wrap up this course investigating how we can get the best of both worlds: algorithms that can combine model-based planning (similar to dynamic programming) and temporal difference updates to radically accelerate learning.
Este curso faz parte do Programa de cursos integrados Reforço de aprendizagem
Sample-based Learning Methods
oferecido por
Universidade de AlbertaUniversidade de Alberta
Alberta Machine Intelligence Institute
Informações sobre o curso
80,899 visualizações recentes
Habilidades que você terá
Artificial Intelligence (AI)Machine LearningReinforcement LearningFunction ApproximationIntelligent Systems
100% on-line
Comece imediatamente e aprenda em seu próprio cronograma.
Curso 2 de 4 no
Prazos flexíveis
Redefinir os prazos de acordo com sua programação.
Nível intermediário
Probabilities & Expectations, basic linear algebra, basic calculus, Python 3.0 (at least 1 year), implementing algorithms from pseudocode
Aprox. 18 horas para completar
Sugerido: 4-6 hours/week
Inglês
Legendas: Inglês
Programa - O que você aprenderá com este curso
1 hora para concluir
Welcome to the Course!
Welcome to the second course in the Reinforcement Learning Specialization: Sample-Based Learning Methods, brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, and get a flavour of what the course has in store for you. Make sure to introduce yourself to your classmates in the "Meet and Greet" section!
1 hora para concluir
2 vídeos (Total 10 mín.), 2 leituras
2 leituras
Reinforcement Learning Textbook10min
Read Me: Pre-requisites and Learning Objectives10min
4 horas para concluir
Monte Carlo Methods for Prediction & Control
This week you will learn how to estimate value functions and optimal policies, using only sampled experience from the environment. This module represents our first step toward incremental learning methods that learn from the agent’s own interaction with the world, rather than a model of the world. You will learn about on-policy and off-policy methods for prediction and control, using Monte Carlo methods---methods that use sampled returns. You will also be reintroduced to the exploration problem, but more generally in RL, beyond bandits.
4 horas para concluir
11 vídeos (Total 58 mín.), 2 leituras, 1 teste
11 videos
Using Monte Carlo for Prediction6min
Using Monte Carlo for Action Values2min
Using Monte Carlo methods for generalized policy iteration2min
Solving the Blackjack Example3min
Epsilon-soft policies5min
Why does off-policy learning matter?4min
Importance Sampling4min
Off-Policy Monte Carlo Prediction5min
Emma Brunskill: Batch Reinforcement Learning12min
Week 1 Summary3min
2 leituras
Weekly Reading40min
Chapter Summary40min
1 exercício prático
Graded Quiz30min
6 horas para concluir
Temporal Difference Learning Methods for Prediction
This week, you will learn about one of the most fundamental concepts in reinforcement learning: temporal difference (TD) learning. TD learning combines some of the features of both Monte Carlo and Dynamic Programming (DP) methods. TD methods are similar to Monte Carlo methods in that they can learn from the agent’s interaction with the world, and do not require knowledge of the model. TD methods are similar to DP methods in that they bootstrap, and thus can learn online---no waiting until the end of an episode. You will see how TD can learn more efficiently than Monte Carlo, due to bootstrapping. For this module, we first focus on TD for prediction, and discuss TD for control in the next module. This week, you will implement TD to estimate the value function for a fixed policy, in a simulated domain.
6 horas para concluir
6 vídeos (Total 37 mín.), 1 leitura, 2 testes
6 videos
Rich Sutton: The Importance of TD Learning6min
The advantages of temporal difference learning5min
Comparing TD and Monte Carlo5min
Andy Barto and Rich Sutton: More on the History of RL12min
Week 2 Summary2min
1 leituras
Weekly Reading40min
1 exercício prático
Practice Quiz30min
8 horas para concluir
Temporal Difference Learning Methods for Control
This week, you will learn about using temporal difference learning for control, as a generalized policy iteration strategy. You will see three different algorithms based on bootstrapping and Bellman equations for control: Sarsa, Q-learning and Expected Sarsa. You will see some of the differences between the methods for on-policy and off-policy control, and that Expected Sarsa is a unified algorithm for both. You will implement Expected Sarsa and Q-learning, on Cliff World.
8 horas para concluir
9 vídeos (Total 30 mín.), 2 leituras, 2 testes
9 videos
Sarsa in the Windy Grid World3min
What is Q-learning?3min
Q-learning in the Windy Grid World3min
How is Q-learning off-policy?4min
Expected Sarsa3min
Expected Sarsa in the Cliff World3min
Generality of Expected Sarsa1min
Week 3 Summary2min
2 leituras
Weekly Reading40min
Chapter summary40min
1 exercício prático
Practice Quiz30min
Avaliações
Principais avaliações do SAMPLE-BASED LEARNING METHODS
por KMJan 9, 2020
Really great resource to follow along the RL Book. IMP Suggestion: Do not skip the reading assignments, they are really helpful and following the videos and assignments becomes easy.
por KNOct 2, 2019
Great course! The notebooks are a perfect level of difficulty for someone learning RL for the first time. Thanks Martha and Adam for all your work on this!! Great content!!
por UZNov 22, 2019
Good balance of theory and programming assignments. I really like the weekly bonus videos with professors and developers. Recommend to everyone.
por IFSep 28, 2019
Great course. Clear, concise, practical. Right amount of programming. Right amount of tests of conceptual knowledge. Almost perfect course.
por TGMar 19, 2020
Pretty clear explanations! Nice starting point if you want to deep dive into RL. It gives clear picture over some confusing terms in RL.
por KOJan 6, 2020
This course is well explained, easy to follow and made me understand much better the tabular RL methods. I liked it very much.
por NGNov 24, 2019
Excellent course companion to the textbook, clarifies many of the vague topics and gives good tests to ensure understanding
por NSFeb 1, 2020
I like the teaching style the emphasis on understanding and the fruitful combination with the textbook. Highly recommended!
oferecido por
Universidade de AlbertaUniversidade de Alberta
UAlberta is considered among the world’s leading public research- and teaching-intensive universities. As one of Canada’s top universities, we’re known for excellence across the humanities, sciences, creative arts, business, engineering and health sciences.
Alberta Machine Intelligence Institute
The Alberta Machine Intelligence Institute (Amii) is home to some of the world’s top talent in machine intelligence. We’re an Alberta-based
Sobre Programa de cursos integrados Reforço de aprendizagem
The Reinforcement Learning Specialization consists of 4 courses exploring the power of adaptive learning systems and artificial intelligence (AI).
Harnessing the full potential of artificial intelligence requires adaptive learning systems. Learn how Reinforcement Learning (RL) solutions help solve real-world problems through trial-and-error interaction by implementing a complete RL solution from beginning to end.
By the end of this Specialization, learners will understand the foundations of much of modern probabilistic artificial intelligence (AI) and be prepared to take more advanced courses or to apply AI tools and ideas to real-world problems. This content will focus on “small-scale” problems in order to understand the foundations of Reinforcement Learning, as taught by world-renowned experts at the University of Alberta, Faculty of Science.
The tools learned in this Specialization can be applied to game development (AI), customer interaction (how a website interacts with customers), smart assistants, recommender systems, supply chain, industrial control, finance, oil & gas pipelines, industrial control systems, and more.
Perguntas Frequentes – FAQ
Quando terei acesso às palestras e às tarefas?
Ao se inscrever para um Certificado, você terá acesso a todos os vídeos, testes e tarefas de programação (se aplicável). Tarefas avaliadas pelos colegas apenas podem ser enviadas e avaliadas após o início da sessão. Caso escolha explorar o curso sem adquiri-lo, talvez você não consiga acessar certas tarefas.
O que recebo ao me inscrever nesta Especialização?
Quando você se inscreve no curso, tem acesso a todos os cursos na Especialização e pode obter um certificado quando concluir o trabalho. Seu Certificado eletrônico será adicionado à sua página de Participações e você poderá imprimi-lo ou adicioná-lo ao seu perfil no LinkedIn. Se quiser apenas ler e assistir o conteúdo do curso, você poderá frequentá-lo como ouvinte sem custo.
Qual é a política de reembolso?
Existe algum auxílio financeiro disponível?
Mais dúvidas? Visite o Central de Ajuda ao Aprendiz.
