Move 37
Course Objective
This is the syllabus for "Move 37", Siraj Raval's free reinforcement learning course, as part of School of AI. This course can be taken for free on Youtube in the form of a playlist, or at School of AI for a more immersive learning experience. Reinforcement learning is driving some of the latest advances in AI, from DeepMind's AlphaGo to OpenAI's DOTA bots. Although these AIs are designed for video games, reinforcement learning is a powerful branch of AI that can be applied to endless applications in the real world. In this course, we'll cover various RL techniques in order of increasing complexity, applying them to both simulated and real world problems. Students will develop an intuition around when to use certain RL algorithms and by the end of the course will have the practical skills necessary to apply RL to a problem they are passionate about to make a positive impact in the world.
Prerequisites
- Understand Basic Python Syntax
- Understand how the Backpropagation algorithm works.
Components
- Midterm Project
- Final Project
- Educational Videos
- Quizzes
- Reading Assignments
- Coding Assignments
- Interviews
- Group Discussion in Slack
Course Length
- 10 Weeks
- 10-15 hours of dedicated study per week
- Starts September 10 at 12 PM PST
Tools Used
- Pytorch & Tensorflow (Deep Learning Libraries in Python)
- OpenAI Gym (Reinforcement Learning Library)
- Google Colab (for free GPUs, no need to install/configure dependencies)
Week 1 - Introduction
Topics Covered
Markov Decision Processes, Policy Functions, Value Functions, and the Bellman Equation
Week 2 - Dynamic Programming
- Route Planning
- Options Pricing
- Scheduling
- Operating Systems
Topics Covered
Iterative Policy Evaluation, Policy improvement, Policy iteration, Value iteration
Week 3 - Monte Carlo Methods
- Interview #1
- Medical Diagnosis
- Energy Efficiency
- Physics Research
Topics Covered
Monte Carlo prediction, Monte Carlo control, Greedy & Epsilon-Greedy Policies , Exploration vs Exploitation Dilemma
Week 4 - Model-Free Learning
- Delivery Management
- Automated Trading
- Backgammon
- Dopamine in Neuroscience
Topics Covered
Temporal Difference Learning, SARSA, Q-Learning, Model vs Model Free Intuition
Week 5 - Reinforcement Learning in Continuous Spaces
- Self Driving Cars
- Delivery Drones
- Rescue Robots
- Assembly Robots
Topics Covered
Control Theory, Imitation Learning, The Hamilton-Jacobi-Bellman Equation, Kalman Filters
Midterm Project
- Train a bipedal humanoid robot to walk in simulation!
Week 6 - Deep Reinforcement Learning
- Traffic Optimization
- Gaming
- Meta Learning
- Interview #2
Topics Covered
DQN + Double DQN Networks, Dueling DQN, Prioritized Replay, Value-based Methods for Robotics
Week 7 - Policy Based Methods
- Web System Configuration
- Text Summarization
- AI Assisted Design
- Portfolio Optimization
Topics Covered
Evolutionary Algorithms, Stochastic Policy Search, Policy Gradients, REINFORCE
Week 8 - Policy Gradient Methods
- Dialogue Systems
- Photo Editing
- Language Translation
- Tutoring Systems
Topics Covered
Evolved Policy Gradients, Generalized Advantage Estimation (GAE), Trust Region Policy Optimization, Proximal Policy Optimization (PPO)
Week 9 - Actor Critic Methods
- Advanced Trading Techniques
- Human-Machine Cooperation
- Insurance Cost Analysis
- Interview #3
Topics Covered
Actor Critic Algorithms, Asynchronous Advantage Actor Critic, Deep Deterministic Policy Gradients (DDPG), Bayesian Actor-Critic
Week 10 - Multi Agent Reinforcement Learning
- Move 37
- Transportation Networks
- Decentralized Autonomous Organizations
- The Future of AI
Topics Covered
Cooperation, Competiton, Parallelism, Inverse Reinforcement Learning
Final Project
- Develop a multi-agent network to solve a real world problem!