Reinforcement Learning: An Interactive Guide

Master RL concepts through visualizations, interactive examples, and hands-on simulations. Based on Sutton & Barto's classic textbook.

See algorithms in action with animated diagrams and real-time simulations.

Play games, adjust parameters, and experiment with different strategies.

All mathematical formulas rendered beautifully with clear explanations.

Chapters

The Reinforcement Learning Problem

Learn the fundamentals of RL: what it is, how it differs from other ML paradigms, and the key elements that make up any RL system.

Evaluative Feedback & Action Selection

Explore the simplest form of RL: the k-armed bandit problem. Learn action-value methods, exploration strategies, and gradient-based approaches.

The MDP Framework

Formalize the RL problem using MDPs. Learn about states, actions, rewards, policies, and value functions.

Planning with a Known Model

Solve MDPs when the environment model is known. Learn policy evaluation, policy improvement, and value iteration.

Learning from Complete Episodes

Learn from experience without a model. Monte Carlo methods average sample returns to estimate value functions.

Bootstrapping Without a Model

Combine the best of MC and DP. TD methods learn from incomplete episodes by bootstrapping from current estimates.

Unifying TD and Monte Carlo

Bridge TD(0) and MC with n-step methods. Control the bias-variance tradeoff by choosing how many steps to look ahead.