A model of the environment
A transition model predicts or describes how a state can change after an action. A model can be given or learned and can be imperfect.
Use it for planning
An agent can evaluate candidate action sequences using predicted rewards and states. In model predictive control, it can plan repeatedly and execute the first action of each plan. Compare the two supplied predictions. A real model-based agent must also learn or specify how actions change future states.
INTERACTIVE EXPLORATION
Compare predicted rewards
Change the predicted reward for A; B stays at 1.
Errors matter
A wrong model can lead to a poor plan. The experiment compares two manually supplied predicted rewards for one step; it does not learn an environment model.
Check your understanding
If an agent believes A yields 2 and B yields 1, which is larger in a one-step comparison?
A, under those predictions.