State and observation
A state describes the relevant condition of an environment in a Markov decision process. An observation is the information available to the agent; it may not fully reveal that state.
Action, transition and reward
A transition model specifies the distribution of the next state given the current state and action. A reward can depend on the transition. Our example is deterministic, but real environments can be stochastic. Choose left and right from the same starting state. Compare the next position and reward; a single transition does not by itself describe a learned policy.
INTERACTIVE EXPLORATION
Choose an action
An agent starts at position 1 on a three-cell track. Move left or right; the rightmost position gives a reward of 1.
Policy versus environment
The policy selects actions from observations or states. The environment determines transitions and rewards. A policy is not the same thing as the transition model, and choosing an action once is not itself learning.
Check your understanding
Does an action always determine the same next state?
No. Transition models may be stochastic; the interactive example is deterministic.