A policy chooses actions
A policy describes how an agent chooses an action. A stochastic policy assigns probabilities to available actions; a deterministic policy selects one action for each state.
Value looks beyond one action
A state-value function Vπ(s) is the expected return starting from state s and following policy π. An action-value function Qπ(s,a) conditions additionally on the first action. Both depend on the environment, policy and definition of return. Change the probability of choosing A. The expected immediate reward is the weighted average of the two action rewards, not a guarantee of the next outcome.
INTERACTIVE EXPLORATION
Mix two actions
Action A gives reward 1 with probability 0.8; B gives reward 1 with probability 0.2. Adjust the chance of choosing A.
Expectation is not a guarantee
In the experiment each action has an illustrative chance of a reward of 1. The policy’s expected immediate reward is a probability-weighted average. Actual outcomes vary, and long-term RL can involve many steps.
Check your understanding
If an agent selects action A with probability 1, which action determines its expected one-step reward?
Only action A contributes; the probability of B is zero.