Add rewards over a trajectory
For a finite sequence r₁, r₂, r₃, a discounted return is G = r₁ + γr₂ + γ²r₃. Each future term receives another factor of γ. Different tasks may use different horizons and conventions.
Change the discount
When γ = 0, only the first reward contributes to this return. When γ = 1, all three finite-horizon rewards have equal weight. For longer or continuing tasks, the choice and definition of return need care. Compare the three reward contributions as γ changes. Their sum is the return for this fixed trajectory, not an expected value over possible trajectories.
INTERACTIVE EXPLORATION
Discount three rewards
For fixed rewards [1, 2, 4], change γ and see each contribution to the return.
A return is not yet a policy
Return scores an outcome sequence. A value function is an expected return conditioned on a state or state-action pair and a policy. Learning a policy requires additional machinery.
Check your understanding
For rewards [1, 2, 4] with γ = 0.5, what is the return?
1 + 0.5 × 2 + 0.25 × 4 = 3.