Replace the table
A deep Q-network approximates action values with a neural network. It can take a high-dimensional observation and return estimated values for actions.
Stabilize learning
DQN uses mechanisms including experience replay and a target network. These help address correlations and changing targets, but do not guarantee stable learning in every setting. Move Q(A) across Q(B) = 1. The greedy action changes at the crossing; the numbers are supplied estimates, not outputs of a trained network.
INTERACTIVE EXPLORATION
Compare action values
Change the supplied Q estimate for action A. Action B is fixed at 1.
Choosing an action
A greedy choice selects an action with the highest estimated Q value. Exploration may choose differently. The experiment compares supplied scores; it does not train a DQN.
Check your understanding
Does the highest estimated Q always equal the truly best action?
No. Estimates can be inaccurate.