Reinforcement Learning
Correct a visited action's value
Ignore enormous values after a true terminal
Botie observed one transition and must update its visited action. A true terminal state has no continuation value to borrow.
BOTIE
The inputs we are comparing
Visited entry's old Q value
Continuing transition
2
True terminal
4
Observed reward
Continuing transition
1
True terminal
-2
Next-state action values
Continuing transition
- 04
- 16
True terminal
- 0100
- 1200
True terminal flag: 0 or 1
Continuing transition
0
True terminal
1
Discount
Continuing transition
0.5
True terminal
0.9
Learning rate
Continuing transition
0.25
True terminal
0.5
Result for continuing transition
2.5
For a list use brackets, such as [2, 4]. For a matrix put each row in brackets. Text answers can stay as text.
Every next action currently looks costly
This check uses the pieces from “Respect termination inside the Q target”.
BOTIE
- Visited entry's old Q value
1
- Observed reward
-1
- Next-state action values
- 0-5
- 1-2
- True terminal flag: 0 or 1
0
- Discount
0.5
- Learning rate
0.4
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Why must a true terminal flag differ from merely stopping data collection?
Continuation assumptions belong in the update target and must match the environment's episode semantics.
The terminal flag must be zero or one, and alpha and gamma must lie in [0,1]. This is one tabular update, not a convergence demonstration.
Work through the Python challenges →