Reinforcement Learning
Update the tried schedule's estimated reward
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.
Context for this challenge
Increase the count by one. Divide the difference between reward and old mean by that updated count, then add the correction to the old mean.
Botie tried one schedule and observed its reward. Only that selected action's estimate and count receive evidence.
Increment its trial count, divide reward minus old mean by the updated count, and add the correction to the old mean.
Examples and results
- Selected action's previous mean
7
- Selected action's previous count
1
- New observed reward
3
- Required result
5
- Selected action's previous mean
5
- Selected action's previous count
3
- New observed reward
9
- Required result
6
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
The count must be a non-negative whole number. This estimates a stationary action mean, not a policy for a changing state. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
One disappointing delivery after several good ones
This check uses the pieces from “Update the selected schedule's mean”.
BOTIE
- Selected action's previous mean
6
- Selected action's previous count
4
- New observed reward
-4
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Why should trying one schedule leave another schedule's count unchanged?
An observation informs the action actually tried; estimating its value does not itself decide when to explore.
The count must be a non-negative whole number. This estimates a stationary action mean, not a policy for a changing state.
Work through the Python challenges →