Optimisation and Training
Check a local derivative
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.
Context for this challenge
Subtract lower loss from upper loss and divide by twice the positive perturbation.
Before a long training run, Botie can compare loss evaluations on either side of a weight. Their difference estimates a local derivative independently of backpropagation.
Subtract the lower-parameter loss from the upper-parameter loss and divide by the full distance between the two parameter settings.
Examples and results
- Loss at w+h
3.61
- Loss at w-h
4.41
- Positive perturbation
0.1
- Required result
-4
- Loss at w+h
4.41
- Loss at w-h
3.61
- Positive perturbation
0.1
- Required result
4
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
Use deterministic evaluations at a smooth point. Finite differences are approximate. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
Checking a quadratic away from its minimum
This check uses the pieces from “Estimate slope from symmetric perturbations”.
BOTIE
- Loss at w+h
0.5625
- Loss at w-h
0.0625
- Positive perturbation
0.25
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Why must both loss evaluations use the same data and deterministic settings?
A gradient check should isolate the parameter perturbation from sampling noise and other changes.
Use deterministic evaluations at a smooth point. Finite differences are approximate.
Work through the Python challenges →