← All workshopsRead Giving Different Weights Different Step Scales

Optimisation and Training

Scale a corrected Adam direction

BOTIE

Return the square root of each corrected second moment plus positive epsilon.
Ready to test
0 passed · 0 failed · 2 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Return the square root of each corrected second moment plus positive epsilon.

Botie already calculated bias-corrected moments and now needs the parameter update.

Inspect the component calculation
Add(
  First contribution: Square root(
    Squared value: Corrected second moments
  )
  Second contribution: Positive epsilon
)
Examples and results
Corrected second moments
  1. 04
  2. 116
Positive epsilon
0
Required result
  1. 02
  2. 14
Corrected second moments
  1. 04
Positive epsilon
0
Required result
  1. 02
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This final stage assumes valid corrected moments. The full optimiser also maintains state and counts updates. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

Equal moment magnitudes with unequal squared histories

This check uses the pieces from “Scale each parameter's corrected direction”.

BOTIE

Apply the supplied corrected moments to all three parameters. The deliberately large epsilon makes its effect visible, including at the coordinate with zero second moment. Return the updated parameter vector.
Parameters
  1. 02
  2. 1-2
  3. 21
Corrected first moments
  1. 00.1
  2. 1-0.1
  3. 20
Corrected second moments
  1. 00.01
  2. 11
  3. 20
Learning rate
0.1
Positive epsilon
0.1

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Which parts of a full Adam optimiser were supplied rather than maintained by your construction?

Keep moment accumulation and bias correction distinct from this final parameter-update stage.

This final stage assumes valid corrected moments. The full optimiser also maintains state and counts updates.

Work through the Python challenges →
Continue the courseChecking a Model Before a Long Training Run →
← Remembering the Direction of Earlier Updates