Choosing and Checking a Model
Choose a setting without peeking
BOTIE
Examples, experiments and workspace controls
Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.
Try your own inputs
Give one candidate an excellent result on one fold and poor results on the others. Will it still win? Inspect each candidate's mean before looking at the selected setting.
Context for this challenge
Each candidate has one validation loss per fold. Average within each candidate, choose the lowest mean, then retrieve that candidate's setting. The final test set is absent from this decision.
Several settings have losses from the same validation folds. Botie must choose one using comparable averages while leaving the final test untouched.
Average each candidate's loss row, find the smallest mean's position and retrieve the setting at that position.
Examples and results
- Validation losses by candidate
3 4 5 1 3 2 5 1 6 - Candidate settings
- 00.1
- 10.5
- 21
- Required result
0.5
Try a fresh construction
This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.
Stuck? Inspect a worked construction
The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.
Assumptions and test precision
This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.
A candidate wins one fold but loses the average
This check uses the pieces from “Select the setting with the lowest mean fold loss”.
BOTIE
- Validation losses by candidate
2 8 4 4 3 6 - Candidate settings
- 02
- 15
- 29
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Why must the same final test remain outside this choice, and why does winning these folds not guarantee the best future setting?
Separate tuning evidence from final evaluation, and keep candidate identity aligned with its aggregate score.
This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.
Work through the Python challenges →