← All workshopsRead Did the Revision Help on the Same Cases?

Evaluating and Improving Models

Keep repairs and regressions in the same report

BOTIE

Return baseline-minus-candidate loss in the original case order.
Ready to test
0 passed · 0 failed · 2 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Return baseline-minus-candidate loss in the original case order.

Botie needs to compare frozen revisions on shared cases.

Inspect the component calculation
Baseline loss minus candidate loss(
  Baseline losses: Baseline per-case losses
  Candidate losses: Candidate per-case losses
)
Examples and results
Baseline per-case losses
  1. 04
  2. 11
  3. 23
Candidate per-case losses
  1. 02
  2. 12
  3. 22
Required result
  1. 02
  2. 1-1
  3. 21
Baseline per-case losses
  1. 02
  2. 13
Candidate per-case losses
  1. 02
  2. 13
Required result
  1. 00
  2. 10
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This workshop summarises supplied losses. Use the Python exercise and an appropriate sampling design to investigate uncertainty. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

A tie between the models conceals movement

This check uses the pieces from “Count the repairs and regressions”.

BOTIE

Baseline losses [3,1,2] become [1,3,2]. Predict mean gain and each outcome count.
Baseline per-case losses
  1. 03
  2. 11
  3. 22
Candidate per-case losses
  1. 01
  2. 13
  3. 22

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

If the same greenhouse contributes repeated readings, what unit should the resampling preserve?

The uncertainty method must follow the independent unit in the evaluation claim.

This workshop summarises supplied losses. Use the Python exercise and an appropriate sampling design to investigate uncertainty.

Work through the Python challenges →
Continue the courseWho Checks the Model-Choosing Process? →
← Better for Which Cases?