← All workshopsRead Is the Model Wrong, or Is the Record Wrong?

Evaluating and Improving Models

Flag the records Botie needs to inspect

BOTIE

Return development records while rejecting duplicate identities and invalid fields.
Ready to test
0 passed · 0 failed · 2 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Return development records while rejecting duplicate identities and invalid fields.

Strong disagreement may identify a mistaken label or a mistaken model.

Inspect the component calculation
Keep development records for review(
  Label review records: Label review records
)
Examples and results
Label review records
[
  {
    "id": "A",
    "split": "development",
    "label": 0,
    "p": 0.95
  },
  {
    "id": "B",
    "split": "development",
    "label": 1,
    "p": 0.8
  }
]
Required result
[
  {
    "id": "A",
    "split": "development",
    "label": 0,
    "p": 0.95
  },
  {
    "id": "B",
    "split": "development",
    "label": 1,
    "p": 0.8
  }
]
Label review records
[
  {
    "id": "C",
    "split": "development",
    "label": 0,
    "p": 0.6
  }
]
Required result
[
  {
    "id": "C",
    "split": "development",
    "label": 0,
    "p": 0.6
  }
]
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This simple rule is not Confident Learning, an automatic correction method or proof that unflagged records are correct. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

An eligible disagreement lands exactly on the boundary

This check uses the pieces from “Request evidence before changing labels”.

BOTIE

The development record has label 0 and score 0.9 at threshold 0.9. A stronger test disagreement follows it. Predict the review IDs.
Label review records
[
  {
    "id": "edge",
    "split": "development",
    "label": 0,
    "p": 0.9
  },
  {
    "id": "test",
    "split": "test",
    "label": 0,
    "p": 1
  }
]
Disagreement threshold
0.9

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

What evidence would justify changing a label rather than retaining it or marking it unresolved?

Preserve the original record, the review reason and the dataset version.

This simple rule is not Confident Learning, an automatic correction method or proof that unflagged records are correct.

Work through the Python challenges →
Continue the courseWhen Should the Model Ask for Help? →
← Who Checks the Model-Choosing Process?