Choosing and Checking a Model

Compare the greenhouse population mix

Ten times as many records

Two greenhouse reports can contain different numbers of roofs while describing the same population mix. Botie needs a comparison of shares.

BOTIE

Every category count has grown by the same factor. What distance will remain after normalising both reports?
The inputs we are comparing

Reference category counts

Changed roof mix

  1. 080
  2. 120
  3. 20

Only volume changed

  1. 08
  2. 12
  3. 20

Current category counts

Changed roof mix

  1. 040
  2. 160
  3. 20

Only volume changed

  1. 080
  2. 120
  3. 20
Result for changed roof mix
0.4

For a list use brackets, such as [2, 4]. For a matrix put each row in brackets. Text answers can stay as text.

Two roof categories appear in a new survey

This check uses the pieces from “Compare roof shares on a common scale”.

BOTIE

The old survey contained only the middle roof category. The new survey includes all three. Keep category positions aligned and return the total variation distance between their shares.
Reference category counts
  1. 00
  2. 112
  3. 20
Current category counts
  1. 03
  2. 13
  3. 26

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Could this distance tell you whether prediction accuracy changed? What additional observations would you need?

A population comparison can flag changed inputs; labelled outcomes are needed to measure changed performance.

Both reports must have positive totals and aligned, nonnegative counts. This is a distribution audit, not a performance estimate or an automatic retraining rule.

Work through the Python challenges →
Continue the courseHow Much Does Our Estimate Depend on This Sample? →
← Does Eighty Percent Mean Eighty Percent?