When Should a Probability Become an Action?

Botie's model assigns each greenhouse sensor a probability of failure. An inspection costs time, but missing a real failure also has a cost. A probability alone does not decide which action is preferable.

How large should the failure probability be before Botie inspects the sensor? That cutoff is our decision threshold. To choose it, we need to compare the consequences of inspecting with the consequences of leaving the sensor alone.

A missed failure and an unnecessary inspection need not cost the same. Change one cost and see which supplied failure probabilities now justify acting.

A probability describes an estimated frequency under similar circumstances. An action has consequences. Raising the threshold generally reduces positive decisions; lowering it generally increases them. Neither change requires retraining the score model.

The effect on false alarms and missed failures depends on the labelled data and the score ordering.

A conventional cutoff of one half is a convention, not a universal statement about the best decision.

We will simplify the decision to inspect or leave alone. Correct decisions have zero cost. Inspecting a healthy sensor is a false alarm with cost one. Leaving a failed sensor alone is a miss with cost four.

For a failure probability p, inspection incurs the false-alarm cost only when the sensor is healthy. Leaving it alone incurs the missed-failure cost only when it fails.

Expected inspection cost = (1 - p) × 1
Expected leave-alone cost = p × 4

These expectations assume that the estimated probability is suitable for this population and that the stated cost table describes the decision. Real inspections can have costs even when they are correct, which changes the table.

The two expected costs are equal at the cutoff. Above it, a missed failure becomes more expensive in expectation than a false alarm. We choose inspection at equality as our stated tie rule.

(1 - p) × 1 = p × 4
1 = 5p
p = 0.2
General cutoff = false-alarm cost / (false-alarm cost + miss cost)

The rule uses the relative costs. A larger missed-failure cost lowers the cutoff under these assumptions.

A cost-derived rule still needs evaluation. If we tune a threshold to a validation metric, we must keep the final test separate. Choosing the best cutoff on the final test and reporting that same result gives an optimistic estimate.

Calibration also matters: a score labelled as a probability must be credible for expected-cost arithmetic. A ranking alone can support threshold searches on validation data, but does not justify interpreting every score as a probability.

The workshop builds the cost-derived cutoff and applies it. The Python practice also evaluates a changed cost assumption.

Suppose the probability of a failure is trustworthy, a false inspection costs one unit, and a missed failure costs nine. Inspecting is costly only when the sensor is healthy under this simplified loss table. Not inspecting is costly when it is faulty.

Write those two expected costs and choose the smaller one. This explains why a halfway threshold is not a universal decision rule. Different costs, probability quality or capacity constraints change the decision.

Cost(inspect) = 1 × (1 - p)
Cost(do not inspect) = 9 × p
Inspect when 1 - p < 9p
p > 0.1

If Botie has two inspection slots, he may have to rank sensors and take the first two even when more exceed a cost threshold. The number of inspections is fixed, so the effective score cutoff changes with the incoming batch.

State how ties are handled and whether inspection costs or benefits vary by sensor. A top-k policy answers a different operational question from one fixed probability threshold.

import numpy as np
risk = np.array([0.8, 0.6, 0.6, 0.2])
chosen = np.argsort(-risk, kind='stable')[:2]
print(chosen.tolist())
print((risk > 0.1).tolist())

[0, 1]
[True, True, True, True]