Categories Without an Invented Ranking

Botie wants to include greenhouse roof material in a temperature model. The materials are glass, plastic, and mesh. A model needs usable inputs, so assigning each name an ID seems like a reasonable start.

The IDs tell us which material a row describes. They don't measure anything about the roof. If we multiply them by one shared coefficient, however, their numerical gaps affect the prediction. We need to tell the model which material is present without making an arbitrary choice of IDs look like a measured difference.

A larger category ID would not mean a better or thicker roof. Give each material a separate membership question and inspect the resulting row for each supplied roof.

A temperature difference has units and a physical interpretation. A material ID does not. Renaming glass from ID one to ID seven should not change the properties of the roof.

Some model families handle categorical IDs specially. A plain numerical coefficient does not: it will multiply the number it receives. We therefore need to consider how the model interprets the representation.

Do not conclude that every integer-coded feature is invalid. An ordered category or a categorical-aware algorithm has different assumptions. Our problem concerns unordered names passed to ordinary numerical operations.

Choose a vocabulary, then compare each material with every entry. A matching column receives one and the other columns receive zero. This is called one-hot encoding when exactly one column is active.

Column order: glass, plastic, mesh
glass → [1, 0, 0]
mesh  → [0, 0, 1]

The zeros do not say that a roof has no properties. They say that it is not a member of those particular categories.

The model's coefficients attach to column positions. If we fit with glass first and later move mesh into the first position, the arithmetic still runs while the meaning changes. Fit the vocabulary on training data and preserve its order.

An unfamiliar material needs an explicit policy. We can reserve an unknown category, reject the record for review, or use an all-zero representation with care. This workshop uses all zeros and makes that policy visible.

Vocabulary: [glass, plastic, mesh]
Unknown foil roof → [0, 0, 0]

All zeros cannot identify which unfamiliar material appeared. If one known category is also represented by all zeros after dropping a reference column, unknowns become ambiguous unless handled separately.

Separate indicators let a model learn separate contributions. They do not teach it that two materials have similar insulation or cost. If those measured properties matter, we may want additional features that describe them directly.

With an intercept in a linear model, all category columns together create a redundant description. Dropping a reference column is one way to remove that redundancy. We keep all columns here so the membership mechanism is easy to inspect.

Botie's workshop constructs the indicator table. Later modelling lessons decide how to fit coefficients to it.

Once the indicators exist, a linear model assigns a separate contribution to each column. The intercept supplies a shared starting prediction. Choosing a reference category removes redundant columns, but changes how the remaining coefficients should be read.

Here we retain every indicator and supply illustrative coefficients so that the routing is visible. We are applying a stated rule, not claiming these materials have measured thermal effects.

import numpy as np
vocabulary = np.array(['glass', 'plastic', 'mesh'])
materials = np.array(['mesh', 'foil', 'glass'])
X = (materials[:, None] == vocabulary).astype(float)
weights = np.array([2., -1., -3.])
print((20 + X @ weights).tolist())

[17.0, 20.0, 22.0]

A vocabulary can contain many rare names. Giving each its own coefficient asks the model to learn effects from very few observations. Grouping rare categories into an other category or regularising the model can reduce that instability, with different tradeoffs.

Fit any frequency threshold on training data. Keep unknown and missing categories distinct if their meanings differ. If a category is a near-unique identifier, consider whether the prediction task permits remembering that identity at all, especially when evaluating on new greenhouses.