Detecting Patterns a Model Cannot Reconstruct

A model trained to reproduce normal sensor patterns can expose departures through reconstruction error. The error still needs a validated alert rule.

13 sections · 4 questions · 2 coding challenges

1 / 13

The arrow keys move between sections too.

Using Reconstruction as Evidence

BOTIE

Hinton and Salakhutdinov demonstrated learning compact representations with neural networks by reconstructing data. A reconstruction objective teaches what information must survive compression.

Using reconstruction error as an anomaly signal is an additional modelling choice. A network may reconstruct some anomalies well, and a normal condition absent from training may reconstruct poorly.

Sources for this history

Hinton and Salakhutdinov: Reducing the Dimensionality of Data with Neural Networks (2006)

What Does Botie’s Model Fail to Rebuild?

Normal pump operation produces related pressure and flow readings. A reconstruction model learns to reproduce those patterns through a restricted representation.

A new pressure-flow combination may be hard to reconstruct even when each reading looks ordinary on its own. We can measure that reconstruction error and decide when it warrants an alert. Let's keep the predicted reconstruction visible so we can see which relationship the model failed to reproduce.

The reconstructions and fixed threshold are illustrative model outputs.

Step 1 of 3

Prepare comparable features

Scores should not be dominated by arbitrary units.

An invented example for this lesson.
Prepared feature	Prepared input	Reconstruction
Feature 1	1	1
Feature 2	3	1

Feature 1
Prepared input
1
Reconstruction
1
Feature 2
Prepared input
3
Reconstruction
1

Step 2 of 3

Calculate feature errors

Keep both errors visible before reducing.

An invented example for this lesson.
Prepared feature	Squared errors
Feature 1	0
Feature 2	4

Feature 1
Squared errors
0
Feature 2
Squared errors
4

Step 3 of 3

Apply a declared alert rule

Average error and compare with a validation-selected threshold.

Squared errors: [0, 4]
Mean squared error = (0 + 4) / 2 = 2
Alert rule: error > 1.5
2 > 1.5: send an alert

Average the two prepared feature errors before comparing with the declared threshold.

Reconstruction error supplies evidence for review, not a diagnosis

The score aggregates feature mismatch before the threshold turns it into an alert.

Which record is hardest for the model to reproduce?

Compare each prepared record with its reconstruction before looking at the alert threshold. A large error identifies a mismatch, which still needs an explanation.

BOTIE

Botie has prepared readings, their reconstructions, and a fixed threshold.

Choose an example

Prepared records

Reconstructed records

Fixed error threshold

Alert decisions for the reconstructed records

Follow the operations

1. Prepared records
1 3
2 2
2. Reconstructed records
1 1
2 2
3. Subtract
0 2
0 0
4. Square
0 4
0 0
5. Mean within each row
1. 02
2. 10
6. Fixed error threshold
```
1.5
```
7. Strict threshold
1. 01
2. 10
8. Result
1. 01
2. 10

Models and preprocessing must already be fitted. Alerts need investigation and are not fault diagnoses.

1. Fit the Representation on a Reference Set

Train preprocessing and the reconstruction model on suitable reference records. If training is dominated by faults, the model may learn to reproduce them. If it omits a normal operating mode, that mode may look unusual later.

Feature scales matter because squared error mixes coordinate contributions. Fit a scale on training records and retain it for validation and deployment.

Our supplied arrays are already on comparable scales.

3. Select the Alert Rule Separately

A model score is continuous. A threshold turns it into a decision. Choose the threshold using validation data and costs, then assess the final fixed rule on untouched records.

A reference percentile controls an empirical tail fraction in those records. It is not a universal guarantee of future false-alarm rate, particularly under drift.

Use greater-than or greater-than-or-equal deliberately, including at the exact boundary.

Inspect the mismatched features before raising an alert

The reconstruction gets the first feature right and misses the second. Square the feature errors before averaging them into one record score.

Python

import numpy as np
record = np.array([2., 4.])
reconstructed = np.array([2., 2.])
squared_errors = (record - reconstructed) ** 2
print(squared_errors.tolist())
print(float(squared_errors.mean()))

Output

[0.0, 4.0]
2.0

The single score hides which feature caused the error, so inspecting the contributions remains useful. Practice scores several records and applies a fixed alert threshold.

4. Investigate Failure Modes

What happens if the model can copy almost any input? It may reconstruct a faulty case well too. A model with too little capacity has the opposite problem: it may struggle with normal variation. Compare the alert results with simpler robust-score or density methods and inspect them within each operating mode.

Keep the scores and model versions when sending an alert for inspection. If the diagnosis arrives later, those records let us recover why the alert was raised. Until then, it remains a case to investigate rather than a confirmed fault.

The workshop returns an alert mask while preserving intermediate reconstruction errors for inspection.

5. Keep the Residual Vector before Reducing It

A single reconstruction score hides which measurements the model failed to reproduce. Retain the residual for pressure, flow and every other feature before averaging. Scale features deliberately so a large-unit column does not own the result by accident.

The reconstruction may look plausible while one critical feature is badly wrong. An operational alert may therefore need a per-feature rule as well as a total score.

Two equal totals with different failure locations

Columns represent standardised pressure and flow residuals.

Python

import numpy as np
residual = np.array([[2.,0.],[0.,2.]])
print((residual**2).mean(axis=1).tolist())
print(np.argmax(np.abs(residual), axis=1).tolist())

Output

[2.0, 2.0]
[0, 1]

The total scores are equal, but the first example disagrees on pressure and the second on flow. That distinction matters to an inspection.

6. A Flexible Reconstructor Can Reproduce an Anomaly

A model with enough capacity may learn to reconstruct patterns beyond the intended normal regime. Low reconstruction error then does not guarantee normality. Conversely, a harmless new operating mode can reconstruct poorly and trigger false alerts.

Compare with a simple reference model, reserve a validation period for the threshold and inspect labelled faults when available. Do not tune architecture and threshold on the same small collection of known anomalies and call the resulting score an independent test.

7. Training Contamination Changes What Counts as Ordinary

If faulty operation is common in the reference set, a reconstructor may learn it as part of the normal pattern. Inspect reference collection rules and retain a record of exclusions.

When new labels arrive, revisit both the threshold and the fitted representation through a versioned evaluation. Automatically retraining on every alert can teach the model to accept the very faults the system was intended to detect.