← All workshopsRead Ridge & Lasso

Classical Machine Learning

Which coefficient can disappear?

Two coefficients fall inside the penalty

Small fitted coefficients may not survive a lasso penalty. Botie's simplified orthonormal example makes the shrinkage and its zero boundary visible.

BOTIE

With coefficients [0.2, -0.5, 4] and penalty 1, what shrunk coefficient vector should result?
The inputs we are comparing

Unpenalised coefficients

Example 1

  1. 02
  2. 10.5
  3. 2-3

Example 2

  1. 00.2
  2. 1-0.5
  3. 24

Penalty

Same in both cases

1
Result for example 1
  1. 01
  2. 10
  3. 2-2

For a list use brackets, such as [2, 4]. For a matrix put each row in brackets. Text answers can stay as text.

Coefficients exactly at the threshold

This check uses the pieces from “Shrink magnitudes, stop at zero, restore signs”.

BOTIE

Two coefficients have magnitude exactly equal to the penalty, one is smaller, and one is already zero. Predict the soft-thresholded coefficients.
Unpenalised coefficients
  1. 0-2
  2. 12
  3. 20.5
  4. 30
Penalty
2

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

What would go wrong if you subtracted the penalty directly from signed coefficients, and why is this formula restricted to the stated geometry?

A penalty's effect depends on both its form and the optimisation problem; this closed-form lasso rule is a special case.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the coursek-Nearest Neighbours →
← Fitting by Walking