← All workshopsRead Supervised Fine-Tuning

Adapting and Evaluating Modern Models

Learn from the demonstrated answer

BOTIE

Select the labelled probability from every softmax row and convert it to a negative log loss.
Ready to test
0 passed · 0 failed · 1 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Select the labelled probability from every softmax row and convert it to a negative log loss.

We want a text predictor to learn how to answer instructions. The training sequence contains both the prompt and a demonstrated response, but our objective here should measure its predictions on the response. Let's use the labels and mask to average only those losses.

Inspect the component calculation
Multiply(
  Value: -1
  Multiplier: Natural logarithm(
    Positive values: Read each labelled probability(
      Probability rows: Softmax within each row(
        Rows of scores: Next-token score rows
      )
      Class IDs: Known next-token IDs
    )
  )
)
Examples and results
Next-token score rows
31
13
20
Known next-token IDs
  1. 00
  2. 11
  3. 20
Required result
  1. 00.1269
  2. 10.1269
  3. 20.1269
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

A difficult prompt token is outside the training target

This check uses the pieces from “Average the demonstrated-response losses”.

BOTIE

Botie has three next-token score rows over a three-token vocabulary. The first row belongs to the prompt and the remaining rows belong to the response. Calculate the mean response-only negative log likelihood using the supplied zero-based token IDs.
Next-token score rows
050
000
0.693100
Known next-token IDs
  1. 00
  2. 12
  3. 20
Response-position mask
  1. 00
  2. 11
  3. 21

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

How would including prompt losses change the objective even with the same training text?

A dataset format and its loss mask jointly specify what a fine-tuning run learns from.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseLow-Rank Adaptation →
← Diffusion Models