← All workshopsRead Pooling a Text

Natural Language Processing

Keep the padding out of the summary

BOTIE

Use the mask to remove padding from the token rows, then average the remaining vectors. Both the sum and the count must exclude padding.
Ready to test
0 passed · 0 failed · 1 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Use the mask to remove padding from the token rows, then average the remaining vectors. Both the sum and the count must exclude padding.

Botie needs a fixed-size text summary, but padding rows are storage placeholders rather than words.

Use the real-token mask before taking coordinate-wise means. Preserve the vector coordinates while reducing the token axis.

Examples and results
Token and padding rows
13
31
00
Real-token mask
  1. 01
  2. 11
  3. 20
Required result
  1. 02
  2. 12
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Padding contains old buffer values

This check uses the pieces from “Average only real token rows”.

BOTIE

Botie receives four stored token rows. The mask marks only the first and third as real tokens; the other rows contain leftover numbers. Return the coordinate-wise mean of the real tokens.
Token and padding rows
2-14
909090
630
-20-20-20
Real-token mask
  1. 01
  2. 10
  3. 21
  4. 30

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Could two texts with the same uncontextualised token vectors in different orders share this summary?

Mean pooling keeps a coordinate average while discarding order unless the input vectors already contain it.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseParagraph Vectors →
← Attention