← All workshopsRead Word2vec

Natural Language Processing

Learn from the nearby words

BOTIE

Turn the negative pair's dot product into the probability used by its log-loss term.
Ready to test
0 passed · 0 failed · 1 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Turn the negative pair's dot product into the probability used by its log-loss term.

A useful word vector should help predict which words appear nearby. We have one observed pair and one contrasting pair. Let's build a loss that rewards the first relationship and discourages the second, making the small learning task behind the representation visible.

Inspect the component calculation
Sigmoid(
  Score: Multiply(
    Value: -1
    Multiplier: Dot product(
      First vector: Centre-word vector
      Second vector: Contrasting context vector
    )
  )
)
Examples and results
Centre-word vector
  1. 01
  2. 10
Contrasting context vector
  1. 0-1
  2. 11
Required result
0.7311
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

The contrasting word looks more plausible

This check uses the pieces from “Give the two word pairs opposite targets”.

BOTIE

Botie has an observed word pair and a contrasting pair. These supplied vectors currently favour the contrast. Use both targets to calculate the negative-sampling loss for this one observation.
Centre-word vector
  1. 01
  2. 11
Observed context vector
  1. 01
  2. 1-1
Contrasting context vector
  1. 01
  2. 11

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Why would rewarding only the observed pair leave an incomplete learning problem?

A representation's training objective determines which relationships its coordinates are pressured to retain.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseFastText →
← The Embedding Layer