← All workshopsRead GloVe

Natural Language Processing

Fit the log of a co-occurrence count

BOTIE

Combine the vector interaction with both word biases before measuring error.
Ready to test
0 passed · 0 failed · 1 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Combine the vector interaction with both word biases before measuring error.

Our corpus tells us how often a word and a context occur together. We want the vectors to retain information about those counts. Let's calculate one pair's fitted score and compare it with the logarithm of its observed count. The vectors and biases are supplied for this step.

Inspect the component calculation
Add(
  First contribution: Dot product(
    First vector: Word vector
    Second vector: Context vector
  )
  Second contribution: Add a list(
    Numbers: Word and context biases
  )
)
Examples and results
Word vector
  1. 01
  2. 10
Context vector
  1. 02
  2. 11
Word and context biases
  1. 00.1
  2. 10.2
Required result
2.3
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

A frequent pair with a low fitted score

This check uses the pieces from “Compare a fitted pair with its corpus count”.

BOTIE

Botie finds this word pair together 16 times in the corpus. Combine the word and context vectors with both biases, then measure its squared discrepancy from the log count. This check leaves out GloVe's frequency-weighting factor.
Word vector
  1. 01
  2. 12
Context vector
  1. 02
  2. 1-0.5
Word and context biases
  1. 00.2
  2. 1-0.2
Co-occurrence count
16

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Which parts of the score describe the pair interaction and which describe individual words?

Similar vector operations can learn different representations when their targets differ.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the coursePositional Encoding →
← FastText