← All workshopsRead Retrieval-Augmented Generation

Adapting and Evaluating Modern Models

Bring the answer its evidence

BOTIE

Carry the nearest vector's index into the aligned document list.
Ready to test
0 passed · 0 failed · 1 not run

An empty floor. Start with the information your machine needs.

Received result

Your answer will arrive here.

Examples, experiments and workspace controls
Live values

Connect a tool's inputs to see what it produces. You can try a branch before connecting it to Result.

Try your own inputs

Use a copy of the example to try your own data. Keep the problem's assumptions in mind, and predict the effect of a change before checking the output. You can also add another branch to compare two ways of doing the calculation.

Start at an output arrow, then click or drag to an input. Drag empty space to pan. Delete removes a selection. Ctrl/Cmd+Z undoes a change.
Context for this challenge

Carry the nearest vector's index into the aligned document list.

A question may need information in a reference document rather than in the model's stored parameters. We have vectors for the question and candidate documents. Let's retrieve the closest document's text and place it beside the question for a later generation step.

Inspect the component calculation
Choose an entry(
  Table or list: Document texts
  Index: Find the smallest(
    Scores: Squared distances(
      Query vector: Question vector
      Candidate rows: Document vectors
    )
  )
)
Examples and results
Document texts
  1. 0The workshop opens at nine.
  2. 1The roof is blue.
Question vector
  1. 01
  2. 10
Document vectors
10
01
Required result
The workshop opens at nine.
Try a fresh construction

This clears this draft, including its undo history. Earlier successful attempts stay recorded. Build again without opening hints to record an independent attempt.

Stuck? Inspect a worked construction

The tools below form one possible solution. You can assemble it, follow its signals, and undo that change to return to your own construction.

Assumptions and test precision

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out. Numerical tests allow a difference of 0.001. Lists and structures must also have the right shape.

Your last tested pieces can start the full construction. An existing draft there keeps its own pieces.

A nearby document answers the return question

This check uses the pieces from “Put the retrieved evidence beside the question”.

BOTIE

A reader asks Botie when borrowed tools must be returned. Find the closest document vector by Euclidean distance, then join the question and the retrieved text in that order, separated by a newline. Retrieval stops before any answer generation.
Question vector
  1. 00.9
  2. 10.1
Document vectors
01
10
-10
Document texts
  1. 0The workshop opens at nine.
  2. 1Borrowed tools must be returned by five.
  3. 2Safety glasses are stored beside the door.
Question text
When must I return the borrowed tools?

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

What would you check in a generated answer after verifying that the correct evidence was retrieved?

Evaluate retrieval and evidence use separately so a fluent answer cannot hide a retrieval failure.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseEvaluating Generative Models →
← Low-Rank Adaptation