Prompting, Evaluation and Iteration
Follow the technician’s request
The manual and equipment lookups run together
The maintenance assistant works across tools, but its answer cannot explain a slow request. Botie needs to follow the technician’s own execution through the recorded spans.
BOTIE
The inputs we are comparing
Recorded span list
Lookups happen one after another
[
{
"id": "request",
"parent": null,
"start": 0,
"end": 800,
"trace": "pump-7"
},
{
"id": "manual",
"parent": "request",
"start": 0,
"end": 200,
"trace": "pump-7"
},
{
"id": "equipment",
"parent": "request",
"start": 200,
"end": 500,
"trace": "pump-7"
},
{
"id": "answer",
"parent": "request",
"start": 500,
"end": 800,
"trace": "pump-7"
}
]Two lookups overlap
[
{
"id": "request",
"parent": null,
"start": 0,
"end": 900,
"trace": "pump-7"
},
{
"id": "manual",
"parent": "request",
"start": 50,
"end": 350,
"trace": "pump-7"
},
{
"id": "equipment",
"parent": "request",
"start": 100,
"end": 500,
"trace": "pump-7"
},
{
"id": "answer",
"parent": "request",
"start": 500,
"end": 850,
"trace": "pump-7"
}
]Trace ID to inspect
Same in both cases
pump-7
Result for lookups happen one after another
800
The overnight export begins with someone else’s request
This check uses the pieces from “Measure the interval the technician waited”.
BOTIE
- Trace ID to inspect
pump-7
- Recorded span list
[ { "id": "other", "parent": null, "start": 0, "end": 5000, "trace": "pump-9" }, { "id": "manual", "parent": "request", "start": 240, "end": 700, "trace": "pump-7" }, { "id": "request", "parent": null, "start": 200, "end": 1000, "trace": "pump-7" }, { "id": "equipment", "parent": "request", "start": 260, "end": 850, "trace": "pump-7" } ]
Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.
Build a construction in the workspace above first. This case will test those pieces.
Which connection prevents another technician’s root span from changing your answer, and why can sibling durations overlap?
Preserve request identity and causal context before aggregating timings. A trace is evidence of recorded operations, not a guarantee of answer quality.
These invented complete traces have a single root and a shared clock. Real exports can be sampled, incomplete or affected by clock skew.
Work through the Python challenges →