polyphony · claude-opus-5-5

✗ failincomplete: outline errored; continuity, prose, revise, ensemble, attribution skipped

Pass rate

33%

Cases

3 (2 failed)

Mode

eval

Suite

frankenstein

Env

Started

2026-10-24 00:24 UTC

Duration

—

Cost

—

Tokens in/out

— / —

Commit

2de0544

Branch

main

Source

CI run ↗

Metrics

extraction.f1	0.435
extraction.precision	0.313
extraction.recall	0.714
extraction.score	0.435
ingestion.score	0.467
retrieval.mrr	0.833
retrieval.precision_at_k	0.815
retrieval.score	0.815

Labels

cadence	on-demand
errored	outline
experiment	w15-pre-regrade
harness_sha	2de0544a2a8ca0fe8c7057502e9c7ce20e793d6c
judge_provider	groq
judge_self_graded	False
repeat	1
skipped	continuity,prose,revise,ensemble,attribution
trigger	workflow_dispatch

Compared with the previous run

Baseline 2026-10-23 08:18 UTC · 86% → this run 33% · code diff ↗

✗ newly failing · 1

eval:ingestion

cases added / removed

− eval:attribution
− eval:continuity
− eval:outline
− eval:prose
− eval:revise

Cases (3, 2 failed, failures first)

eval:extractionscore 0.43✗ fail

f1=0.435score=0.435recall=0.714windows=2precision=0.313f1_selected=0.615recall_selected=0.571precision_selected=0.667principal_fields_rate=1principal_mentions_min=4

Output

F1=0.4348 P=0.3125 R=0.7143 predicted=['Emeric Vandel', 'the father of Fenwick', 'the stranger on the sledge', 'Alina', 'my dear sister', 'Hollis Crane', 'my cousin', 'Rowan Merrow', 'Uncle Thomas', "Crane's father", 'The creature (unnamed)', 'The lieutenant', 'The master', "The master's Russian betrothed", "Vandel's father", "Vandel's mother"]

eval:ingestionscore 0.47✗ fail

score=0.467grounded=7avg_chunks=9.93n_characters=15

Output

grounded=7/15 avg_chunks=9.93

eval:retrievalscore 0.81✓ pass

mrr=0.833score=0.815n_queries=9precision_at_k=0.815

Output

precision@k=0.8148 mrr=0.8333