Tracer
← polyphony · eval · dracula · ci

polyphony · claude-opus-5-5

✗ failincomplete: ensemble skipped
Pass rate
86%
Cases
8 (1 failed)
Mode
eval
Suite
dracula
Env
ci
Started
2026-10-24 05:21 UTC
Duration
—
Cost
—
Tokens in/out
— / —
Commit
Branch
claude/determined-bell-wo0vsx
Source

Metrics

attribution.accuracy0.633
attribution.chance0.333
attribution.score0.633
continuity.detection_recall1
continuity.false_positive_rate0.500
continuity.score1
extraction.f10.480
extraction.precision0.333
extraction.recall0.857
extraction.score0.480
ingestion.score0.611
outline.score1
prose.score0.932
prose.words3,640
retrieval.mrr0.955
retrieval.precision_at_k0.818
retrieval.score0.818
revise.score0.822

Labels

cadenceon-demand
experimentw15-pre-regrade
harness_shab46d695794a77670e2a34b6d9a4520e1ccb00850
judge_providergroq+anthropic
judge_self_gradedTrue
repeat1
skippedensemble
triggerworkflow_dispatch

Compared with the previous run

No earlier complete run in this series (polyphony · eval · dracula · ci · claude-opus-5-5), so there is no baseline yet.

Cases (8, 1 failed, failures first)

gated 6/7 passed

eval:extractionscore 0.48✗ fail
f1=0.480score=0.480recall=0.857windows=3precision=0.333f1_selected=0.556recall_selected=0.714precision_selected=0.455principal_fields_rate=1principal_mentions_min=19
Output
F1=0.48 P=0.3333 R=0.8571 predicted=['Cora Ellis', 'our young lover', 'Dr. Elias Ward', 'the foreign gentleman', 'Aldous Kerr', "Cora's mother", 'Peter Falk', 'R. M. Renwick', 'Count Vasska', 'Nora Vance', 'Cassidy P. Boone', 'two of the maids', 'Mina Vance', 'Samuel F. Billington & Son', 'Sister Agatha', 'old Mr. Swales', 'Vanderpool', 'my zoöphagous patient']
eval:continuityscore 1.00✓ pass
score=1precision=1n_findings=3n_injected=2quote_verbatim=1detection_recall=1n_control_findings=1false_positive_rate=0.500
Output
detection_recall=1.0 fpr=0.5
eval:ingestionscore 0.61✓ pass
score=0.611grounded=11avg_chunks=9.94n_characters=18
Output
grounded=11/18 avg_chunks=9.94
eval:outlinescore 1.00✓ pass
judge=1score=1n_nodes=6chapters_target=6unknown_rate_canon=0.080
Output
nodes=6 structural_ok=True
⚖︎ Judge rationale

R1: All principal canon characters appear with correct names and act consistently with their canon descriptions. | R6: The outline fully realizes the confirmed premise, with each act’s turn reflected in chapter changes, no static chapters, and correct chapter count. | R3: All planned beats appear in the correct order with their intended changes and no unplanned events.

eval:prosescore 0.93✓ pass
score=0.932words=3,640queued=4deferred=0n_scenes=4r2_lines=0cast_named=1prose_chapter=1r2_applicable=0cast_from_beat=1r1_capped_scenes=0estimate_over_actual=0.705beat_seam_max_overlap=0scene_seam_max_overlap=0
Output
words=3640 path=planned rubrics={'R1': 0.98, 'R2': None, 'R3': 0.975, 'R4': 0.97, 'R5': 0.9} seams=0/0
⚖︎ Judge rationale

scene 1 R1: All principal canon characters required for the beat (Aldous Kerr and Count Vasska) appear correctly with accurate names and behavior, meeting the rubric fully. | scene 1 R3: All planned beats appear in correct order with their intended changes and no extra unplanned events. | scene 1 R4: The scene maintains consistent first‑person‑limited POV, past tense, and a courtly yet chilling tone throughout, with varied, concrete sentences and no structural violations. | scene 1 R5: The prose aligns with canon and planned beats, contains no contradictions, and fulfills the continuity criteria fully. | scene 1 R2: not applicable: under two cast speakers with voice samples have attributed lines | scene 2 R1: All principal characters present as required, correctly named, and act in line with canon; no extraneous or mischaracterized figures. | scene 2 R3: All five planned beats appear in correct order with their intended changes and no unplanned material. | scene 2 R4: The scene maintains consistent Aldous POV and past tense, matches the tone, and is a polished, narrative prose without structural errors. | scene 2 R5: The prose aligns fully with the established canon and prior text, introducing no contradictions and matching the planned beats. | scene 2 R2: not applicable: under two cast speakers with voice samples have attributed lines | scene 3 R1: All principal characters required (Aldous Kerr and Count Vasska) appear correctly with faithful behavior, no missing or miscast figures. | scene 3 R3: The text realizes every planned beat in the correct order, with each described change occurring and no unplanned events added. | scene 3 R4: The passage maintains consistent POV and past tense, matches the intended tone, and reads as polished narrative prose without structural errors. | scene 3 R5: The scene follows the canon, the prior text (untouched food, cold hand, grey dawn, the Count's parting words) and all five planned beats with no real contradictions; the only blemish is the minor 'ring in the dust' where a propped shaving glass stood, which sits oddly with the brief's 'bare spot'. | scene 3 R2: not applicable: under two cast speakers with voice samples have attributed lines | scene 4 R1: The scene uses only Aldous Kerr and the Count, with no invented characters. Aldous behaves as the plan and canon describe, and the Count appears only by reference (eats nothing, no reflection), which matches the beats since none put him on the page; the full name Vasska is never used and Aldous is n | scene 4 R3: All five planned beats appear in order with their stated changes landing (panic-driven exit, locked doors, sheer windows and the prisoner realization, restraint in the chamber, and the resolve to feign contentment), with only minor embellishments such as the barred window, the bird, and the Purfleet | scene 4 R4: The scene keeps close third-person past tense from Aldous's POV, has a claustrophobic Victorian register that matches the prior text, and renders the beats as concrete scene without labels or prefixes; minor slips include a few generic phrases, the repeated 'faint ring in the dust' echoing the prior | scene 4 R5: The scene follows the prior text and planned beats closely, but it has one minor discrepancy: the prior text puts the dust ring where the glass stood on the sill, while the scene moves it to a 'little table by the window'. It also has small object-handling slips, such as the towel and rosary while h | scene 4 R2: not applicable: under two cast speakers with voice samples have attributed lines

eval:retrievalscore 0.82✓ pass
mrr=0.955score=0.818n_queries=11precision_at_k=0.818
Output
precision@k=0.8182 mrr=0.9545
eval:revisescore 0.82✓ pass
score=0.822tighten_n=3n_revisions=5
Output

reported 1 reported

eval:attribution0.63 · reported
n=30empty=0score=0.633chance=0.333correct=19accuracy=0.633mean_gap=0.035generated=30
Output
accuracy=0.6333 chance=0.3333 n=30