polyphony · claude-opus-5-5
✗ failincomplete: ensemble skippedMetrics
| attribution.accuracy | 0.533 |
| attribution.chance | 0.333 |
| attribution.score | 0.533 |
| continuity.detection_recall | 1 |
| continuity.false_positive_rate | 0.500 |
| continuity.score | 1 |
| extraction.f1 | 0.480 |
| extraction.precision | 0.333 |
| extraction.recall | 0.857 |
| extraction.score | 0.480 |
| ingestion.score | 0.750 |
| outline.score | 0.500 |
| prose.score | 0.766 |
| prose.words | 3,707 |
| retrieval.mrr | 0.955 |
| retrieval.precision_at_k | 0.818 |
| retrieval.score | 0.818 |
| revise.score | 0.838 |
Labels
| cadence | on-demand |
| experiment | w15-pre-claude |
| harness_sha | a52817cd8a3d5ac5af676829a3c31445fc86d830 |
| judge_provider | groq |
| judge_self_graded | False |
| repeat | 1 |
| skipped | ensemble |
| trigger | workflow_dispatch |
Compared with the previous run
No earlier complete run in this series (polyphony · eval · dracula · ci · claude-opus-5-5), so there is no baseline yet.
Cases (8, 1 failed, failures first)
gated 6/7 passed
F1=0.48 P=0.3333 R=0.8571 predicted=['Cora Ellis', 'our young lover', 'Dr. Elias Ward', 'the foreign gentleman', 'Aldous Kerr', 'Peter Falk', "Cora's mother", 'R. M. Renwick', 'Count Vasska', 'Nora Vance', 'Cassidy P. Boone', 'The coachman', 'Mina Vance', 'Sister Agatha', "Cora's maid", 'Vanderpool', 'old Mr. Swales', 'zoöphagous patient']
detection_recall=1.0 fpr=0.5
grounded=12/16 avg_chunks=10.25
nodes=6 structural_ok=True
R1: All principal canon characters appear with correct names and roles, and no major invented characters or misbehaviors are present. | R6: The outline faithfully follows the confirmed premise, each act’s turn is realized, every chapter shows change, and the chapter count matches the request. | R3: No text was provided to evaluate against the outlined beats, so it fails the rubric entirely.
words=3707 path=planned rubrics={'R1': 1.0, 'R2': 0.1, 'R3': 1.0, 'R4': 0.9625, 'R5': 1.0} seams=0/0scene 1 R1: All principal characters required appear correctly and behave as canon describes, with no deviations. | scene 1 R2: Only half the lines can be confidently matched to the correct character, and the voices lack clear distinguishing diction from the samples. | scene 1 R3: All planned beats appear in correct order with their required changes and no unplanned material. | scene 1 R4: The scene maintains consistent first‑person‑like limited POV in past tense, matches tone, has varied concrete sentences, and contains no structural violations. | scene 1 R5: The prose aligns with canon and planned beats, contains no contradictions, and fulfills the continuity requirements fully. | scene 2 R1: All required canon characters appear correctly and act as described, with no extraneous or mischaracterized figures. | scene 2 R2: The provided lines lack any evidence of distinct voices matching the samples, making characters indistinguishable. | scene 2 R3: The scene includes all five planned beats in the correct order, with each described change occurring as specified and no extra unplanned events. | scene 2 R4: The scene maintains consistent POV and tense, matches the courteous menace tone, and reads as publishable prose with only minor stylistic slips. | scene 2 R5: The scene aligns with the canon and prior text without contradictions, meeting continuity fully. | scene 3 R1: All principal characters required for this beat (Aldous Kerr and Count Vasska) appear correctly and act according to canon, with no missing or mischaracterized figures. | scene 3 R2: No dialogue was provided to assess voice distinctness, so attribution cannot be made. | scene 3 R3: All four planned beats appear in the correct order with their required changes, and no unplanned major events are added. | scene 3 R4: The scene maintains consistent first‑person‑limited past tense POV, matches the tone and rhythm of the brief, and is a polished, publishable‑draft without structural errors. | scene 3 R5: The scene aligns with canon and planned beats, contains no contradictions, and faithfully follows the continuity requirements. | scene 4 R1: All required canon characters (Aldous Kerr and Count Vasska) appear correctly and act as described, with no extraneous or mischaracterized figures. | scene 4 R2: With only a single ambiguous line, the evaluator cannot attribute ≥50% of lines to distinct voices, failing the rubric. | scene 4 R3: All five planned beats appear sequentially, each realized as described, with no extra or out‑of‑order material. | scene 4 R4: The passage maintains consistent POV and past tense, matches the intended tone and style, and is free of structural errors, with only minor occasional phrasing slips. | scene 4 R5: The prose aligns with canon and prior text, containing no contradictions and fulfills all planned beats.
precision@k=0.8182 mrr=0.9545
reported 1 reported
accuracy=0.5333 chance=0.3333 n=30