polyphony · claude-opus-5-5
✗ failincomplete: ensemble skippedMetrics
| attribution.accuracy | 0.567 |
| attribution.chance | 0.333 |
| attribution.score | 0.567 |
| continuity.detection_recall | 0.500 |
| continuity.false_positive_rate | 0 |
| continuity.score | 0.500 |
| extraction.f1 | 0.571 |
| extraction.precision | 0.429 |
| extraction.recall | 0.857 |
| extraction.score | 0.571 |
| ingestion.score | 0.429 |
| outline.score | 0.200 |
| prose.score | 0.781 |
| prose.words | 3,564 |
| retrieval.mrr | 0.833 |
| retrieval.precision_at_k | 0.815 |
| retrieval.score | 0.815 |
| revise.score | 0.875 |
Labels
| cadence | on-demand |
| experiment | w15-pre-claude |
| harness_sha | a52817cd8a3d5ac5af676829a3c31445fc86d830 |
| judge_provider | groq |
| judge_self_graded | False |
| repeat | 1 |
| skipped | ensemble |
| trigger | workflow_dispatch |
Compared with the previous run
Baseline 2026-10-23 08:18 UTC · 86% → this run 71% · code diff ↗
- eval:ingestion
- eval:outline
- eval:extraction
- eval:revise: 0.94 → 0.88
Cases (8, 2 failed, failures first)
gated 5/7 passed
grounded=6/14 avg_chunks=6.43
nodes=6 structural_ok=True
R1: Several listed characters (Alina, Cosima Lorenzi, Fenwick, Soraya) do not appear at all, violating cast fidelity. | R6: The outline faithfully follows the confirmed premise, each act’s turn is realized, all chapters show change, and the chapter count matches the request. | R3: No artifact text was provided, so it fails to meet any rubric criteria.
detection_recall=0.5 fpr=0.0
F1=0.5714 P=0.4286 R=0.8571 predicted=['The narrator (the creature)', 'Emeric Vandel', 'Fenwick', 'Alina', "Fenwick's father", 'Isolde Sarrow', 'Hollis Crane', 'Cosima', 'Rowan Merrow', 'Uncle Thomas', "Crane's father", 'The lieutenant', 'The master', "Vandel's father"]
words=3564 path=planned rubrics={'R1': 1.0, 'R2': 0.175, 'R3': 1.0, 'R4': 0.95, 'R5': 1.0} seams=0/0scene 1 R1: All required canon characters (Rowan Merrow, Hollis Crane) appear correctly and act as described, with no extraneous or missing principals. | scene 1 R2: No dialogue provided to assess voice distinctness, so cannot attribute lines to characters. | scene 1 R3: The scene follows all five planned beats in order, realizing each change without adding unplanned events. | scene 1 R4: The passage maintains consistent Rowan POV and present tense, matches tone, and is polished narrative prose without structural errors. | scene 1 R5: The prose aligns with canon and planned beats, contains no contradictions, and would identify all planted errors (none present) without false positives. | scene 2 R1: All principal characters required by the beat (Rowan Merrow and Emeric Vandel) appear with correct names and behavior, and no unauthorized major characters are introduced. | scene 2 R2: No dialogue was provided to evaluate voice distinctness, so attribution cannot be assessed. | scene 2 R3: All five planned beats appear in order, each with the required changes and no extraneous material. | scene 2 R4: The scene maintains consistent POV and tense, matches the intimate foreboding tone, and is polished narrative prose without headings or labels. | scene 2 R5: The passage aligns with the established canon and prior scene, introducing no contradictions and faithfully continuing the storyline. | scene 3 R1: All principal characters (Emeric Vandel and Rowan Merrow) appear correctly and act in line with canon, with only minor supporting mention of Hollis Crane. | scene 3 R2: The lines mostly reflect Emeric's gothic, obsessive voice, allowing about 70‑80% correct attribution, but some generic phrasing prevents reaching the 90% threshold. | scene 3 R3: All five planned beats appear in order, each achieving its intended change with no unplanned additions. | scene 3 R4: The scene maintains consistent POV and tense, matches the dark, obsessive tone, and is well‑crafted prose with only minor slips, meeting the rubric near the top tier. | scene 3 R5: The scene aligns with the established canon and prior text, containing no contradictions and fulfilling the continuity criteria fully. | scene 4 R1: All principal characters required (Emeric Vandel and Rowan Merrow) appear correctly with accurate names and behavior, and no extraneous or mischaracterized cast members are introduced. | scene 4 R2: All lines match Emeric’s voice; none can be attributed to Rowan, so voice distinctness is essentially nonexistent. | scene 4 R3: All five planned beats appear sequentially, each achieving its described change, with no extra unplanned material. | scene 4 R4: The passage maintains consistent POV and tense, captures the intended horror‑guilt tone, and is well‑crafted prose with only minor occasional phrasing issues. | scene 4 R5: The passage aligns fully with the established canon and prior scene, containing no contradictions.
precision@k=0.8148 mrr=0.8333
reported 1 reported
accuracy=0.5667 chance=0.3333 n=30