polyphony · claude-opus-5-5
✗ failLabels
| legacy_env | ci-frankenstein |
Compared with the previous run
No earlier complete run in this series (polyphony · eval · frankenstein · ci · claude-opus-5-5), so there is no baseline yet.
Cases (8, 1 failed, failures first)
F1=0.4545 P=0.3333 R=0.7143 predicted=['the being of gigantic stature', 'Emeric Vandel', 'Fenwick', 'Alina', 'The old cottager', 'my dear sister', 'Hollis Crane', 'Cosima', 'Robert (not used)', 'our good Uncle Thomas', "Crane's father", 'The creature (unnamed)', 'The lieutenant', 'The master', "Vandel's father"]
accuracy=0.6 chance=0.3333 n=30
detection_recall=0.5 fpr=0.0
grounded=7/13 avg_chunks=15.08
nodes=6 structural_ok=True
R1: All principal canon characters appear in the outlined scenes with correct roles and no deviations from their described behavior. | R6: The outline faithfully restates the premise, each act’s turn is realized, all chapters show change, and the 6‑chapter count meets the request. | R3: No artifact text was provided, so it fails to meet any beat adherence criteria.
words=3603 path=planned rubrics={'R1': 1.0, 'R2': 0.05, 'R3': 1.0, 'R4': 0.9625, 'R5': 1.0} seams=0/0scene 1 R1: All principal canon characters required for the beat (Rowan Merrow) appear correctly, with only minor invented crew members that do not violate cast fidelity. | scene 1 R2: The untagged lines do not reflect Rowan Merrow's formal, elaborate diction, making voice attribution essentially impossible. | scene 1 R3: All four planned beats appear in order, each change occurs as described, with no extra or missing content. | scene 1 R4: The scene maintains consistent past‑tense POV, matches the restless exalted tone, avoids labels or summaries, and shows varied concrete prose with only minor generic moments. | scene 1 R5: The prose contains no contradictions to the established canon or prior text and fulfills all planned beats without error. | scene 2 R1: All required canon characters for this beat (only Rowan Merrow) appear correctly and act as described. | scene 2 R2: The few untagged lines use terse, colloquial speech that does not reflect Rowan Merrow's elaborate, formal diction, making voice attribution poor. | scene 2 R3: The text realizes every planned beat in the correct order with the required changes and adds no unplanned major events. | scene 2 R4: The scene maintains consistent POV, tense, and tone with varied, concrete prose and no structural violations, though a few minor phrasing issues prevent a perfect score. | scene 2 R5: The passage aligns with the canon and planned beats, contains no contradictions, and there were no planted errors to miss. | scene 3 R1: All principal canon characters required for the scene (Rowan Merrow and Emeric Vandel) appear correctly with proper names and behavior, meeting the rubric fully. | scene 3 R2: No dialogue was provided to assess voice distinctness, so attribution cannot be evaluated. | scene 3 R3: All planned beats appear in correct order with their required changes and no unplanned major events. | scene 3 R4: The scene maintains consistent Rowan POV and past tense, matches tone, and is a polished narrative without structural errors. | scene 3 R5: The passage aligns with the established canon and prior scene, introducing no contradictions and faithfully following the planned beats. | scene 4 R1: All principal characters required appear correctly with proper names and behavior, no extraneous or missing canon figures. | scene 4 R2: No dialogue was provided to evaluate voice distinctness, so attribution cannot be assessed. | scene 4 R3: All five planned beats appear sequentially and fully, with no extra or missing content. | scene 4 R4: The scene maintains consistent POV and tense, matches the intimate solemn tone, and reads as polished narrative prose without structural errors. | scene 4 R5: The passage aligns with the established canon and prior scene, containing no contradictions and fulfilling the planned beats without errors.
precision@k=0.8148 mrr=0.8333