Tracer
← polyphony · eval · frankenstein · ci

polyphony · claude-opus-5-5

✗ fail
Pass rate
86%
Cases
8 (1 failed)
Mode
eval
Suite
frankenstein
Env
ci
Started
2026-10-23 08:18 UTC
Duration
—
Cost
—
Tokens in/out
— / —
Commit
e1aba3d
Branch
—
Source
—

Labels

legacy_envci-frankenstein

Compared with the previous run

No earlier complete run in this series (polyphony · eval · frankenstein · ci · claude-opus-5-5), so there is no baseline yet.

Cases (8, 1 failed, failures first)

eval:extractionscore 0.45✗ fail
Output
F1=0.4545 P=0.3333 R=0.7143 predicted=['the being of gigantic stature', 'Emeric Vandel', 'Fenwick', 'Alina', 'The old cottager', 'my dear sister', 'Hollis Crane', 'Cosima', 'Robert (not used)', 'our good Uncle Thomas', "Crane's father", 'The creature (unnamed)', 'The lieutenant', 'The master', "Vandel's father"]
eval:attributionscore 0.60✓ pass
Output
accuracy=0.6 chance=0.3333 n=30
eval:continuityscore 0.50✓ pass
Output
detection_recall=0.5 fpr=0.0
eval:ingestionscore 0.54✓ pass
Output
grounded=7/13 avg_chunks=15.08
eval:outlinescore 0.50✓ pass
Output
nodes=6 structural_ok=True
⚖︎ Judge rationale

R1: All principal canon characters appear in the outlined scenes with correct roles and no deviations from their described behavior. | R6: The outline faithfully restates the premise, each act’s turn is realized, all chapters show change, and the 6‑chapter count meets the request. | R3: No artifact text was provided, so it fails to meet any beat adherence criteria.

eval:prosescore 0.75✓ pass
Output
words=3603 path=planned rubrics={'R1': 1.0, 'R2': 0.05, 'R3': 1.0, 'R4': 0.9625, 'R5': 1.0} seams=0/0
⚖︎ Judge rationale

scene 1 R1: All principal canon characters required for the beat (Rowan Merrow) appear correctly, with only minor invented crew members that do not violate cast fidelity. | scene 1 R2: The untagged lines do not reflect Rowan Merrow's formal, elaborate diction, making voice attribution essentially impossible. | scene 1 R3: All four planned beats appear in order, each change occurs as described, with no extra or missing content. | scene 1 R4: The scene maintains consistent past‑tense POV, matches the restless exalted tone, avoids labels or summaries, and shows varied concrete prose with only minor generic moments. | scene 1 R5: The prose contains no contradictions to the established canon or prior text and fulfills all planned beats without error. | scene 2 R1: All required canon characters for this beat (only Rowan Merrow) appear correctly and act as described. | scene 2 R2: The few untagged lines use terse, colloquial speech that does not reflect Rowan Merrow's elaborate, formal diction, making voice attribution poor. | scene 2 R3: The text realizes every planned beat in the correct order with the required changes and adds no unplanned major events. | scene 2 R4: The scene maintains consistent POV, tense, and tone with varied, concrete prose and no structural violations, though a few minor phrasing issues prevent a perfect score. | scene 2 R5: The passage aligns with the canon and planned beats, contains no contradictions, and there were no planted errors to miss. | scene 3 R1: All principal canon characters required for the scene (Rowan Merrow and Emeric Vandel) appear correctly with proper names and behavior, meeting the rubric fully. | scene 3 R2: No dialogue was provided to assess voice distinctness, so attribution cannot be evaluated. | scene 3 R3: All planned beats appear in correct order with their required changes and no unplanned major events. | scene 3 R4: The scene maintains consistent Rowan POV and past tense, matches tone, and is a polished narrative without structural errors. | scene 3 R5: The passage aligns with the established canon and prior scene, introducing no contradictions and faithfully following the planned beats. | scene 4 R1: All principal characters required appear correctly with proper names and behavior, no extraneous or missing canon figures. | scene 4 R2: No dialogue was provided to evaluate voice distinctness, so attribution cannot be assessed. | scene 4 R3: All five planned beats appear sequentially and fully, with no extra or missing content. | scene 4 R4: The scene maintains consistent POV and tense, matches the intimate solemn tone, and reads as polished narrative prose without structural errors. | scene 4 R5: The passage aligns with the established canon and prior scene, containing no contradictions and fulfilling the planned beats without errors.

eval:retrievalscore 0.81✓ pass
Output
precision@k=0.8148 mrr=0.8333
eval:revisescore 0.94✓ pass
Output