Tracer
← polyphony · eval · dracula · ci

polyphony · claude-opus-5-5

✗ failincomplete: ensemble skipped
Pass rate
86%
Cases
8 (1 failed)
Mode
eval
Suite
dracula
Env
ci
Started
2026-10-23 22:22 UTC
Duration
—
Cost
—
Tokens in/out
— / —
Commit
Branch
main
Source

Metrics

attribution.accuracy0.533
attribution.chance0.333
attribution.score0.533
continuity.detection_recall1
continuity.false_positive_rate0.500
continuity.score1
extraction.f10.480
extraction.precision0.333
extraction.recall0.857
extraction.score0.480
ingestion.score0.750
outline.score0.500
prose.score0.766
prose.words3,707
retrieval.mrr0.955
retrieval.precision_at_k0.818
retrieval.score0.818
revise.score0.838

Labels

cadenceon-demand
experimentw15-pre-claude
harness_shaa52817cd8a3d5ac5af676829a3c31445fc86d830
judge_providergroq
judge_self_gradedFalse
repeat1
skippedensemble
triggerworkflow_dispatch

Compared with the previous run

No earlier complete run in this series (polyphony · eval · dracula · ci · claude-opus-5-5), so there is no baseline yet.

Cases (8, 1 failed, failures first)

gated 6/7 passed

eval:extractionscore 0.48✗ fail
f1=0.480score=0.480recall=0.857windows=3precision=0.333f1_selected=0.556recall_selected=0.714precision_selected=0.455principal_fields_rate=0.667principal_mentions_min=19
Output
F1=0.48 P=0.3333 R=0.8571 predicted=['Cora Ellis', 'our young lover', 'Dr. Elias Ward', 'the foreign gentleman', 'Aldous Kerr', 'Peter Falk', "Cora's mother", 'R. M. Renwick', 'Count Vasska', 'Nora Vance', 'Cassidy P. Boone', 'The coachman', 'Mina Vance', 'Sister Agatha', "Cora's maid", 'Vanderpool', 'old Mr. Swales', 'zoöphagous patient']
eval:continuityscore 1.00✓ pass
score=1precision=1n_findings=3n_injected=2quote_verbatim=1detection_recall=1n_control_findings=1false_positive_rate=0.500
Output
detection_recall=1.0 fpr=0.5
eval:ingestionscore 0.75✓ pass
score=0.750grounded=12avg_chunks=10.25n_characters=16
Output
grounded=12/16 avg_chunks=10.25
eval:outlinescore 0.50✓ pass
judge=0.667score=0.500n_nodes=6chapters_target=6unknown_rate_canon=0
Output
nodes=6 structural_ok=True
⚖︎ Judge rationale

R1: All principal canon characters appear with correct names and roles, and no major invented characters or misbehaviors are present. | R6: The outline faithfully follows the confirmed premise, each act’s turn is realized, every chapter shows change, and the chapter count matches the request. | R3: No text was provided to evaluate against the outlined beats, so it fails the rubric entirely.

eval:prosescore 0.77✓ pass
score=0.766words=3,707queued=4deferred=0n_scenes=4cast_named=1prose_chapter=1cast_from_beat=1r1_capped_scenes=0estimate_over_actual=0.703beat_seam_max_overlap=0scene_seam_max_overlap=0
Output
words=3707 path=planned rubrics={'R1': 1.0, 'R2': 0.1, 'R3': 1.0, 'R4': 0.9625, 'R5': 1.0} seams=0/0
⚖︎ Judge rationale

scene 1 R1: All principal characters required appear correctly and behave as canon describes, with no deviations. | scene 1 R2: Only half the lines can be confidently matched to the correct character, and the voices lack clear distinguishing diction from the samples. | scene 1 R3: All planned beats appear in correct order with their required changes and no unplanned material. | scene 1 R4: The scene maintains consistent first‑person‑like limited POV in past tense, matches tone, has varied concrete sentences, and contains no structural violations. | scene 1 R5: The prose aligns with canon and planned beats, contains no contradictions, and fulfills the continuity requirements fully. | scene 2 R1: All required canon characters appear correctly and act as described, with no extraneous or mischaracterized figures. | scene 2 R2: The provided lines lack any evidence of distinct voices matching the samples, making characters indistinguishable. | scene 2 R3: The scene includes all five planned beats in the correct order, with each described change occurring as specified and no extra unplanned events. | scene 2 R4: The scene maintains consistent POV and tense, matches the courteous menace tone, and reads as publishable prose with only minor stylistic slips. | scene 2 R5: The scene aligns with the canon and prior text without contradictions, meeting continuity fully. | scene 3 R1: All principal characters required for this beat (Aldous Kerr and Count Vasska) appear correctly and act according to canon, with no missing or mischaracterized figures. | scene 3 R2: No dialogue was provided to assess voice distinctness, so attribution cannot be made. | scene 3 R3: All four planned beats appear in the correct order with their required changes, and no unplanned major events are added. | scene 3 R4: The scene maintains consistent first‑person‑limited past tense POV, matches the tone and rhythm of the brief, and is a polished, publishable‑draft without structural errors. | scene 3 R5: The scene aligns with canon and planned beats, contains no contradictions, and faithfully follows the continuity requirements. | scene 4 R1: All required canon characters (Aldous Kerr and Count Vasska) appear correctly and act as described, with no extraneous or mischaracterized figures. | scene 4 R2: With only a single ambiguous line, the evaluator cannot attribute ≥50% of lines to distinct voices, failing the rubric. | scene 4 R3: All five planned beats appear sequentially, each realized as described, with no extra or out‑of‑order material. | scene 4 R4: The passage maintains consistent POV and past tense, matches the intended tone and style, and is free of structural errors, with only minor occasional phrasing slips. | scene 4 R5: The prose aligns with canon and prior text, containing no contradictions and fulfills all planned beats.

eval:retrievalscore 0.82✓ pass
mrr=0.955score=0.818n_queries=11precision_at_k=0.818
Output
precision@k=0.8182 mrr=0.9545
eval:revisescore 0.84✓ pass
score=0.838tighten_n=3n_revisions=4
Output

reported 1 reported

eval:attribution0.53 · reported
n=30empty=0score=0.533chance=0.333correct=16accuracy=0.533mean_gap=0.046generated=30
Output
accuracy=0.5333 chance=0.3333 n=30