Compare models
The latest run of each model on one project, mode, suite and env, aligned case by case. Cases the models disagree on are listed first.
Pass rate by model
| Case | ae666ab · 10-26 · ae666ab 100% | pg_kafka · 10-26 · ae666ab 100% |
|---|---|---|
| consume · 1024B | ✓ pass1.00 judge12,299 msgs/sec · p50 0ms · p99 0.03ms | ✓ pass1.00 judge12,299 msgs/sec · p50 0ms · p99 0.03ms |
| end-to-end · 1024B | ✓ pass0.08 judge100 msgs/sec · p50 14.72ms · p99 28.69ms | ✓ pass0.08 judge100 msgs/sec · p50 14.72ms · p99 28.69ms |
| produce · batched(100) · 1024B | ✓ pass1.00 judge6,965 msgs/sec · p50 14.18ms · p99 16.39ms | ✓ pass1.00 judge6,965 msgs/sec · p50 14.18ms · p99 16.39ms |
| produce · batched(100) · shadow-async · 1024B | ✓ pass1.00 judge6,944 msgs/sec · p50 13.47ms · p99 25.72ms | ✓ pass1.00 judge6,944 msgs/sec · p50 13.47ms · p99 25.72ms |
| produce · batched(100) · shadow-sync · 1024B | ✓ pass1.00 judge4,421 msgs/sec · p50 21.41ms · p99 33.32ms | ✓ pass1.00 judge4,421 msgs/sec · p50 21.41ms · p99 33.32ms |
| produce · unbatched · 1024B | ✓ pass0.27 judge331 msgs/sec · p50 2.95ms · p99 5.57ms | ✓ pass0.27 judge331 msgs/sec · p50 2.95ms · p99 5.57ms |
Scope: pg_kafka · benchmark · (no suite) · ci