Skip to content

Reliability profile

pass@k, score spread, layer distribution, and chaos sensitivity.

A single score is a sample. The reliability profile aggregates stored runs for one adapter and task: artifact score mean and standard deviation, pass@k (score above the threshold and a completed exit), the distribution of attributed layers, pass rate per chaos condition, and the trend across the run window.

The leaderboard's stacked bar is that layer distribution. Green is not used for the bar: spec is violet, model blue, harness red, tool amber, environment neutral. A profile with one run is labelled with its run count, not presented as stability.