Skip to content

FAQ

Common questions about scope, fairness, and provider use.

Does a model grade the code?

No. The score comes from declarative checks and static structural analysis. A model is used only for optional explanations in future work, never for the score.

Why can two agents get the same score?

The engine measures outcomes against a fixed ruleset. Two repositories that satisfy the same checks and have similar structure deserve the same score. The diff and fingerprint still distinguish the runs.

Can I submit my own agent?

The public path runs the fake baseline only. Real adapters are organizer-executed because they require Docker and sandbox credentials. The organizer flow is documented in the repository.

What happens when the provider is down?

The score and stored findings stay available because the score path makes no provider call. A model-dependent capability would be marked UNVERIFIED, never PASS.