Submit Tier 0 answers and receive a score report
submit_evalSubmit Tier 0 answers for automatic scoring. Pass the attemptId from start_eval_attempt together with the answers to BOTH gates in one answers array, each keyed by the id exactly as it was served (Gate B ids are opaque per-attempt handles) — that is the only path to a score report, and the attempt is consumed once submitted. Without an attemptId the submission is scored on Gate A alone and nothing is issued. Scoring is deterministic: per-item conformance 0–1 (exact hierarchy match 1.0, adjacent code 0.5), weighted mean per gate. THERE IS NO PASS THRESHOLD: every completed dual-gate attempt yields a signed score report whatever the scores are. The report carries the Gate A and Gate B scores, the per-provision breakdown under the real article names, the measurement conditions, and a descriptive referenceBand saying whether each score falls below, within, or above the range a reference panel reached without being shown the pack — no band is a pass. It also carries a signed margin: the Gate A model-nondeterminism floor, the Gate B draw-noise standard error predicted for that pack (null where the pack has no reference data), and an empirical upper bound from the reference campaign — READ IT BEFORE COMPARING ANY TWO SCORES, because two numbers inside the margin are not meaningfully different, and the figures are provisional pending large-N repeats. Where the pack's reference entry shows adaptation clearing the noise floor on Gate A but not on Gate B, a gateNote describes which gate currently separates models on that pack; it is an observation about the reference panel, not a verdict. The optional conditions object lets the caller declare the maxTokens and temperature it actually used: these are echoed into the signed payload marked selfDeclared, since AIO cannot observe them, and anything not declared is recorded as null rather than defaulted — do not invent values. The report is Ed25519-signed, verifiable with verify_certification, and published to the public registry under the model version and operator recorded on the attempt, so ask the user before calling this. The 0.7 gate figure and 0.5 per-provision figure still appear in the response as reported diagnostics; they gate nothing. A score report is not certification, not a legal conformity assessment, and does not assess organizational or management-system obligations of the reference norm.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| packId | No | Required without an attemptId, e.g. "eu-ai-act". With one, cross-checked against the attempt. | |
| answers | Yes | One entry per item. With an attemptId, both gates go in this one array; ids the attempt did not serve are ignored and reported. Unanswered items score zero. | |
| attemptId | No | The attempt id from start_eval_attempt. Required for a score report; the model, version, operator, and pack recorded on the attempt are authoritative. Omit it to score Gate A only. | |
| modelName | No | Required without an attemptId. With one, optional and cross-checked against the attempt. | |
| conditions | No | Self-declared runner conditions, echoed into the signed report under `conditions.runner` with `selfDeclared: true`. Declare only what the runner actually used — AIO cannot check these, so the signature attests that you stated them, not that they happened. Omit a key you do not know: it is recorded as null (undeclared) and nothing is defaulted in its place. | |
| operatorUrl | No | Optional http(s) URL of the operator. Published in the registry. | |
| modelVersion | No | Required without an attemptId. With one, optional and cross-checked against the attempt. | |
| operatorName | No | Required without an attemptId. The organization accountable for operating the model. | |
| operatorEmail | No | Required without an attemptId. Recorded, never published in the registry. | |
| registrationId | No | Optional id returned by register_for_certification. Recorded but not verified. |