start_eval_run
Starts an eval run in the background and returns it queued. Read it with get_eval_run until its status is finished, invalid or error. Each run counts against the organisation's eval runs: 20 a month, or 3 during a trial. Results are informational and do not change scores.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | paired runs a task suite with and without skills; tool_grades grades an MCP server's tool descriptions. | |
| label | No | A label for the run, such as Infra Pack v0.9. Optional. | |
| suite_id | No | The task suite of a paired run: infra-ops-40, appsec-40, agent-sec-40 or data-ops-40. | |
| server_id | No | The MCP server for a tool_grades run. | |
| skill_ids | No | 1 to 20 scanned skills for a paired run. One skill records the result on it. | |
| organisation | No | The organisation's URL name (slug), from list_organisations. Leave it out: a connection signed in through OAuth reaches only the one organisation the person chose at sign-in, and an API key only its own. Only a connection approved before that choice existed, which reaches several organisations, needs it. | |
| idempotency_key | No | Required. Make one new key for each run you mean to start, 1 to 128 letters, digits, _ or -, and reuse the same key when you retry that start. |