Skip to main content
Glama
Coval MCP Server

Coval MCP Server

Official
by coval-ai

Create report

create_report

Generate an organization-private saved report from specified evaluation run IDs, with optional grouped views and comparison dimensions for reviewing results.

Instructions

Create an organization-private saved report over explicitly identified evaluation runs. Public publishing and arbitrary custom dimensions are intentionally unavailable.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>",
      "run_ids": "<run_ids>"
    },
    "name": "create_report"
  }
}

Input Schema

NameRequiredDescriptionDefault
nameYesName for the saved report.
run_idsYesEvaluation run IDs to include (1-200).
view_modeNoOptional report presentation mode.
compare_byNoOptional dimension used to compare report results.
metadata_keyNoMetadata key used when compare_by is metadata.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed8 schema fields changedv0.5.0
    • addedInput schema / $schema
      Added value: +"http://json-schema.org/draft-07/schema#"
    • addedInput schema / additionalProperties
      Added value: +false
    • addedInput schema / properties / compare_by
      Added value: +{
      +  "description": "Optional dimension used to compare report results.",
      +  "enum": [
      +    "none",
      +    "run",
      +    "agent",
      +    "mutation",
      +    "persona",
      +    "test_case",
      +    "metadata"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / metadata_key
      Added value: +{
      +  "description": "Metadata key used when compare_by is metadata.",
      +  "maxLength": 200,
      +  "minLength": 1,
      +  "type": "string"
      +}
    • addedInput schema / properties / name
      Added value: +{
      +  "description": "Name for the saved report.",
      +  "maxLength": 200,
      +  "minLength": 1,
      +  "type": "string"
      +}
    • addedInput schema / properties / run_ids
      Added value: +{
      +  "description": "Evaluation run IDs to include (1-200).",
      +  "items": {
      +    "maxLength": 128,
      +    "minLength": 1,
      +    "pattern": "^[A-Za-z0-9_-]+$",
      +    "type": "string"
      +  },
      +  "maxItems": 200,
      +  "minItems": 1,
      +  "type": "array"
      +}
    • addedInput schema / properties / view_mode
      Added value: +{
      +  "description": "Optional report presentation mode.",
      +  "enum": [
      +    "rows",
      +    "grouped"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / required
      Added value: +[
      +  "name",
      +  "run_ids"
      +]
  2. Addedv0.4.1

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is a non-read-only, non-destructive, non-idempotent, closed-world mutation. The description adds meaningful behavioral constraints beyond the annotations: the report is organization-private, public publishing is intentionally unavailable, and arbitrary custom dimensions are unsupported. It stops short of explaining return values or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with zero waste. The core action and scope are front-loaded, followed immediately by the key limitations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the essential scope and restrictions for a mutation tool, and annotations plus full schema coverage handle safety and input semantics. It still omits useful context such as what the created report returns and how it relates to sibling tools like create_scheduled_run or list_reports.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the input schema. The description reinforces that runs must be explicitly identified and that custom dimensions are constrained, but it adds little syntax or semantic detail beyond the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description has a specific verb ('Create') and resource ('organization-private saved report') and states the scope ('over explicitly identified evaluation runs'), so the agent can distinguish this from list_reports or get_report. It does not explicitly name or contrast with a sibling such as create_scheduled_run, which keeps it just short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives useful negative guidance: public publishing and arbitrary custom dimensions are unavailable. However, it does not state when to choose create_report over alternatives like create_scheduled_run, nor does it describe required preconditions beyond needing explicit run IDs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.