Skip to main content
Glama
Coval MCP Server

Coval MCP Server

Official
by coval-ai

Consult Sofia

consult_sofia
Read-onlyIdempotent

Delegate read-only Coval evaluation questions to Sofia, who answers using playbooks, runs, conversations, metrics, agents, personas, test sets, and dashboards without changing data.

Instructions

Delegate a read-only Coval evaluation question to Sofia. Sofia can use Coval playbooks and the authenticated organization's runs, simulated conversations, uploaded conversations, metrics, agents, personas, test sets, and dashboards. It cannot create, modify, run, or delete anything.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "prompt": "<prompt>"
    },
    "name": "consult_sofia"
  }
}

Input Schema

NameRequiredDescriptionDefault
promptYesThe question or task to delegate to Sofia.
session_idNoOptional stable caller-session identifier. It must not contain customer secrets.
conversationNoOptional prior turns needed for the current request. Do not include secrets or API keys.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv0.5.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • removedInput schema / properties / conversation / items / additionalProperties
      Removed value: -false
  2. Addedv0.4.1

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true and openWorldHint=false, so the safety profile is covered; the description usefully reinforces the read-only boundary in human terms. What it omits is the behavior an agent actually needs before invoking an LLM-delegation tool: there is no output schema, and the description never says what comes back (a natural-language answer?), nor anything about latency, cost, or whether session_id/conversation are required for multi-turn continuity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler: the first states the action and the read-only constraint, the second enumerates the accessible data domain. The key qualifier ("read-only") is front-loaded before the resource list, so a skimming agent gets the gist immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a delegation-to-another-agent tool with no output schema and no explicit return-value description, the definition leaves the agent guessing about response shape and turn-taking semantics between prompt, session_id, and conversation. The read-only scope and accessible-data list are genuinely complete, but the interaction contract is not.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% — each of the three parameters (prompt, session_id, conversation) is documented inline with length bounds and secret-handling warnings. The description adds no parameter-level meaning beyond that, so the baseline of 3 applies when the schema does all the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and object ("Delegate a read-only Coval evaluation question to Sofia") and then enumerates exactly which resources Sofia may consult (playbooks, runs, simulated/uploaded conversations, metrics, agents, personas, test sets, dashboards). This makes the tool's remit far clearer than the opaque name 'consult_sofia' alone. It stops short of 5 because it never contrasts the umbrella delegation with the many granular read siblings (get_run, list_runs, list_metrics), so an agent cannot tell from the text alone why it would delegate rather than call those directly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"It cannot create, modify, run, or delete anything" implicitly routes the agent away from create_*/update_* siblings toward this tool for read-only questions, which is real guidance. However there is no explicit statement of when to prefer consult_sofia over the numerous read siblings (list_runs, get_metric, get_report), nor any guidance on question shape or size that suits delegation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.