Skip to main content
Glama

list_agent_evaluations

Read-only

Lists Otto and Now Assist evaluation runs for AI agents and skills, showing status, evaluation type, run type, dataset and strategy. Filter by use case, status or run type, and optionally include available eval strategies.

Instructions

List Otto / Now Assist evaluation runs (sys_one_extend_batch_run) for AI agents and skills: status, evaluation type (agentic_ai, conversational_ai), run type, dataset and strategy. Optionally list the available evaluation strategies.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "list_agent_evaluations"
  }
}

Input Schema

NameRequiredDescriptionDefault
limitNoMax records (default 25)
statusNodraft, in_progress, completed, error, cancelled, archived
run_typeNoeval_run, test_run or auto_prompt_tuning
usecase_idNosys_id of the agent / use case / skill under test
evaluation_typeNoagentic_ai or conversational_ai
include_strategiesNoAlso list eval strategies (sys_one_extend_eval_strategy)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv5.0.1

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=true, so the safety profile is covered. The description usefully adds that this reads the sys_one_extend_batch_run table and optionally pulls eval strategies from sys_one_extend_eval_strategy, but says nothing about pagination, permissions, or ordering.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the resource and scope come first, the filterable fields follow, and the optional strategy listing is last. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, six-parameter list tool with no output schema, the description covers the record type, the table, the filterable dimensions, and the optional extra dataset. It could say more about result shape or paging, but nothing critical to calling it is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all six parameters, including the accepted status, run_type, and evaluation_type values. The description mostly restates those fields and adds little syntax or format detail beyond the schema, which is the expected baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: 'List Otto / Now Assist evaluation runs (sys_one_extend_batch_run) for AI agents and skills', backed by named fields (status, evaluation type, run type, dataset, strategy). It does not explicitly route the agent away from near neighbours such as list_ai_agent_executions or get_agent_execution_trace, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the filter list and the closing note that strategies can optionally be listed via include_strategies. There is no explicit statement of when to choose this over execution-listing siblings, and no when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools