Quantitative report
get_quantitative_reportReturns the quantitative analytics for a UT (overall counts, per-task success distribution, duration stats, variant info).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| utId | Yes |
get_quantitative_reportReturns the quantitative analytics for a UT (overall counts, per-task success distribution, duration stats, variant info).
| Name | Required | Description | Default |
|---|---|---|---|
| utId | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation covers the safety profile, so the description only needs to add non-obvious behavioral context. It lists the data categories returned, which is useful, but doesn't mention error handling, output format, or limitations. This is acceptable but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, about 14 words, front-loaded with the main action ('Returns the quantitative analytics') and no extraneous wording. Perfectly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a one-parameter tool with read-only annotation and no output schema, the description covers the core purpose but lacks a description of the return format or detailed data structure. It is adequate for simple invocations but not fully complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema provides only a bare 'utId' string with no description (0% coverage). The description's phrase 'for a UT' implies the parameter is a UT identifier, but it does not explain how to obtain the ID, expected format, or any constraints, leaving the agent to infer the semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Uses specific verb 'Returns' with resource 'quantitative analytics for a UT' and enumerates content categories (overall counts, per-task success distribution, duration stats, variant info). Clearly distinguishes from siblings like get_transcript which would handle qualitative data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description merely states what it returns, leaving the agent to infer when quantitative analytics are needed without any exclusions or comparisons to sibling getters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add one secure layer between your agents and this server.
Every tool targets a distinct resource and action. Even get_ut, which returns comprehensive data, doesn't replace the specialized get_session, get_transcript, or get_quantitative_report for detailed views. No two tools have overlapping purposes.
All names follow a verb_noun pattern with underscores (e.g., add_task, get_ut, list_workspaces). The main inconsistency is using 'create' for UTs (create_ut) but 'add' for child entities (add_use_case, add_task), and having a separate update_scenario instead of folding it into update_ut, but the convention is still predictable.
With 18 tools, the set is slightly above the typical 3-15 well-scoped range, but the domain of usability testing requires distinct operations for drafting (use cases, tasks, scenario), reading results (reports, sessions, transcripts), and comparison. Each tool has a clear role, so the count feels reasonable rather than excessive.
The tool surface covers the core lifecycle: create, read, update, and delete for tasks/use cases, plus comprehensive retrieval of results and notes. Minor gaps include no delete_ut (UTs cannot be removed) and no explicit status change or cloning mechanism, but these are likely intentional app-side actions, so the surface is not severely incomplete.