Skip to main content
Glama

sumo_qa_register_known_good_test_data

Register or update known-good test data entries in YAML, flagging duplicates by environment, domain, product/SKU, and overlapping scenario tags or validity periods.

Instructions

Add or update an entry in knowledge/test_data/<domain>/known_good.yaml, flagging a duplicate (same environment, domain, product/SKU, and both scenario_tags and known_valid_for overlap). entry is a dict, not a YAML string: id, environment, domain, product_id, sku, scenario_tags, known_valid_for, constraints, owner, last_validated_at, confidence (low / medium / high; default low), source, notes.

Input Schema

NameRequiredDescriptionDefault
entryYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.42.0

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the write/safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=false), so the description's job is to add behavior, and it does: it discloses a duplicate-detection rule keyed on environment, domain, product/SKU plus overlapping scenario_tags and known_valid_for. Gaps remain on what actually happens when a duplicate is flagged (rejected, overwritten, or only reported) and whether the write creates the file when absent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the target file and the duplicate rule before the field list; nothing is padding. The first sentence is dense — the duplicate criterion is parenthetically packed in — but every clause conveys actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with a nested-object parameter and no output schema, the description covers inputs well but leaves the post-call picture undefined: no return value, no duplicate outcome, no error conditions. An agent can call it, but cannot predict what it gets back or how to react to a flagged duplicate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the single parameter is an untyped object with additionalProperties=true, so the description carries the full burden — and it enumerates every expected key plus the confidence enum values and its 'low' default. It does not mark which keys are required versus optional, and gives no type hints for list-like fields such as scenario_tags or known_valid_for.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pair ('Add or update') and resource ('an entry in knowledge/test_data/<domain>/known_good.yaml'), so an agent knows exactly what is written and where. It does not, however, distinguish itself from closely named siblings such as sumo_qa_validate_test_data or sumo_qa_find_test_data, which the description never mentions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the agent must infer that this is the tool for persisting a validated test-data entry into the known-good catalogue. There is no explicit 'use when', no prerequisite (e.g. validate first), and no statement of what to do instead if the entry already exists.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.