IA-QA — 152 QA & Dev Tools for AI Agents

test_skill

Read-only

Test a SKILL.md. No model, every call: lint (gate on lint.valid) and review (description lacks what/when, words shared with other_skills, leftovers, no done-check, missing files; why + fix each). Unless lint_only, a model picks a skill per request from names+descriptions: trigger_accuracy PASS ≥80, FIX ≥50, else BLOCK; your scenarios = gate, generated = signal; unanswered = INCONCLUSIVE. Server key 10 runs/day.

Input Schema

Name	Required	Description
`files`	No	Paths in its folder.
`model`	No	Default openai/gpt-oss-20b.
`api_key`	No	Used once, not stored.
`dir_name`	No	Its folder.
`skill_md`	Yes	Full SKILL.md, frontmatter included.
`lint_only`	No
`scenarios`	No	Max 20, near misses included.
`other_skills`	No	Installed next to it.
`scenario_count`	No

Output Schema

Name	Required	Description	Default
`lint`	No
`review`	No
`verdict`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed15 schema fields changedNov 7, 2026

changedInput schema / properties / api_key / description

Previous value: -"Key for the chosen model's provider; not needed for the default."New value: +"Used once, not stored."

changedInput schema / properties / dir_name / description

Previous value: -"Folder holding the SKILL.md; name must match it."New value: +"Its folder."

addedInput schema / properties / files

Added value: +{
+  "description": "Paths in its folder.",
+  "items": {
+    "type": "string"
+  },
+  "type": "array"
+}

removedInput schema / properties / lint_only / description

Removed value: -"Format check only: no model, no key. The CI gate for a SKILL.md."

changedInput schema / properties / model / description

Previous value: -"e.g. gpt-4o-mini. Default openai/gpt-oss-20b (Groq, server key)."New value: +"Default openai/gpt-oss-20b."

changedInput schema / properties / other_skills / description

Previous value: -"The skills installed next to this one. Default: six generic built-ins."New value: +"Installed next to it."

removedInput schema / properties / scenario_count / description

Removed value: -"Requests to generate when scenarios is omitted (default 6)."

changedInput schema / properties / scenarios / description

Previous value: -"Fixed requests, near misses included (should_trigger false). Max 20."New value: +"Max 20, near misses included."

changedInput schema / properties / skill_md / description

Previous value: -"Full content of the SKILL.md file to test, YAML frontmatter included."New value: +"Full SKILL.md, frontmatter included."

addedOutput schema / properties / review
```
Added value: +{
+  "type": "object"
+}
```

removedOutput schema / properties / scenarios

Removed value: -{
-  "items": {
-    "type": "object"
-  },
-  "type": "array"
-}

removedOutput schema / properties / scenarios_source
```
Removed value: -{
-  "type": "string"
-}
```

removedOutput schema / properties / score

Removed value: -{
-  "type": "number"
-}

removedOutput schema / properties / step_adherence
```
Removed value: -{
-  "type": "number"
-}
```
removedOutput schema / properties / trigger_accuracy
```
Removed value: -{
-  "type": "number"
-}
```

Changed8 schema fields changedOct 31, 2026

addedInput schema / examples

Added value: +[
+  {
+    "skill_md": "---\nname: json-formatter\ndescription: Pretty-print a JSON string the user pastes.\n---\nFormat the JSON with 2-space indentation."
+  }
+]

changedInput schema / properties / api_key / description

Previous value: -"API key for the chosen model provider. Not required when using the default Groq model."New value: +"Key for the chosen model's provider; not needed for the default."

changedInput schema / properties / dir_name / description

Previous value: -"Name of the folder holding the SKILL.md. The spec requires name to match it; checked only when given."New value: +"Folder holding the SKILL.md; name must match it."

changedInput schema / properties / lint_only / description

Previous value: -"Run the format check only: no model call, no key, deterministic. The CI gate for a SKILL.md — gate on lint.valid."New value: +"Format check only: no model, no key. The CI gate for a SKILL.md."

changedInput schema / properties / model / description

Previous value: -"LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to openai/gpt-oss-20b (Groq, server key)."New value: +"e.g. gpt-4o-mini. Default openai/gpt-oss-20b (Groq, server key)."

changedInput schema / properties / other_skills / description

Previous value: -"The other skills installed next to this one, [{\"name\", \"description\"}], so triggering is tested against the real neighbours. Default: six built-in generic skills."New value: +"The skills installed next to this one. Default: six generic built-ins."

changedInput schema / properties / scenario_count / description

Previous value: -"Number of requests to generate when scenarios is omitted: half should-load, half near misses. Default: 6."New value: +"Requests to generate when scenarios is omitted (default 6)."

changedInput schema / properties / scenarios / description

Previous value: -"Fixed user requests for a reproducible gate: [{\"prompt\": \"...\", \"should_trigger\": true}]. Include near misses with should_trigger false. Max 20. When omitted, scenario_count requests are generated."New value: +"Fixed requests, near misses included (should_trigger false). Max 20."

Changed1 schema field changedOct 21, 2026

addedInput schema / properties / lint_only

Added value: +{
+  "description": "Run the format check only: no model call, no key, deterministic. The CI gate for a SKILL.md — gate on lint.valid.",
+  "type": "boolean"
+}

Changed7 schema fields changedOct 21, 2026

addedInput schema / properties / dir_name

Added value: +{
+  "description": "Name of the folder holding the SKILL.md. The spec requires name to match it; checked only when given.",
+  "type": "string"
+}

addedInput schema / properties / other_skills

Added value: +{
+  "description": "The other skills installed next to this one, [{\"name\", \"description\"}], so triggering is tested against the real neighbours. Default: six built-in generic skills.",
+  "items": {
+    "properties": {
+      "description": {
+        "type": "string"
+      },
+      "name": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "name",
+      "description"
+    ],
+    "type": "object"
+  },
+  "type": "array"
+}

changedInput schema / properties / scenario_count / description

Previous value: -"Number of test scenarios to generate: half trigger-positive, half trigger-negative. Default: 6."New value: +"Number of requests to generate when scenarios is omitted: half should-load, half near misses. Default: 6."

addedInput schema / properties / scenarios

Added value: +{
+  "description": "Fixed user requests for a reproducible gate: [{\"prompt\": \"...\", \"should_trigger\": true}]. Include near misses with should_trigger false. Max 20. When omitted, scenario_count requests are generated.",
+  "items": {
+    "properties": {
+      "prompt": {
+        "type": "string"
+      },
+      "should_trigger": {
+        "type": "boolean"
+      }
+    },
+    "required": [
+      "prompt",
+      "should_trigger"
+    ],
+    "type": "object"
+  },
+  "type": "array"
+}

changedInput schema / properties / skill_md / description

Previous value: -"Full content of the SKILL.md file to test. Must include a name, a \"Use when:\" trigger description, and at least one step."New value: +"Full content of the SKILL.md file to test, YAML frontmatter included."

addedOutput schema / properties / lint
```
Added value: +{
+  "type": "object"
+}
```
addedOutput schema / properties / scenarios_source
```
Added value: +{
+  "type": "string"
+}
```

Changed1 schema field changedAug 20, 2026

changedInput schema / properties / model / description

Previous value: -"LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to llama-3.3-70b-versatile (Groq, server key)."New value: +"LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to openai/gpt-oss-20b (Groq, server key)."

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "scenarios": {
+    "items": {
+      "type": "object"
+    },
+    "type": "array"
+  },
+  "score": {
+    "type": "number"
+  },
+  "step_adherence": {
+    "type": "number"
+  },
+  "trigger_accuracy": {
+    "type": "number"
+  },
+  "verdict": {
+    "type": "string"
+  }
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

B3.3/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds substantial context beyond them: lint is a hard gate on lint.valid, the model-based phase only runs when lint_only is false, scoring thresholds are PASS ≥80 / FIX ≥50 / else BLOCK, and unanswered scenarios yield INCONCLUSIVE. It also discloses a server-key rate limit of 10 runs/day, which is exactly the kind of constraint annotations cannot express. The one gap is that non-idempotence (idempotentHint=false) is never explained as model nondeterminism.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is compact and roughly front-loaded, with the core operations stated first. But the compression is aggressive to the point of opacity: 'leftovers', 'no done-check', 'why + fix each', and 'your scenarios = gate, generated = signal' are fragments that require re-reading. Length is fine; the telegraphic style costs clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain return values, and it does cover the lint gate, the review checks, the model phase, thresholds, and rate limits — a lot for a 9-parameter tool. What is missing is any usage framing relative to siblings and any clarification of the vague review check names, which leave an agent unsure what 'leftovers' or 'no done-check' actually detect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 78%, so the schema already documents most parameters and the baseline is 3. The description does add real meaning for a few: scenarios act as the gate while generated scenarios are only a signal, other_skills is used for description word-overlap checks, and lint_only short-circuits the model phase. It says nothing about model, api_key, dir_name, files, or scenario_count beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb and resource ('Test a SKILL.md') and then decomposes it into concrete operations: lint, review, and optional model-based trigger scoring. An agent can tell this validates a skill file rather than, say, interviewing its author. However, it never distinguishes itself from the sibling tool 'skill_interview', which plausibly covers adjacent ground.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit 'use this when' statement and no exclusion of alternatives such as skill_interview or run_eval_contract. The only routing signal is implicit: 'Unless lint_only, a model picks a skill...', which describes internal behavior rather than when an agent should choose this tool. The agent must infer usage from the operation list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All IA-QA — 152 QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

test_skill

Input Schema

Output Schema

Schema Changelog

TDQS

Glama MCP Gateway

Resources