IA-QA — 130+ QA & Dev Tools for AI Agents

response_quality_score

Read-onlyIdempotent

Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length. Returns a 0-100 score over the criteria actually measured — or total_score null with the reason when none is given, because relevance and correctness cannot be read off word overlap (a correct one-word answer shares no words with its question). For meaning against a reference answer, use run_semantic_tests. Also returns unscored signals: question-term overlap, average sentence length, markdown.

Input Schema

TableJSON Schema

Name	Required	Description
`question`	Yes	The original question/prompt
`response`	Yes	The LLM response to score
`max_length`	No	Maximum character length. Scored as a proportional penalty beyond it.
`expected_keywords`	No	Keywords a correct answer must contain (case-insensitive). Scored as coverage.

Output Schema

TableJSON Schema

Name	Required	Description	Default
`grade`	No
`stats`	No
`reason`	No
`signals`	No
`breakdown`	No
`max_score`	No
`total_score`	No
`not_measured`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed5 schema fields changedSep 13, 2026

changedInput schema / properties / expected_keywords / description

Previous value: -"Keywords that should appear in a good answer"New value: +"Keywords a correct answer must contain (case-insensitive). Scored as coverage."

changedInput schema / properties / max_length / description

Previous value: -"Ideal max character length (penalize if exceeded)"New value: +"Maximum character length. Scored as a proportional penalty beyond it."

addedOutput schema / properties / not_measured
```
Added value: +{
+  "type": "array"
+}
```
addedOutput schema / properties / reason
```
Added value: +{
+  "type": "string"
+}
```
addedOutput schema / properties / signals
```
Added value: +{
+  "type": "object"
+}
```

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "breakdown": {
+    "type": "object"
+  },
+  "grade": {},
+  "max_score": {
+    "type": "number"
+  },
+  "stats": {
+    "type": "object"
+  },
+  "total_score": {}
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

A4.6/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), and the description adds substantial non-obvious behavior: the 0-100 scale is computed only over criteria actually measured, total_score is null with a reason when no criteria are given, and unscored signals (question-term overlap, avg sentence length, markdown) are also returned. This is context the annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, followed by return behavior and the sibling pointer. The parenthetical rationale about a one-word answer is a slight digression, but every sentence still carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not document return values, yet it usefully explains the null-score edge case and the extra unscored signals. Combined with the sibling routing and scope limits, an agent has everything needed to call and interpret this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both optional criteria parameters and the required ones are already documented in the schema, including the 'scored as coverage' and 'proportional penalty' semantics. The description reinforces but does not materially extend that meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('score an LLM response') and immediately names the exact criteria it scores against (expected_keywords coverage, max_length compliance). It also explicitly differentiates itself from the run_semantic_tests sibling, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the alternative ('For meaning against a reference answer, use run_semantic_tests') and states the condition that selects it. It also defines scope boundaries by noting that relevance and correctness cannot be measured here, which tells the agent when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All IA-QA — 130+ QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?