IA-QA — 152 QA & Dev Tools for AI Agents

response_quality_score

Read-onlyIdempotent

Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length. Returns a 0-100 score over the criteria actually measured — or total_score null with the reason when none is given, because relevance and correctness cannot be read off word overlap (a correct one-word answer shares no words with its question). Meaning against a reference answer: run_semantic_tests. Also returns unscored signals: question-term overlap, average sentence length, markdown.

Input Schema

Name	Required	Description
`question`	Yes	The original question/prompt
`response`	Yes	The LLM response to score
`max_length`	No	Maximum character length. Scored as a proportional penalty beyond it.
`expected_keywords`	No	Keywords a correct answer must contain, as whole words (case-insensitive; a plural in -s/-es counts). Scored as coverage.

Output Schema

Name	Required	Description	Default
`grade`	No
`stats`	No
`reason`	No
`signals`	No
`breakdown`	No
`max_score`	No
`total_score`	No
`not_measured`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedOct 31, 2026

changedInput schema / properties / expected_keywords / description

Previous value: -"Keywords a correct answer must contain (case-insensitive). Scored as coverage."New value: +"Keywords a correct answer must contain, as whole words (case-insensitive; a plural in -s/-es counts). Scored as coverage."

Changed1 schema field changedOct 31, 2026

addedInput schema / examples

Added value: +[
+  {
+    "question": "q",
+    "response": "anything"
+  }
+]

Changed5 schema fields changedSep 13, 2026

changedInput schema / properties / expected_keywords / description

Previous value: -"Keywords that should appear in a good answer"New value: +"Keywords a correct answer must contain (case-insensitive). Scored as coverage."

changedInput schema / properties / max_length / description

Previous value: -"Ideal max character length (penalize if exceeded)"New value: +"Maximum character length. Scored as a proportional penalty beyond it."

addedOutput schema / properties / not_measured
```
Added value: +{
+  "type": "array"
+}
```
addedOutput schema / properties / reason
```
Added value: +{
+  "type": "string"
+}
```
addedOutput schema / properties / signals
```
Added value: +{
+  "type": "object"
+}
```

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "breakdown": {
+    "type": "object"
+  },
+  "grade": {},
+  "max_score": {
+    "type": "number"
+  },
+  "stats": {
+    "type": "object"
+  },
+  "total_score": {}
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

Score is being calculated.

All IA-QA — 152 QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.