IA-QA — 152 QA & Dev Tools for AI Agents

similarity_score

Read-onlyIdempotent

Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.

Input Schema

Name	Required	Description
`batch`	No	Batch mode: array of {reference, hypothesis} pairs.
`metrics`	No	Metrics to compute (default: all). Options: "cosine_bow", "cosine_tfidf", "jaccard", "rouge1", "rouge2", "rougeL", "bleu"
`reference`	No	Reference / expected text (ground truth)
`threshold`	No	Optional pass/fail threshold (0-1). Applies to ROUGE-L F1 score.
`hypothesis`	No	Hypothesis / actual text (LLM output)

Output Schema

Name	Required	Description	Default
`f1`	No
`mode`	No
`count`	No
`recall`	No
`results`	No
`precision`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedOct 31, 2026

addedInput schema / examples

Added value: +[
+  {
+    "hypothesis": "the cat sat on the mat",
+    "reference": "the cat sat on the mat",
+    "threshold": 0.5
+  }
+]

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "count": {
+    "type": "number"
+  },
+  "f1": {
+    "type": "number"
+  },
+  "mode": {
+    "type": "string"
+  },
+  "precision": {
+    "type": "number"
+  },
+  "recall": {
+    "type": "number"
+  },
+  "results": {}
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

A3.6/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds only 'No API key needed' and 'Supports batch mode'; it says nothing about determinism of scoring, cost/latency for many metrics, or what happens when both single and batch inputs are given. Useful but thin beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with the verb and resource, followed by metrics, then use cases. Each sentence carries information; only the metric enumeration partially duplicates the schema, which is the one minor redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 5 documented parameters and an output schema present, the description need not explain return values. The main remaining gap is that zero parameters are marked required, yet the description never states the calling contract — supply either reference+hypothesis or a batch array — which an agent could get wrong.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents reference, hypothesis, batch, metrics, and the threshold-to-ROUGE-L-F1 rule. The description's metric listing and batch mention restate what the schema already says, adding no syntax, mutual-exclusivity, or default guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Compute text similarity between reference and hypothesis') and enumerates the concrete metric families (Cosine BoW/TF-IDF, Jaccard, ROUGE-1/2/L, BLEU), which clearly delimits it as a lexical/overlap scorer. It does not, however, name or distinguish itself from near siblings like embedding_similarity, vector_similarity, bm25_score, or response_quality_score, leaving the agent to infer the boundary from the metric list alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear positive usage contexts ('Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking') and notes batch mode, so an agent knows when the tool fits. There are no exclusions or explicit alternatives ('use embedding_similarity for semantic similarity'), so the guidance is one-sided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All IA-QA — 152 QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.