Skip to main content
Glama

IA-QA — 152 QA & Dev Tools for AI Agents

Server Details

152 free QA tools for agents: injection scan, evals, RAG, web audits. Smaller packs: /mcp/{pack}.

If you are the author of this connector, you can claim ownership by verifying the domain or GitHub account it belongs to. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over 84 days
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
Repository
ia-qa-com/mcp
GitHub Stars
0
Server Listing
IA-QA MCP server
ChatGPT
–
Claude
–

TDQS

B3.3/5.0

Scored across 153 tools

Disambiguation2/5

Several tools are near-duplicates or heavy overlaps: detect_secrets and secret_scan, cron_parse and cron_validator, and a dense cluster of response-comparison tools (consistency_check, compare_responses, analyze_responses, similarity_score, run_semantic_tests, embedding_similarity) plus llm_json_schema_check vs json_schema_validate. Descriptions are long and often disambiguate pairwise ('use X instead'), but with 153 tools an agent must read paragraphs to choose, and many boundaries remain fuzzy in practice. The set overall has multiple tools that appear to do the same thing.

Naming Consistency3/5

Names are uniformly snake_case, which helps, but verb placement is inconsistent: detect_language vs bias_detect, analyze_responses vs cot_analyzer, generate_uuid vs model_info. A meaningful subset are noun phrases (ab_test_report, pr_gatekeeper, text_stats, word_frequency, sandbox_scenario) rather than verb_noun, so no single predictable pattern holds across the set.

Tool Count1/5

153 tools for a generic 'QA & Dev' grab bag is an extreme mismatch — far beyond the 3-15 range where each tool earns its place. Many tools are micro-utilities (format_bytes, lorem_ipsum, number_base_convert) that inflate the surface and force discovery tools like find_tool to exist just to navigate the server.

Completeness3/5

Coverage is broad across text, encoding, JSON, security, LLM eval, RAG, and integrations, so most obvious workflows have an entry point. But there are notable gaps for a 'dev tools' surface — e.g. no general-purpose HTTP client (only latency_benchmark), no file read/write or shell tool, and several subdomains are reachable only through compound orchestration tools. Depth exists in the QA/eval area while other areas are shallow one-off utilities.

Available Tools

153 tools
ab_test_reportA
Read-onlyIdempotent
Inspect

Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns mean, median, std deviation, improvement percentage, and a Welch t-test at 95 %: winner names a variant only when the difference is significant — otherwise "no_significant_difference", or "insufficient_samples" under 2 scores per variant.

NameRequiredDescriptionDefault
variant_aYesFirst variant configuration with name and score array
variant_bYesSecond variant configuration with name and score array

Output Schema

NameRequiredDescription
maxNo
minNo
meanNo
countNo
medianNo
winnerNo
std_devNo
variant_aNo
variant_bNo
recommendationNo
improvement_percentNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered; the description adds real behavioral detail beyond that — the significance test used (Welch t-test at 95%), the rule that `winner` is only named when significant, and the fallback states "no_significant_difference" and "insufficient_samples". This is meaningful outcome-condition disclosure that the annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences: the first states purpose, the second delivers output list and decision rules. Nothing is redundant and the most decision-relevant detail (the winner/significance logic) is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so enumerating return fields is arguably redundant, but the description adds value by explaining the conditional semantics behind `winner`. Combined with the minimum-sample rule and the statistical method, it covers what an agent needs; only the absence of guidance on sample-size adequacy or non-normal data keeps it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both nested params are documented, so the baseline is 3. The description adds a constraint the schema omits: at least 2 scores per variant are required, otherwise "insufficient_samples" is returned. That is a genuine semantic addition beyond the structured fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ("Generate an A/B test report") and scopes it to comparing two prompts or model configurations via score arrays, which separates it from generic comparators like compare_responses. It stops short of naming a sibling alternative explicitly, so it is clear but not maximally differentiating.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: you supply two score arrays and get a comparison. There is no explicit statement of when to use this over compare_models or compare_responses, nor any exclusion of pre-computed metrics versus raw prompts. An agent must infer the context from the description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_diff_bugsA
Read-onlyIdempotent
Inspect

Pattern-based diff linter: flags a fixed set of risky shapes in changed code — query-string interpolation (SQL/Cypher/Mongo injection shape), shell interpolation, eval/new Function, empty catch blocks, regex built from a variable, fewer catch blocks than before, and named authorization guards that disappeared. Every finding cites the line that produced it. It does NOT do data-flow analysis: it cannot follow a value to a sink, across functions or files, and an empty result is not a safety verdict (the response lists what it did not analyse). Advisory triage — use a static analyser for a real security gate.

NameRequiredDescriptionDefault
contextNoOptional PR title or feature context for better analysis
version1NoOriginal code (before changes). If omitted, only the new version is analysed.
version2YesNew/modified code (after changes)

Output Schema

NameRequiredDescription
bugsNo
disclaimerNo
notAnalysedNo
overallRiskNo
rulesAppliedNo
scannedLinesNo
totalSuggestionsNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses the analysis boundary (no data-flow, cannot follow a value across functions or files), how findings are reported (each cites the producing line), and that the response lists what was NOT analysed. The warning that an empty result is not proof of safety is exactly the kind of limitation an agent would otherwise get wrong, and nothing here conflicts with readOnlyHint/idempotentHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the category label followed by the concrete check list, then limitations, then the advisory caveat — a logical order. The middle sentence is long and list-heavy, but every clause is substantive; only minor tightening would help.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Annotations cover read-only/idempotent safety, the schema fully documents all three parameters, and an output schema exists so return shape needn't be explained. What remains — scope of the checks, analytical limits, and the non-verdict of an empty result — is all present, leaving no meaningful gap for an agent deciding whether and how to call it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already defines version1, version2 and context, including the note that omitting version1 analyses only the new version. The description adds the concept of a diff between changed code but nothing further about parameter formats or interactions, which is the expected baseline when the schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific role ("Pattern-based diff linter") and then enumerates the exact risky shapes it detects in changed code, so an agent knows precisely what class of tool this is. The closing line ("Advisory triage") distinguishes it from a full security scanner, separating it from the static-analysis and security siblings in spirit even if not by name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames the intended use (advisory triage) and names the alternative class of tool: "use a static analyser for a real security gate." It also warns that an empty result is not a safety verdict, which is actionable usage guidance. It stops short of naming any of the many security-adjacent siblings (shield_analyze, secret_scan, detect_secrets), so an agent still has to infer which one to reach for instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_responses
Read-onlyIdempotent
Inspect

Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox). Without a reference: computes consensus — pairwise cosine agreement, the most-representative output, and the outlier. With a reference (ground truth): also ranks every output by closeness (token cosine + ROUGE-L composite) and names the closest. No LLM, no key, runs locally. The reference mode is gate-able in CI; the consensus/outlier mode is a signal (no reference to fail against). You bring the outputs (2+). A 2-way head-to-head with structural JSON diff is compare_responses.

NameRequiredDescriptionDefault
referenceNoOptional ground-truth answer. If set, each output is also ranked by closeness to it and the closest one is named.
responsesYesThe outputs to analyze (same task, N models/prompts/versions). Each item is a plain string or { "label": "GPT-4o", "text": "..." }. At least 2 required.

Output Schema

NameRequiredDescription
countNo
summaryNo
consensusNo
reference_rankingNo
base64_decodeA
Read-onlyIdempotent
Inspect

Decode a Base64 string back to UTF-8 text. Use for inspecting Base64-encoded API responses, JWT payload claims, config file values, or attachment data.

NameRequiredDescriptionDefault
inputYesBase64 string to decode

Output Schema

NameRequiredDescription
decodedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so safety is covered. The description adds the meaningful behavioral detail that the output is UTF-8 text, which tells the agent how to interpret the result; it stops short of stating what happens with invalid Base64 or non-UTF-8 binary bytes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with what the tool does and followed by when to use it. Every clause earns its place with no restating of the name or redundant framing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema handling return values, rich annotations handling the safety profile, and a fully documented single parameter, the description covers nearly everything needed to invoke it correctly. The remaining minor gap is error behavior on invalid or non-UTF-8 input, which neither the description nor the visible schema addresses.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single required parameter with 100% schema description coverage, so the schema already carries parameter semantics. The description restates only that it is a Base64 string and adds no format or validation guidance beyond the schema — baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Decode a Base64 string') plus the exact output interpretation ('back to UTF-8 text'), making the operation unambiguous. The verb contrast with base64_encode and the explicit UTF-8 output type let an agent distinguish it from encode and from text-formatting siblings without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives concrete usage contexts (API responses, JWT payload claims, config values, attachment data), which is clear guidance on when to reach for this tool. It does not name an alternative or exclusion — notably it overlaps with the decode_jwt sibling for JWT payloads without routing the agent between them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

base64_encodeA
Read-onlyIdempotent
Inspect

Encode a UTF-8 string to Base64. Use when you need to embed binary data, multi-line text, or special characters safely inside JSON fields, HTTP headers, or data URIs.

NameRequiredDescriptionDefault
inputYesText to encode

Output Schema

NameRequiredDescription
encodedNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so the safety profile needs no repetition. The description adds the useful fact that input is treated as UTF-8 text rather than raw bytes, but says nothing about padding, URL-safe alphabets, or error behavior on invalid input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste: the purpose is front-loaded and the guidance follows. No restatement of the name or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, idempotent, read-only transform with an output schema, the description covers everything an agent needs to select and invoke it. Only edge-case behavior (padding, URL-safe variant, invalid input) is absent, which is minor here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With a single parameter at 100% schema coverage, the baseline is 3, and the description earns one point above it by specifying that the string must be UTF-8 and may contain multi-line or special characters — meaningful beyond the schema's terse "Text to encode".

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description leads with a specific verb+resource ("Encode a UTF-8 string to Base64") that unambiguously contrasts with the base64_decode sibling. It stops short of explicitly naming or contrasting a sibling, which is the only thing separating it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives concrete use cases (embedding data in JSON fields, HTTP headers, data URIs) and the input characteristics that motivate them (binary data, multi-line text, special characters). It provides no exclusions or alternatives (e.g., url_encode for URL contexts), so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bias_detectA
Read-onlyIdempotent
Inspect

Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone). Returns a bias score (0-100), sentiment analysis per variant, pairwise Jaccard similarity, and a human-readable verdict. No API key needed — runs entirely locally.

NameRequiredDescriptionDefault
responsesYesArray of variant responses to compare for bias

Output Schema

NameRequiredDescription
verdictNo
lengthCVNo
biasScoreNo
sentimentsNo
avgSimilarityNo
minSimilarityNo
sentimentVarianceNo
pairwiseSimilaritiesNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, so the safety profile is covered. The description still adds genuine context beyond them: 'No API key needed — runs entirely locally' tells the agent there is no network/auth dependency, and the enumerated outputs set expectations for result shape. It stops short of noting cost, runtime, or minimum-variant requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core operation, then the return payload, then the operational note. No filler and nothing buried at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be described, yet the description summarizes them anyway, and the local-execution note closes the auth question. The only gap is that it doesn't state how many variants are needed or how variantId values affect scoring.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is well documented, so the baseline would be 3. The description goes further by explaining how the array must be constructed — one entry per demographic variant drawn from the same prompt template — which is a semantic constraint the schema only partially encodes via minItems.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Analyse a set of LLM responses generated from the same prompt template but with different demographic variants') and enumerates exactly what it returns (bias score, sentiment, Jaccard similarity, verdict). It never distinguishes itself from nearby siblings like toxicity_scan or guardrail_test, which is the only thing keeping it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The precondition that responses must come from the same prompt template with differing demographic variants is implied strongly enough to guide correct input construction. However, there is no explicit statement of when to reach for this tool versus toxicity_scan or guardrail_test, and no exclusions or alternatives are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bm25_scoreA
Read-onlyIdempotent
Inspect

Compute BM25 relevance score between a query and one or more documents. BM25 is the industry-standard keyword-based ranking algorithm used in Elasticsearch, OpenSearch, and Weaviate hybrid search. Returns ranked results with normalized scores.

NameRequiredDescriptionDefault
bNoLength normalization factor (default: 0.75)
k1NoTerm frequency saturation (default: 1.5)
queryYesThe search query
top_kNoReturn top K results (default: all)
documentsYesArray of documents to rank

Output Schema

NameRequiredDescription
bNo
k1No
indexNo
queryNo
resultsNo
bm25_scoreNo
doc_lengthNo
doc_previewNo
avg_doc_lengthNo
documents_countNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive behavior, so safety is covered. The description adds genuinely new behavioral context: it returns ranked results with normalized scores, which tells the agent what to expect from the call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with the core action front-loaded. The middle sentence is mildly promotional ('industry-standard') but earns its place by orienting the algorithm against known search systems.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, a fully documented parameter schema, and complete annotations, the description only needs to convey purpose and output character, which it does. Missing only explicit guidance on ordering versus its ranking/similarity siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so query, documents, b, k1, and top_k are all documented in the schema. The description adds no tuning guidance for b/k1 or explanation of top_k beyond what the schema provides, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Compute BM25 relevance score between a query and one or more documents.' The keyword-based ranking framing implicitly distinguishes it from vector or embedding sibling tools, but no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Mentions the algorithm's provenance (Elasticsearch, OpenSearch, Weaviate hybrid search), which hints at context, but never states when to choose this over alternatives such as embedding_similarity, similarity_score, or rag_relevance_rank. Usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

build_rag_promptA
Read-onlyIdempotent
Inspect

Assemble a complete RAG (Retrieval-Augmented Generation) prompt from retrieved context chunks and a user query. Handles token budgeting, citation numbering, system instruction injection, and source attribution.

NameRequiredDescriptionDefault
queryYesThe user question to answer
chunksYesRetrieved context chunks with .text (required), .source (optional), .score (optional)
languageNoResponse language instruction (e.g. "French", "Spanish")
cite_sourcesNoAdd [1], [2] citation numbers (default: true)
max_context_tokensNoMax tokens for context section (default: 2000)
system_instructionNoCustom system instruction (default: standard RAG grounding instruction)

Output Schema

NameRequiredDescription
promptNo
system_promptNo
chunks_includedNo
included_chunksNo
chunks_truncatedNo
total_tokens_estimateNo
context_tokens_estimateNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, so safety is covered. The description then adds real behavioral value beyond that: it discloses that the tool actually performs token budgeting, injects a system instruction, numbers citations, and attributes sources — transformations the agent would otherwise not know happen inside the call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste: the first states what is built and from what, the second enumerates the four processing behaviors. Front-loaded and appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Return values are covered by the output schema and safety by the annotations, so the description only needs to explain the transformation, which it does. Minor gaps: it does not say whether low-score chunks are dropped when the token budget is exceeded, or how chunks are ordered before numbering.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description's feature list maps onto the optional parameters and adds meaning — 'token budgeting' tells you max_context_tokens actually truncates context, and 'citation numbering' tells you cite_sources changes the output text, not just metadata. It does not clarify how chunk.score is used (ordering/filtering), which is the one semantic gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (assemble) and resource (RAG prompt) plus the exact inputs (retrieved chunks + user query), which clearly separates it from prompt_template_fill or system_prompt_builder. It stops short of explicitly naming a sibling it supersedes, so an agent still has to infer its place among prompt/context tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the phrase 'from retrieved context chunks and a user query' — use it when you already have retrieval results. There is no explicit when-not, no mention of where it sits relative to split_chunks, rag_relevance_rank, or truncate_to_tokens, and no prerequisites stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

calculate_readabilityA
Read-onlyIdempotent
Inspect

Calculate readability scores: Flesch Reading Ease, Flesch-Kincaid Grade Level, Coleman-Liau Index, and Automated Readability Index. Useful for evaluating LLM output quality.

NameRequiredDescriptionDefault
inputYesText to analyze for readability

Output Schema

NameRequiredDescription
levelNo
statsNo
coleman_liau_indexNo
flesch_reading_easeNo
flesch_kincaid_gradeNo
automated_readability_indexNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is fully covered. The description adds the enumerated metrics but says nothing about the return shape or failure modes for very short inputs, which is the only behavior an agent might need here.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, metric list front-loaded, zero filler. The second sentence is a soft usage hint rather than a requirement, so it is slightly expendable but not wasteful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

A single-parameter, read-only utility with annotations and an output schema, so return values need not be described. The description covers purpose and metric set; only edge-case handling (empty/short text) is unstated, which is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter ('input'), and the schema documents it at 100% coverage with an example. The description adds no syntax, length, or language constraints beyond the schema, so the high-coverage baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Calculate') and resource ('readability scores') and names the four concrete metrics produced (Flesch, Flesch-Kincaid, Coleman-Liau, ARI). This separates it from generic siblings like text_stats or word_frequency without naming them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Useful for evaluating LLM output quality' gives an implied context but no explicit when-to-use versus siblings such as text_stats, word_frequency, or response_quality_score. The agent must infer the selection criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

case_convertA
Read-onlyIdempotent
Inspect

Convert a string between naming conventions: camelCase, PascalCase, snake_case, kebab-case, UPPER_SNAKE_CASE, dot.case, Title Case. Essential for code generation and refactoring.

NameRequiredDescriptionDefault
toYesTarget case: "camel", "pascal", "snake", "kebab", "upper_snake", "dot", "title"
inputYesString to convert (e.g., "myVariableName", "my-css-class")

Output Schema

NameRequiredDescription
resultNo
from_wordsNo
target_caseNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and closed-world, so the safety profile is covered. The description adds nothing behavioral beyond that: it doesn't say how multi-word or mixed input is tokenized (the schema example passes "helloWorld fooBar"), how non-alphanumeric characters are treated, or what happens for an unknown target case.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, the supported convention list front-loaded, and no filler. Every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and both parameters are fully described in the schema. What's missing is only the tokenization/edge-case behavior for mixed-format input, which is a modest gap for a two-argument text utility.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the "to" parameter enumerates the accepted values, so the schema does the heavy lifting. The description lists the conventions in human labels (UPPER_SNAKE_CASE, dot.case) that don't exactly match the schema's codes (upper_snake, dot), so it neither adds nor cleanly reinforces parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (convert) plus the resource (a string) and enumerates the exact naming conventions supported, so the operation is unambiguous. Sibling utilities are unrelated one-shot converters (generate_slug, text_stats), so no sibling discrimination is needed.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Essential for code generation and refactoring" gives a loose usage context but never says when to use this versus related helpers like generate_slug, nor does it set any preconditions or exclusions. The usage guidance is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_contrast_ratioA
Read-onlyIdempotent
Inspect

Calculate WCAG 2.1 contrast ratio between two colors. Returns ratio and compliance for AA/AAA normal and large text.

NameRequiredDescriptionDefault
backgroundYesBackground color in hex (e.g., "#ffffff")
foregroundYesForeground color in hex (e.g., "#333333")

Output Schema

NameRequiredDescription
ratioNo
AA_largeNo
AAA_largeNo
AA_normalNo
AAA_normalNo
backgroundNo
foregroundNo
ratio_textNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/closed-world, so the safety profile is covered. The description adds real value beyond that by stating the return payload (ratio plus AA/AAA compliance for normal and large text), which tells the agent what to expect from a pure computation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler: the operation first, the return value second. Nothing is repeated from the schema and nothing needs to be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deterministic two-parameter calculator with a full schema and an output schema, this is essentially complete — input format and output content are both known. Only minor gaps remain (e.g., shorthand hex acceptance, invalid-input behavior), which do not block correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters already document hex format with examples. The description mentions 'two colors' but adds no format, ordering, or edge-case detail beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb and resource: 'Calculate WCAG 2.1 contrast ratio between two colors.' The WCAG framing distinguishes it cleanly from the neighboring color_convert tool, and it also names what comes back.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the WCAG framing (accessibility check), but the description never states when to prefer this over color_convert or other color utilities, and gives no exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

color_convertA
Read-onlyIdempotent
Inspect

Convert a color between HEX, RGB, and HSL formats. Use when translating design tokens between CSS notations, verifying color accessibility, or normalizing color values from user input. Accepts #rrggbb, #rgb, rgb(r,g,b), or hsl(h,s%,l%).

NameRequiredDescriptionDefault
inputYesColor value to convert, e.g. "#ff6b6b", "rgb(255,107,107)", "hsl(0,100%,71%)"

Output Schema

NameRequiredDescription
bNo
gNo
rNo
hexNo
hslNo
rgbNo
inputNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so safety is covered. The description adds useful behavioral detail by listing accepted input notations, including shorthand #rgb and CSS rgb()/hsl() forms, though it does not discuss invalid-input handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three tight sentences: purpose first, then usage contexts, then accepted formats. Every sentence adds useful information, and there is no repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one fully documented parameter, rich annotations, and an output schema, the description provides everything needed for correct invocation. Return-value behavior can be delegated to the output schema, so no important context is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents the single input parameter and provides examples. The description still adds meaning by enumerating supported notations (#rrggbb, #rgb, rgb(...), hsl(...)), including the #rgb shorthand that the schema examples do not explicitly mention.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: convert a color between HEX, RGB, and HSL. This clearly distinguishes it from other conversion siblings such as number_base_convert or timestamp_convert, since the resource is explicitly color formats.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear when-to-use contexts: translating design tokens between CSS notations, verifying accessibility, and normalizing user input. It does not name alternatives or exclusions, but for this simple utility the intended usage is well specified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_modelsA
Read-onlyIdempotent
Inspect

Put 2-5 AI models side by side: price per 1k tokens, context window, multimodal, reasoning, provider, pricing page. Facts only: names the cheapest and the largest context, never a "best model" — run your cases on each to choose.

NameRequiredDescriptionDefault
modelsYesArray of 2-5 model names (e.g. ["gpt-5","claude-sonnet-4-6","gemini-3.5-flash"])

Output Schema

NameRequiredDescription
rowsNo
cheapestNo
prices_as_ofNo
largest_contextNo
models_comparedNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the safety profile is covered. The description adds valuable behavioral context about output content: it reports facts only, names the cheapest and largest context, and avoids subjective recommendations. It does not mention data freshness or error handling, but that is a minor gap given the annotations and output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with zero waste. The core purpose and scope are front-loaded, followed by the key behavioral constraint. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, full annotation coverage, and a single well-documented parameter, the description provides everything an agent needs: what is compared, the input range, and the factual (non-recommendation) nature of the output. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the single 'models' parameter is fully documented in the schema, including the 2-5 constraint and examples. The description reinforces the 2-5 range but adds no new syntax or format details beyond what the schema provides. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Put ... side by side') and resource ('AI models'), enumerates the exact attributes compared (price, context window, multimodal, reasoning, provider, pricing page), and scopes the input to 2-5 models. This clearly distinguishes it from single-model siblings like model_info or list_llm_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear context for use ('run your cases on each to choose') and explicitly states what it will not do ('never a "best model"'), which helps an agent understand its role. However, it does not name specific alternative tools (e.g., model_info for single-model details) or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_responsesA
Read-onlyIdempotent
Inspect

Compare two ALREADY-PRODUCED outputs (e.g. model A vs model B on the same task) side by side. Returns deterministic metrics (token cosine, ROUGE-L, Jaccard, length/structure deltas, JSON diff) and a verdict. If a reference (ground truth) is given, scores each output against it and picks the closer one. If model + api_key are given, an LLM judge also picks a qualitative winner for the task. No re-execution — you bring the outputs.

NameRequiredDescriptionDefault
taskNoThe task/prompt both outputs were answering — used by the LLM judge for context
modelNoOptional judge model id (BYOK). When set with api_key, an LLM judge picks a qualitative winner.
api_keyNoOptional API key for the judge model (BYOK). Used only for the judge call; never stored.
label_aNoLabel for output A (e.g. "GPT-4o", "v1.0")
label_bNoLabel for output B (e.g. "GPT-5-nano", "v1.1")
referenceNoOptional ground-truth / expected answer. If set, each output is scored against it and the closer one wins (deterministic).
check_jsonNoTry to parse as JSON and compare structurally (keys, types, values)
response_aYesFirst output (e.g. model A's answer)
response_bYesSecond output (e.g. model B's answer)

Output Schema

NameRequiredDescription
judgeNo
labelANo
labelBNo
metricsNo
summaryNo
verdictNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly/idempotent/non-destructive, and the description adds real context beyond them: the deterministic metric set, that a verdict is returned, that the LLM judge is opt-in and requires BYOK credentials, and that inputs are never re-executed. It does not mention the cost/latency implications of the judge call, which is the main remaining behavioral unknown.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with what it does, followed by the two optional modes, ending with the key scoping constraint. No filler and no repetition of the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain return shapes, and it still summarizes them usefully. All 9 parameters are schema-documented, annotations cover the safety profile, and the mode logic is fully explained — nothing an agent needs to invoke this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds cross-parameter meaning the schema cannot: reference triggers per-output scoring and picks the closer one, while model+api_key activates a qualitative judge. That conditional wiring is genuinely useful beyond the individual field docs; only check_json and task go unmentioned.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Compare two ALREADY-PRODUCED outputs ... side by side") and enumerates the exact artifacts it produces (cosine, ROUGE-L, Jaccard, deltas, JSON diff, verdict). The closing "No re-execution — you bring the outputs" sharply separates it from execution-based siblings like compare_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly lays out the three operating modes: bare comparison, reference-based scoring, and model+api_key LLM judging, so an agent knows which parameter combination selects which behavior. It stops short of naming sibling alternatives (similarity_score, diff_text, analyze_responses) explicitly for the cases where those would be preferable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

consistency_check
Read-onlyIdempotent
Inspect

Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison). Fast, deterministic, no API key needed. Limitations: relies on surface-level word matching — "Paris is the capital of France" vs "Paris is the French capital" may score low despite semantic equivalence. True semantic consistency: run_semantic_tests in embedding mode. Essential for determinism testing.

NameRequiredDescriptionDefault
responsesYesArray of 2+ LLM responses to compare (same prompt, different runs)
check_factsNoCheck for contradictory numbers/facts across responses (default: true)

Output Schema

NameRequiredDescription
verdictNo
fact_driftNo
avg_similarityNo
response_countNo
pairwise_scoresNo
fact_contradictionNo
length_variance_percentNo
context_window_checkA
Read-onlyIdempotent
Inspect

Given an array of message objects [{role, content}], estimate total token usage and check if it fits in the target model's context window. Warns about truncation risk.

NameRequiredDescriptionDefault
modelYesModel id from list_llm_models (e.g. gpt-6.1-sol, claude-sonnet-5-5)
messagesYesArray of messages (system/user/assistant)
max_output_tokensNoReserved tokens for output (default: 4096)

Output Schema

NameRequiredDescription
fitsNo
roleNo
charsNo
indexNo
modelNo
tokensNo
warningsNo
breakdownNo
per_messageNo
total_tokensNo
message_countNo
context_windowNo
total_input_tokensNo
utilization_percentNo
reserved_output_tokensNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so safety is covered. The description contributes one extra behavioral trait, that it emits truncation-risk warnings, but says nothing about estimation accuracy/approximation, per-message overhead handling, or how the max_output_tokens reservation affects the verdict.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with the operation and the warning behavior front-loaded; no filler. Minor redundancy in re-specifying the message object shape that the schema already defines.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not explain return values, and it correctly omits them. It covers the core call intent, but given three sibling token tools, the missing disambiguation and estimation-caveats keep it from fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description restates the message shape (role/content) already documented in the schema and adds no semantics for max_output_tokens or how model ids affect the estimate; the pointer to list_llm_models lives in the schema, not the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource set: estimate token usage for a message array against a target model's context window. It clearly conveys the check/fit assessment and adds 'warns about truncation risk'. It stops short of naming the near-duplicate siblings (count_tokens, token_budget_calculator), so an agent must infer the boundary itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use is implied ('check if it fits in the target model's context window') as a pre-flight validation step, but there is no explicit when-to-use/when-not guidance and no mention of how it differs from count_tokens or token_budget_calculator. The agent must disambiguate from three similar token-related siblings unaided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

conversation_analyzeA
Read-onlyIdempotent
Inspect

Analyze a multi-turn conversation for context retention, topic drift, instruction following, and repetition. Accepts messages array [{role, content}]. Essential for chatbot QA.

NameRequiredDescriptionDefault
messagesYesConversation messages in order

Output Schema

NameRequiredDescription
turn_countNo
repetitionsNo
topic_driftNo
user_messagesNo
context_retentionNo
has_system_promptNo
assistant_messagesNo
avg_response_lengthNo
repetition_detectedNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world), so the description's obligation is lighter. It adds the diagnostics performed, but says nothing about output shape, scoring scale, or length limits on the messages array beyond what the annotations and output schema already convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the analysis dimensions and followed by the input shape and the use case. No filler, though the trailing marketing sentence ("Essential for chatbot QA") carries less information than the first two.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not explain return values, and annotations handle safety. It successfully communicates what is analyzed and what input to supply, so an agent has enough to call it correctly; only sibling differentiation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter is fully documented (messages array with role enum and content). The description restates "[{role, content}]" without adding ordering, size, or formatting constraints beyond the schema. Baseline 3 is appropriate for a one-parameter tool whose schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Analyze) and resource (multi-turn conversation) and enumerates the four dimensions checked: context retention, topic drift, instruction following, repetition. It is distinguishable from single-response scorers like response_quality_score, though it never names its nearest sibling analyze_responses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Essential for chatbot QA" implies the QA context in which to apply it, but there is no explicit when-to-use/when-not guidance and no routing toward alternatives such as analyze_responses or compare_responses. Usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cors_checker
Read-only
Inspect

Would a browser on this origin be allowed to read this URL? Verdict allowed | blocked | unknown, the reason, and the header to add. Only GET and HEAD are really sent: other methods measure the preflight alone. Many origins at once: cors_test.

NameRequiredDescriptionDefault
urlYesFull URL to test, e.g. https://api.example.com/resource
methodNoMethod the page uses (default: GET)
originNoOrigin of the calling page (default: https://yourdomain.com)
credentialsNoTrue for fetch with credentials: "include"
request_headersNoHeaders the page sets, e.g. ["Authorization", "Content-Type: application/json"]. Names only are sent.

Output Schema

NameRequiredDescription
fixNo
checksNo
reasonNo
statusNo
verdictNo
preflightNo
corsHeadersNo
request_sentNo
cors_testA
Read-only
Inspect

Audit a URL for CORS misconfigurations across several origins: wildcard with credentials, reflected origin, null origin, subdomain bypass, missing Vary. Returns risk level (safe/low/medium/high/critical) plus per-origin results. "unknown" means nothing was tested: every origin failed to connect, answered 5xx, or was refused with no CORS header (a 404, a firewall); never read it as "safe".

NameRequiredDescriptionDefault
urlYesFull URL to test (e.g. https://api.example.com/endpoint)
originNoCustom Origin header to test (default: tests multiple origins automatically)

Output Schema

NameRequiredDescription
urlNo
errorNo
testsNo
warningNo
risk_levelNo
origins_testedNo
total_findingsNo
origins_reachableNo
origins_conclusiveNo
origins_inconclusiveNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, and destructiveHint=false, so the safety profile is covered. The description goes meaningfully beyond that by disclosing the return vocabulary (safe/low/medium/high/critical plus per-origin results) and, critically, warning that "unknown" means nothing was tested and must never be read as "safe" — an anti-misinterpretation note that directly prevents a false negative. It does not explain why idempotentHint=false (results can differ between runs) or mention timing/rate considerations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences, zero filler, and the core action is front-loaded before the return semantics and the "unknown" caveat. Every sentence carries information an agent needs to act or to interpret output correctly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only two parameters, an output schema present, and annotations covering safety, the description is nearly complete: it explains the scan scope, the risk grading, and the failure-mode interpretation. The remaining gap is disambiguation from cors_checker, which matters for tool selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented, including that omitting origin tests multiple origins automatically. The description's phrase "across several origins" and "per-origin results" restates this rather than adding new meaning; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Audit a URL for CORS misconfigurations") and enumerates the concrete checks performed: wildcard with credentials, reflected origin, null origin, subdomain bypass, missing Vary. It does not, however, differentiate itself from the very similar sibling cors_checker, leaving the agent to guess which of the two CORS tools to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the verb and the required url parameter, but there is no explicit when-to-use / when-not guidance and no named alternative. The presence of a near-duplicate sibling (cors_checker) makes this omission more costly than it would otherwise be.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cot_analyzerA
Read-onlyIdempotent
Inspect

Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM. Detects step count, logical flow, conclusion presence, backtracking, and estimates reasoning depth. Useful for o1/o3/DeepSeek-R1 evaluation.

NameRequiredDescriptionDefault
reasoningYesThe CoT / reasoning trace text (e.g. from <think> tags or step-by-step output)
expected_conclusionNoExpected final answer to check against (optional)

Output Schema

NameRequiredDescription
markersNo
step_countNo
total_charsNo
total_linesNo
has_conclusionNo
reasoning_depthNo
backtracking_signalsNo
reasoning_depth_labelNo
conclusion_matches_expectedNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true and destructiveHint=false, so the safety profile is covered. The description adds real behavioral value beyond that by listing exactly which signals the analysis emits (step count, logical flow, backtracking, depth estimate). It does not say anything about thresholds, scoring scale, or what happens with an unstructured trace, but given the annotation coverage that is a tolerable gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, and each sentence earns its place: one states what is analyzed and what is detected, the second gives the target use case. No filler or restated title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain return values, and annotations cover the read-only/idempotent profile; the description fills in the analytical scope well. The remaining shortfall is that it doesn't set expectations about input quality (e.g. that traces without explicit steps are handled differently), but overall an agent has enough to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both parameters carry their own descriptions, so the baseline of 3 applies. The description never mentions 'expected_conclusion' or the optional comparison behavior it enables, adding essentially nothing about parameters beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Analyze') and resource ('Chain-of-Thought (CoT) or reasoning trace from an LLM') and enumerates the outputs it produces: step count, logical flow, conclusion presence, backtracking, reasoning depth. It is clear what it does, but it never names a sibling it is distinct from, so the agent must infer the boundary with analyze_responses, response_quality_score, or validate_agent_trajectory.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The sentence 'Useful for o1/o3/DeepSeek-R1 evaluation' implies a context (reasoning-model traces) but not a decision rule. There is no when-not guidance, no prerequisite (e.g. trace must be a CoT rather than a final answer only), and no routing to or away from overlapping siblings such as analyze_responses or response_quality_score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

count_code_linesA
Read-onlyIdempotent
Inspect

Count lines of code: total, code lines, comment lines, blank lines, and comment density. Supports JS/TS, Python, Java/C/C++, Ruby, Go, Shell, HTML/XML, and CSS.

NameRequiredDescriptionDefault
inputYesSource code to analyze
languageNoLanguage hint: "js", "ts", "py", "java", "c", "rb", "go", "sh", "html", "css" (auto-detect if omitted)

Output Schema

NameRequiredDescription
languageNo
code_linesNo
blank_linesNo
total_linesNo
comment_linesNo
comment_densityNo
code_to_comment_ratioNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, closed-world behavior, so the bar is lower. The description adds genuinely new behavioral context: the tool is language-aware and understands comment syntax for nine language families, which is why its output differs from naive line counting. It omits only minor details such as auto-detection fallback behavior (though the schema covers that).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler: the first lists what is measured, the second lists supported inputs. Both are front-loaded and every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering the safety profile, the description needn't explain return values. Parameters are fully documented in schema. The only real gap is disambiguation from text_stats, a same-family sibling with overlapping purpose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds value by spelling out the supported language families by full name (JS/TS, Python, Java/C/C++, Ruby, Go, Shell, HTML/XML, CSS), which clarifies the abbreviated enum-less hints in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb ('Count') and resource ('lines of code') and enumerates exactly what is produced: total, code, comment, blank lines, plus comment density. It is clearly a code-specific line counter, but never distinguishes itself from the sibling text_stats, which an agent could easily confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the domain (use it on source code), but there is no explicit 'use this when' statement or mention of the closest alternative, text_stats. The agent must infer the boundary itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

count_tokens
Read-onlyIdempotent
Inspect

Estimate the token count of a text string using the cl100k_base approximation (~4 chars/token). Shows whether a text fits the context window of a model and what it costs. Returns token estimate, character count, and word count.

NameRequiredDescriptionDefault
inputYesText to count tokens for

Output Schema

NameRequiredDescription
charsNo
wordsNo
tokens_estimateNo
create_confluence_page
Destructive
Inspect

Create a new Confluence page from the output of jira_to_test_suite. Formats Gherkin, E2E steps, API tests, and test data as a properly structured Confluence page with code blocks and tables. STATEFUL — creates a new page in the specified space.

NameRequiredDescriptionDefault
titleNoPage title. Defaults to "Test Plan: {issue_key}"
issue_keyNoSource Jira issue key (for the page title and source link)
issue_urlNoSource Jira issue URL (added as a link in the page)
space_keyYesConfluence space key where the page will be created, e.g. "QA", "ENG"
test_suiteYesThe test_suite object from jira_to_test_suite result
parent_page_idNoOptional parent page ID — page will be created as a child of this page
confluence_emailYesAtlassian account email
confluence_tokenYesAtlassian API token
confluence_base_urlYesAtlassian base URL

Output Schema

NameRequiredDescription
titleNo
page_idNo
successNo
page_urlNo
cron_parseA
Read-onlyIdempotent
Inspect

Parse a cron expression into a human-readable schedule description. Supports standard 5-field cron (minute hour day month weekday).

NameRequiredDescriptionDefault
expressionYesCron expression (e.g., "0 9 * * 1-5", "*/15 * * * *")

Output Schema

NameRequiredDescription
fieldsNo
expressionNo
human_readableNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so safety needs no restating. The description adds genuine scope context beyond the annotations: 'standard 5-field cron (minute hour day month weekday)' tells the agent that 6-field/Quartz-style or seconds-prefixed expressions are out of scope and would likely fail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero padding, with the primary action stated first and the format constraint second. Every clause earns its place and nothing is buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return-value explanation is unnecessary, and the single required parameter is fully covered. The only minor gap is error behavior for malformed or non-standard expressions and any timezone/Day-of-week numbering assumptions, which would help an agent predict failures.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single 'expression' parameter is documented with two examples in the schema itself, so the baseline is 3. The description's field-order parenthetical reinforces the format but adds little the schema and its examples do not already convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Parse a cron expression') and names the exact output ('human-readable schedule description'), so the input-to-output transformation is unambiguous. No sibling tool touches cron, so an agent can route to it without inspecting the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the name and one-line purpose – pass a cron string, get readable text – but the description never says when to reach for this instead of, say, timestamp_convert, nor does it flag any preconditions. For a self-evident pure utility this is adequate but unstated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cron_validatorA
Read-onlyIdempotent
Inspect

Validate a 5-field cron expression, explain the schedule, and preview the next execution times. Use this to debug cron jobs before they reach production. Returns parsed fields, a human-readable description, and upcoming ISO timestamps.

NameRequiredDescriptionDefault
expressionYesCron expression with 5 fields, e.g. "*/15 9-18 * * 1-5"
next_runs_countNoHow many upcoming runs to return (1-50, default: 10)

Output Schema

NameRequiredDescription
validNo
fieldsNo
next_runsNo
expressionNo
human_readableNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, so the safety profile is fully covered. The description adds that the tool returns parsed fields, a human-readable description, and upcoming ISO timestamps, but since an output schema exists this is largely redundant rather than genuine extra behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the verb+resource and followed by purpose and return value. Nearly every sentence earns its place, with only minor redundancy given the existing output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter read-only tool with an output schema and full annotation coverage, the description is essentially complete. The one gap is the relationship to the cron_parse sibling, which an agent would have to resolve on its own.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% – both expression and next_runs_count are documented in the schema with examples and ranges. The description reinforces the 5-field constraint and the next-run preview, but adds no syntax or format information beyond the schema, matching the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Validate a 5-field cron expression') plus two concrete adjacent capabilities (explain the schedule, preview next runs). It is clear what the tool does, but it never distinguishes itself from the sibling cron_parse, which plausibly covers the same parsing/explaining ground.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this to debug cron jobs before they reach production' gives a usage context, so guidance is implied rather than absent. However, there is no when-not guidance, no mention of the cron_parse alternative, and no statement about how to handle 6-field (seconds) expressions besides the implicit rejection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

decode_jwtA
Read-only
Inspect

Decode a JWT (JSON Web Token) and return its header and payload without verifying the signature. Also reports whether the token is expired and the exact expiry date. Use to inspect claims (sub, iss, exp, roles) during debugging or when integrating with an auth provider.

NameRequiredDescriptionDefault
tokenYesThe JWT string to decode (header.payload.signature)

Output Schema

NameRequiredDescription
noteNo
headerNo
expiredNo
payloadNo
expiresAtNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the single most important behavioral trait beyond the annotations: the signature is NOT verified, so the output is untrusted inspection data. It also states it reports expiry status and the exact expiry date, giving the agent concrete output expectations. The annotations cover safety (readOnly, non-destructive), so this addition is exactly the value-add layer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the first defines the operation and its semantics, the second gives usage context. No redundant restatement of the name or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full parameter coverage, the description need only convey purpose, the no-verification caveat, and when to reach for it — all of which it does. Nothing required to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one required parameter at 100% schema coverage, with the schema documenting the 'header.payload.signature' format. The description adds no format or syntax detail beyond that, so it neither compensates nor detracts — the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Decode a JWT') plus the exact return content (header and payload) and a key scoping caveat ('without verifying the signature'). This clearly separates it from siblings like base64_decode or extract_json_path, which handle unrelated formats.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the use case — inspecting claims (sub, iss, exp, roles) during debugging or auth-provider integration — which tells the agent when this is the right tool. It does not, however, name an alternative for validation/verification scenarios, so the agent must infer that no sibling verifies signatures.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

detect_languageA
Read-onlyIdempotent
Inspect

Detect the natural language of a text using n-gram frequency analysis and common word markers. Supports 15 languages: English, French, Spanish, German, Italian, Portuguese, Dutch, Russian, Chinese, Japanese, Korean, Arabic, Polish, Turkish, Swedish.

NameRequiredDescriptionDefault
inputYesText to detect language from (min 20 chars for accuracy)

Output Schema

NameRequiredDescription
langNo
nameNo
scoreNo
methodNo
matchedNo
languageNo
confidenceNo
top_candidatesNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context: the detection method and the exact set of 15 supported languages, which helps an agent understand capability boundaries. It does not discuss confidence or failure behavior, but that is a minor gap given the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no wasted language. The core purpose and method come first, followed by the supported-language list, which is a useful scoping detail front-loaded after the main action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained in the description. The description covers the method, supported languages, and purpose, while annotations and schema cover safety and parameter constraints. An agent has enough to select and call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter, and the schema description coverage is 100%; the schema already explains that 'input' must be text and recommends at least 20 characters. The description adds no parameter-level details beyond what the schema provides, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Detect the natural language of a text') and adds the method (n-gram frequency analysis and common word markers). It also enumerates the supported language set, which clearly distinguishes it from generic text tools like text_stats or word_frequency.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for language identification and lists supported languages, but it does not explicitly say when to use this versus alternatives such as analyzing readability or word frequency. No exclusions or preconditions are given beyond what the schema already states.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

detect_secrets
Read-onlyIdempotent
Inspect

Scan code or config files for hardcoded secrets: AWS keys, GitHub tokens, OpenAI/Anthropic API keys, Stripe secrets, JWTs, Slack/Teams/Discord webhook URLs, session cookies, database connection strings, and generic passwords. Returns findings with severity.

NameRequiredDescriptionDefault
inputYesCode or config content to scan (max 500KB)
filenameNoOptional filename for context (e.g. ".env", "config.js")

Output Schema

NameRequiredDescription
filenameNo
findingsNo
risk_levelNo
recommendationNo
total_findingsNo
diff_mappingsA
Read-onlyIdempotent
Inspect

Diff a baseline page mapping against a current one and return a CI-style verdict: PASS / FIX / BLOCK, plus per-element drift (ok, renamed, healable, ambiguous, lost, added, rebound). Pure and deterministic — provide two mappings as JSON with "elements" arrays of {role, name, selector, context?}. Use the companion @ia-qa/self-healing package (npm install -g @ia-qa/self-healing) to capture mappings from your app via its local MCP server ia-qa-heal-mcp, or paste the snippet from ia-qa.com/devtools/selector-drift into your browser console.

NameRequiredDescriptionDefault
afterYesCurrent page mapping: same shape as before, captured after the UI change.
beforeYesBaseline page mapping: { page, url, capturedAt, elements: [{role, name, selector, context?}] }. Captured before a UI change.

Output Schema

NameRequiredDescription
rowsNo
addedNo
countsNo
verdictNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so safety is covered structurally. The description adds genuinely new behavioral context: 'Pure and deterministic' and the enumerated outcome categories (ok, renamed, healable, ambiguous, lost, added, rebound) that signal what the agent should expect back. It doesn't cover edge cases like malformed mapping input, so not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and verdict vocabulary are front-loaded and the input-format sentence earns its place. The trailing install command and devtools URL are useful for capture but lean promotional and add length an agent doesn't strictly need to invoke the tool, keeping it from a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be spelled out, and the schema fully documents both inputs. The description still adds the decisive missing pieces — determinism, verdict semantics, and how to source mappings — making it complete for an agent to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both 'before' and 'after' are already documented in the schema, including the elements shape. The description restates the same element shape ({role, name, selector, context?}) rather than adding new meaning, so it earns the high-coverage baseline of 3 rather than more.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Diff a baseline page mapping against a current one') and names the output contract ('CI-style verdict: PASS / FIX / BLOCK, plus per-element drift'). The domain specificity (page mappings, selector drift categories) distinguishes it from generic siblings like diff_text and json_diff without needing to name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It tells the agent the inputs needed (two mappings as JSON with 'elements' arrays) and how to obtain them (companion @ia-qa/self-healing package or the browser-console snippet), which is clear usage context. What's missing is an explicit exclusion or alternative — e.g. when to prefer diff_text or analyze_diff_bugs over this tool — leaving some routing inference to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

diff_textA
Read-onlyIdempotent
Inspect

Compute a unified line-by-line diff between two text strings (LCS algorithm). Returns added/removed/unchanged line counts and formatted diff hunks with configurable context lines (0–20). Use to compare versions of prompts, configs, code snippets, or any text where you need to see exactly what changed.

NameRequiredDescriptionDefault
aYesOriginal (before) text
bYesModified (after) text
contextNoContext lines around each change (0–20, default: 3)

Output Schema

NameRequiredDescription
diffNo
addedNo
removedNo
unchangedNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior, so the bar is lower. The description adds meaningful context beyond annotations: the LCS algorithm, line-count output, formatted hunks, and configurable context lines, though it does not mention limitations such as line-oriented rather than word-oriented comparison.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tightly written sentences with no filler. It front-loads the core operation and follows immediately with practical use cases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given complete schema coverage, an output schema, and clear safety annotations, the description supplies everything needed to select and invoke the tool correctly. The algorithm and return-summary details are useful additions rather than requirements.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents a as original text, b as modified text, and context as 0–20 with default 3. The description reinforces the context-line range but adds no syntax or edge-case guidance beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Compute a unified line-by-line diff between two text strings' and even names the LCS algorithm. It clearly implies a text-diff use case distinct from structured tools like json_diff, but it does not explicitly name or distinguish itself from that sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage scenarios — comparing prompt versions, configs, code snippets, or any text where exact changes matter. It does not state when not to use it or name an alternative such as json_diff, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

embedding_similarity
Read-onlyIdempotent
Inspect

Compute text similarity using local algorithms (Bag of Words, TF-IDF, Character N-grams). No API key needed — runs entirely in-process. NOT real embeddings: true vector embeddings are run_semantic_tests with mode="embeddings" and an OpenAI key. Supports single pair or batch mode with pipe-separated pairs. Useful for RAG retrieval testing, semantic search evaluation, and text deduplication.

NameRequiredDescriptionDefault
batchNoBatch mode: array of { text_a, text_b } pairs. Overrides text_a/text_b if provided.
text_aNoFirst text to compare (single-pair mode)
text_bNoSecond text to compare (single-pair mode)
methodsNoAlgorithms to use (default: all three). Options: "bow", "tfidf", "ngram"

Output Schema

NameRequiredDescription
modeNo
countNo
scoresNo
text_aNo
text_bNo
resultsNo
env_parseA
Read-onlyIdempotent
Inspect

Parse a .env file content into a JSON object. Handles quoted values (single and double), inline comments, export prefix, and escaped sequences (\n, \t inside double quotes). Returns all key-value pairs. Use in CI/CD pipelines, agent config loaders, or when processing dotenv files programmatically.

NameRequiredDescriptionDefault
inputYes.env file content to parse (e.g. the output of `cat .env`)

Output Schema

NameRequiredDescription
varsNo
countNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/non-open-world, so the safety profile is covered. The description adds meaningful behavioral detail beyond that: quoted values, inline comments, export prefix, escape sequences, and that all key-value pairs are returned. It does not say how malformed lines or duplicate keys are handled, which is the remaining gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core action and followed by supported syntax edge cases that genuinely earn their place. The closing usage sentence is the least essential but remains short and non-redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

A one-parameter, read-only utility with an output schema and full annotation coverage needs little more than this; the description covers purpose, supported syntax, and output shape at a high level. Only edge-case/error behavior for malformed input is unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With one parameter at 100% schema description coverage, the schema already explains that the input is raw .env file content (with a `cat .env` example). The description confirms the input is whole file content rather than a path, but adds no format/size limits beyond what the schema says, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Parse a .env file content into a JSON object') and immediately enumerates the syntax it handles, which pins down the exact transformation. This clearly separates it from format-conversion siblings like yaml_to_json or parse_csv without needing to name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage contexts ('CI/CD pipelines, agent config loaders, or when processing dotenv files programmatically'), which tells an agent the intended scenario. It stops short of stating when NOT to use it or naming an alternative tool, but for a single-purpose formatter the surrounding context is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

escape_html
Read-onlyIdempotent
Inspect

Escape HTML special characters (&, <, >, ", ') to their safe HTML entities. Escaped text inserted into HTML cannot run as script (XSS).

NameRequiredDescriptionDefault
inputYesString to HTML-escape

Output Schema

NameRequiredDescription
escapedNo
original_lengthNo
estimate_llm_costA
Read-onlyIdempotent
Inspect

Estimate the API cost in USD for a given model and token counts, at the price read daily from the provider's own pricing page (prices_as_of, price_status). Any model from list_llm_models.

NameRequiredDescriptionDefault
modelYesModel name, e.g. "gpt-4o", "claude-3.5-sonnet", "deepseek-v3"
input_tokensYesNumber of input/prompt tokens
output_tokensNoNumber of output/completion tokens (default: 0)

Output Schema

NameRequiredDescription
modelNo
ratesNo
input_tokensNo
output_tokensNo
input_cost_usdNo
total_cost_usdNo
output_cost_usdNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the bar is lower, and the description still adds real value: it discloses provenance and freshness (price read daily from the provider's own pricing page) plus the prices_as_of/price_status signals the caller can use to judge staleness. It doesn't mention behavior for unknown or unlisted models, which keeps it out of 5 territory.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the core action, with the provenance caveat in a compact parenthetical. Nothing extraneous.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema covering return fields and annotations covering the safety profile, the description supplies what's missing: provenance and freshness of prices. Only the edge case of an unrecognized model is unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so a 3 is the baseline, and the description earns an extra point by explaining where the `model` value must come from (list_llm_models) rather than leaving it as an arbitrary string. Token parameters are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("Estimate the API cost in USD") with the exact inputs that drive it (model + token counts), which cleanly separates it from token-counting siblings like count_tokens or token_budget_calculator. It also names the model catalog sibling, so the agent knows where valid model names come from.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Any model from list_llm_models" gives a clear prerequisite/discovery path and the pricing-source note sets expectations about the numbers returned. It stops short of explicit when-not guidance or a direct comparison against token_budget_calculator, so no full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_json_from_text
Read-onlyIdempotent
Inspect

Extract the first valid JSON object or array embedded in chaotic LLM output (surrounded by markdown fences, prose, or explanatory text). Handles ```json blocks and inline JSON.

NameRequiredDescriptionDefault
inputYesRaw text (e.g., LLM output) that may contain a JSON object or array

Output Schema

NameRequiredDescription
jsonNo
sourceNo
extract_json_pathA
Read-onlyIdempotent
Inspect

Extract a value from a JSON string using dot-notation path (e.g., "user.address.city", "items.0.name", "meta.tags"). Supports array index access via numeric path segments.

NameRequiredDescriptionDefault
pathYesDot-notation path, e.g. "user.address.city" or "items.0.name"
inputYesThe JSON to traverse — a JSON string, or the object/array itself.

Output Schema

NameRequiredDescription
pathNo
typeNo
valueNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the safety profile is covered. The description adds useful path-syntax context (array index via numeric segments) but omits the most behaviorally relevant fact: behavior on missing paths, type coercion, or how a JSON string input is parsed versus an object input. With annotations carrying the safety burden, this lands at a moderate 3.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the core action and followed by the two syntax variants that matter most. Nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read tool with a full schema and an output schema, the description covers what an agent needs to invoke it correctly. The only real omission is missing-path/edge-case behavior, which keeps it short of a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented, and the description's path examples duplicate the schema's own example. No additional semantics (escaping, edge cases, root path) are provided beyond the structured fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Extract a value from a JSON string') and immediately distinguishes itself from JSON-reshaping siblings like flatten_json or format_json by scoping to path-based traversal. The example paths make the operation unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the examples rather than stated. It never says when to prefer this over flatten_json or json_to_csv, nor what happens if the path doesn't resolve (error vs null). Adequate but with a clear gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_todosA
Read-onlyIdempotent
Inspect

Extract TODO, FIXME, HACK, BUG, NOTE, OPTIMIZE, and custom tags from any source code or text. Returns line numbers, tag types, and message text. Essential for technical debt auditing.

NameRequiredDescriptionDefault
tagsNoCustom tags to add (default set: TODO, FIXME, HACK, NOTE, BUG, OPTIMIZE, XXX)
inputYesCode or text to scan
include_contextNoInclude full line text (default: true)

Output Schema

NameRequiredDescription
itemsNo
totalNo
countsNo
has_criticalNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds that it works on 'any source code or text' and what the result contains, but says nothing about scanning limits, custom-tag merge behavior, or context truncation beyond what the schema hints at.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: capability first, return shape second, motivation last. Nothing is redundant or buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and 100% parameter coverage, the description only needs to frame the capability, which it does. Minor gap: it lists the tag set but omits XXX, which the schema lists as part of the default set.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline applies; the description merely restates that custom tags are supported and that line text is returned. It adds no syntax, case-sensitivity, or default-set detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb 'Extract' plus the exact tag set and the return payload (line numbers, tag types, message text). It is clearly distinguishable from nearby tools like count_code_lines or detect_secrets, though it never names a sibling to route against.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Essential for technical debt auditing' implies the use case, but there is no explicit when-to-use/when-not guidance and no named alternative (e.g. detect_secrets for secrets, count_code_lines for metrics). The agent must infer the boundary itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_confluence_pageA
Read-only
Inspect

Fetch a Confluence page and return its content as clean Markdown. Accepts a numeric page_id or a full page URL. Optionally lists direct child pages. BYOK — credentials transit in-memory only, never stored.

NameRequiredDescriptionDefault
page_idNoConfluence page ID (numeric string), e.g. "123456789"
page_urlNoFull Confluence page URL (alternative to page_id), e.g. "https://mycompany.atlassian.net/wiki/spaces/ENG/pages/123456789"
confluence_emailYesAtlassian account email (same credentials as Jira)
confluence_tokenYesAtlassian API token
include_childrenNoList direct child pages (id + title) (default: false)
confluence_base_urlYesAtlassian base URL, e.g. "https://mycompany.atlassian.net"

Output Schema

NameRequiredDescription
urlNo
titleNo
page_idNo
childrenNo
markdownNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=true, so the safety profile is covered. The description adds genuinely new behavioral context beyond the annotations: credentials transit in-memory only and are never stored (BYOK), which matters for an agent handling secrets. It still omits failure behavior (auth errors, rate limits, missing page).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with zero waste: purpose and output format first, then input modes, then the optional flag, then the credential-handling caveat. Every sentence adds information and nothing is buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain return structure, and it still notes the Markdown format for the page body. All required auth parameters are accounted for by the schema and reinforced by the BYOK sentence. The only gap is handling of the optional child listing's output shape and error paths.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all six parameters including the page_id/page_url alternatives and include_children default. The description restates the page_id-or-URL choice and the optional child listing without adding format details (e.g. that page_id must be numeric, or how to extract the ID from a URL). Baseline 3 applies when the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb+resource ("Fetch a Confluence page") and adds the return format ("clean Markdown"), plus the two input modes (page_id or URL). It is unambiguous about what the call does, but never differentiates itself from the sibling create_confluence_page or fetch_jira_issue, so an agent must infer the fetch-vs-create split.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Accepts a numeric page_id or a full page URL" and "Optionally lists direct child pages" imply when the flags apply, but there is no explicit when-to-use statement, no prerequisite guidance (credentials are required, not optional), and no mention of alternatives such as using a different tool to create or search pages.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_jira_issueA
Read-only
Inspect

Fetch a complete Jira issue: summary, description converted to Markdown, status, assignee, priority, labels, custom fields, and optionally comments and attachment metadata. BYOK — credentials transit in-memory only, never stored on ia-qa.com.

NameRequiredDescriptionDefault
fieldsNoSpecific Jira field names to return. Omit for all standard fields.
issue_keyYesJira issue key, e.g. "PROJ-123"
jira_emailYesAtlassian account email
jira_tokenYesAtlassian API token (from id.atlassian.com > Security > API tokens)
jira_base_urlYesAtlassian base URL, e.g. "https://mycompany.atlassian.net"
include_commentsNoInclude issue comments, up to 20 (default: true)
include_attachmentsNoInclude attachment metadata list (default: false)

Output Schema

NameRequiredDescription
keyNo
urlNo
typeNo
labelsNo
statusNo
summaryNo
assigneeNo
priorityNo
reporterNo
descriptionNo

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly=true, destructive=false, idempotent=false, openWorld=true, so the safety profile is covered. The description adds genuinely new context the annotations cannot convey: the BYOK credential model ('transit in-memory only, never stored') and the fact that descriptions are transformed to Markdown on return. It stops short of covering error behavior for invalid keys or missing permissions, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the payload contents and followed by the security posture that an agent needs before handling credentials. No filler and nothing buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and the description covers content plus the non-obvious BYOK auth requirement for a 4-required-parameter tool. The remaining hole is failure/edge behavior (invalid or inaccessible issue key) for an open-world remote fetch.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% — every one of the 7 parameters, including the token/auth trio and the fields filter, is documented inline. The description restates the optional comment and attachment behavior but adds no defaults, formats, or constraints beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

It names a specific verb and resource ('Fetch a complete Jira issue') and enumerates exactly what comes back: summary, Markdown-converted description, status, assignee, priority, labels, custom fields, plus optional comments and attachment metadata. What's missing is any explicit differentiation from siblings like search_jira_issues — an agent must infer the single-issue-by-key vs. search distinction from the field list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description never states when to reach for this tool versus search_jira_issues, nor does it tell the agent to obtain an issue_key via search first. The only implicit usage signal is the phrase 'optionally comments and attachment metadata', which hints at the two boolean flags rather than providing routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_veille_feed
Read-only
Inspect

Fetch the latest QA & AI/LLM articles aggregated from curated RSS sources (Google Testing Blog, DEV.to Testing/QA/AI/LLM/Agents, Hugging Face Blog, Simon Willison). Each article carries summary_source — the XML tag the summary was read from, or "none" when the feed publishes titles and links only; an empty summary with summary_source "none" is a property of that feed, not a parse failure.

NameRequiredDescriptionDefault
limitNoMax articles to return (default: 20, max: 50)
categoryNoFilter: "qa" (testing/quality), "ai" (AI/LLM/agents), "all" (default — both)

Output Schema

NameRequiredDescription
articlesNo
categoryNo
total_foundNo
sources_queriedNo
few_shot_formatterA
Read-onlyIdempotent
Inspect

Format few-shot examples for LLM prompts. Converts example pairs into formatted blocks. Supports chat format (User/Assistant), XML tags, Markdown, or plain text.

NameRequiredDescriptionDefault
formatNoOutput format (default: chat)
examplesYesArray of {input, output} pairs
input_labelNoLabel for input (default: User / <input>)
output_labelNoLabel for output (default: Assistant / <output>)

Output Schema

NameRequiredDescription
formatNo
formattedNo
example_countNo
token_estimateNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The tool is inherently a transformation tool, and the annotations confirm it is safe (readOnly, non-destructive). The description does not introduce new behavioral constraints (e.g., token limits or specific failure modes), but it is consistent with the safety profile provided by the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is extremely concise. The two sentences are front-loaded and efficient, providing the function and the key features without unnecessary jargon or padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that the complexity of the tool is low (it is a formatter), the description, combined with the 100% schema coverage and safety annotations, provides a complete mental model for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

While the schema provides the structure, the description provides crucial semantic meaning to the 'format' and 'label' parameters by defining the distinct styles supported (e.g., '(User/Assistant)' for chat mode). This elevates the input understanding beyond simple type definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is highly specific, clearly stating the tool's function: 'Format few-shot examples for LLM prompts.' It immediately distinguishes itself by listing the supported outputs (chat, XML, Markdown, plain text), which sets it apart from generic formatting siblings like json_to_yaml.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for LLM prompting but lacks explicit guidance on *when* this is superior to other prompt-building tools (like prompt_template_fill or build_rag_prompt). It is functionally clear but lacks comparative advice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_toolA
Read-onlyIdempotent
Inspect

Search available MCP tools by keyword or category before calling them. Returns names, descriptions and, on request, inputSchemas. Queries are logged 90 days, screened for credentials and personal data. Categories: data, dev, encoding, eval, integrations, llm, meta, qa, rag, security, text, web.

NameRequiredDescriptionDefault
queryYesKeywords, e.g. "cors", "token", "vector", "json"
categoryNoOptional filter, one of the categories above. Any other value is refused.
max_resultsNoMaximum tools to return (default 10, max 50). Best match first.
with_schemaNoSet true to include inputSchema in results (default: false)

Output Schema

NameRequiredDescription
hintNo
toolNo
countNo
queryNo
scoreNo
toolsNo
categoryNo
truncatedNo
total_matchesNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world), but the description adds material context the annotations do not: the 90-day query logging, screening for credentials and personal data, and the fact that inputSchemas are only returned on request. It does not state error behavior beyond what the schema notes about refused categories, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences, front-loaded with the action and the return shape, then the privacy note, then the category list. Nothing is padded, though the logging sentence interrupts the flow between returns and categories.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return structure need not be described; all four parameters are documented at 100% and the category vocabulary is supplied in the description. Combined with annotations covering safety and the added privacy disclosure, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description earns an extra point by enumerating the twelve valid category values that the schema only refers to as "one of the categories above" (enums count is zero). It otherwise restates defaults already documented per-parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource -- search MCP tools -- plus the scope of the search (keyword or category) and what it returns (names, descriptions, optionally inputSchemas). An agent can distinguish it from siblings like rate_tool or model_info without opening the schema, since it is the only discovery tool in the list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Search available MCP tools ... before calling them" gives clear situational context: use it as a discovery step prior to invocation. There are no explicit exclusions or named alternatives, so it sits at 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fix_gherkinA
Read-only
Inspect

Fix Gherkin syntax warnings from a jira_to_test_suite result. Takes the current gherkin text and the _gherkin_warnings array, calls your LLM to fix ONLY the flagged issues (adds missing Given/When/Then steps, etc.), and returns the corrected Gherkin. Lightweight — uses ~300-500 tokens vs ~5k for a full regeneration. Requires BYOK LLM key.

NameRequiredDescriptionDefault
modelYesLLM model to use for the fix, e.g. "gpt-4o-mini". Must belong to the provider whose key you passed in api_key.
api_keyYesYour own LLM provider API key (BYOK) — OpenAI "sk-…", Anthropic "sk-ant-…", Google "AIzaSy…", or Groq "gsk_…". There is no server-side key for this tool: if you do not have one, do not call it and do not invent a value — placeholders like "configured", "your_api_key" or a masked "sk-…***…" are rejected. Used for this call only, never stored.
gherkinYesThe current Gherkin text from the jira_to_test_suite result (test_suite.gherkin).
warningsYesThe _gherkin_warnings array from the jira_to_test_suite result.

Output Schema

NameRequiredDescription
latency_msNo
model_usedNo
fixed_gherkinNo
warnings_afterNo
warnings_beforeNo
remaining_warningsNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true and openWorldHint=true; the description corroborates by disclosing the external LLM call, the BYOK key requirement, and a concrete cost estimate (~300-500 tokens), plus the return ('corrected Gherkin'). It does not describe failure behavior (invalid key, unfixable warnings) or whether output is validated after fixing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three front-loaded sentences: what it does, the constraint on scope ('ONLY the flagged issues'), then the cost and key prerequisite. No filler and no repetition of field-level details already in the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations, a full output schema, and 100% parameter coverage already in structured fields, the description covers the remaining agent-relevant context: provenance of inputs, scope of the fix, cost, and the BYOK prerequisite. Minor omission: what happens when warnings is empty or the key is invalid.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter is already documented, including BYOK key formats and model/provider pairing. The description only restates that gherkin and warnings come from the jira_to_test_suite result, adding no format or edge-case detail beyond the schema — the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Fix Gherkin syntax warnings') and anchors it to a named upstream artifact ('from a jira_to_test_suite result'). An agent can tell this apart from jira_to_test_suite and generate_test_cases: this is a targeted repair pass, not generation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains it consumes the _gherkin_warnings array from a prior jira_to_test_suite call and fixes ONLY flagged issues, implicitly steering the agent away from full regeneration ('~300-500 tokens vs ~5k for a full regeneration'). It does not name the regeneration sibling explicitly or state when fixing is not appropriate, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flatten_jsonA
Read-onlyIdempotent
Inspect

Flatten a nested JSON object to single-level dot-notation keys (e.g. {"a":{"b":1}} → {"a.b":1}), or unflatten dot-notation keys back to a nested object. Supports custom separators.

NameRequiredDescriptionDefault
modeNo"flatten" (default) or "unflatten"
inputYesThe JSON to flatten or unflatten — a JSON string, or the object itself.
separatorNoKey separator (default: ".")

Output Schema

NameRequiredDescription
resultNo
key_countNo
max_depthNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive, so the safety profile is covered. The description adds the transformation semantics (dot-notation, custom separators), but is silent on edge cases beyond the schema example: whether array indices become keys, how key collisions are resolved, or what happens on invalid JSON input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste: the core transformation and example come first, with unflatten and custom separators trailing as qualifications. No boilerplate or repetition of schema text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and 100% parameter coverage, the description need not explain return values, and it correctly covers both modes. The only shortfall is unstated behavior for arrays/collisions and error handling, which matters for a JSON transform that receives arbitrary shapes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so mode, input, and separator are already fully documented in the schema, including defaults. The description only restates the separator support and implies the mode switch, adding no syntax or edge-case meaning beyond the structured fields. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (flatten/unflatten) and resource (nested JSON object) with an inline example transformation, and covers both directions in one definition. An agent can immediately tell this apart from format_json, extract_json_path, or json_to_csv without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear conditions for each direction: nested input → dot-notation keys, dot-notation input → nested object. It does not name an alternative sibling (e.g., when to prefer format_json for pretty-printing), so there are no explicit exclusions, but the mode selection context is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

format_bytesA
Read-onlyIdempotent
Inspect

Convert raw byte counts to human-readable sizes in SI (KB=1000) or IEC (KiB=1024) units, or parse size strings back to bytes. Covers B, KB/KiB, MB/MiB, GB/GiB, TB/TiB, PB/PiB.

NameRequiredDescriptionDefault
bytesNoNumber of bytes to format
standardNoOutput standard (default: both)
size_stringNoSize string to parse to bytes (e.g. "1.5 GB", "512 MiB")

Output Schema

NameRequiredDescription
bytesNo
originalNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered. The description adds the meaningful semantic distinction of SI vs IEC bases, but says nothing about precedence when both 'bytes' and 'size_string' are supplied, or about the 'both' default behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, capability and unit scope front-loaded. Every clause carries information the agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values need no explanation, and all three parameters are documented. The one gap is the relationship between the 'bytes' and 'size_string' inputs, which an agent must guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description genuinely adds value the schema lacks: it defines what 'si' and 'iec' actually mean (KB=1000 vs KiB=1024) and enumerates the supported unit suffixes, which the enum alone does not explain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific bidirectional capability: convert byte counts to human-readable sizes AND parse size strings back to bytes, naming the exact unit families covered. An agent can immediately distinguish this from siblings like number_base_convert or format_table.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'or parse size strings back to bytes' implies the two modes (bytes input vs size_string input), but there is no explicit statement of when to use which, that the two input paths are mutually exclusive, or how this differs from related conversion tools. Usage is inferable but not guided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

format_json
Read-onlyIdempotent
Inspect

Validate and pretty-print a string that is ALREADY valid JSON. Strict by design — it is a validity gate: valid JSON comes back formatted, anything else is rejected with the exact parse error. It never repairs, completes, or guesses. NOT for: plain text or prose (will fail), JSON embedded in markdown/prose (extract_json_from_text pulls it out), JS objects (JSON.stringify them first), YAML (yaml_to_json reads it).

NameRequiredDescriptionDefault
inputYesA raw JSON string, e.g. '{"key":"value"}'. Must already parse as JSON — plain text or truncated JSON is rejected, not repaired.
indentNoIndent size (default: 2)

Output Schema

NameRequiredDescription
validNo
formattedNo
format_tableA
Read-onlyIdempotent
Inspect

Convert a JSON array of objects into a Markdown table. Automatically detects columns, aligns headers, and fills missing keys with empty cells. Use when an agent needs to present structured data — tool results, model comparisons, test reports — as a readable table in a response or document.

NameRequiredDescriptionDefault
inputYesThe array of objects to render — a JSON string, or the array itself.
columnsNoColumn names and order (default: all keys from first row)

Output Schema

NameRequiredDescription
rowsNo
tableNo
columnsNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered; the description goes further by disclosing that columns are auto-detected, headers aligned, and missing keys rendered as empty cells — defaults an agent could not infer from the schema. No detail on how nested or non-object values are handled, but for a pure transform that is minor.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero padding: the operation and its automatic behaviors come first, the usage trigger second. Every clause carries actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering safety, the description needs only purpose, behavior, and usage — all present. A small gap remains around edge-case handling (non-object array elements, nested values), which matters for a formatting tool handling arbitrary JSON.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including the 'default: all keys from first row' note for columns, so the baseline of 3 applies. The description adds only the missing-key fill behavior and nothing new about the input/columns parameters themselves.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Convert a JSON array of objects into a Markdown table') and adds concrete rendering behavior (column detection, header alignment, empty-cell filling). It is effectively distinguishable from siblings like json_to_csv or html_to_markdown, but never names or explicitly contrasts an alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use when an agent needs to present structured data — tool results, model comparisons, test reports — as a readable table in a response or document' gives a clear triggering context with concrete examples. No when-not-to-use clause or named alternative is provided, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

function_call_validateA
Read-onlyIdempotent
Inspect

Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema (types, enums, ranges), no extra/missing args. Reads the definitions as OpenAI (parameters), MCP (inputSchema) or Anthropic (input_schema) write them, or as plain required_args/optional_args; reads the call as arguments (object or JSON string), input (Anthropic tool_use) or an OpenAI {function:{…}} wrapper.

NameRequiredDescriptionDefault
function_callYesThe function call object from LLM (e.g. { "name": "get_weather", "arguments": {"city":"Paris"} })
allowed_functionsYesList of allowed function definitions, in OpenAI, MCP or Anthropic form, or as {name, required_args, optional_args}

Output Schema

NameRequiredDescription
validNo
errorsNo
error_countNo
function_nameNo
provided_argsNo
required_argsNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the safety profile is covered. The description adds useful behavioral context by enumerating the accepted input dialects (OpenAI `parameters`, MCP `inputSchema`, Anthropic `input_schema`, plain required/optional args) and call shapes, though it says nothing about failure reporting (which the output schema covers).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, and every subsequent clause adds distinct information about accepted formats. It is a single dense run-on sentence, which hurts scannability slightly but wastes no content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, both parameters fully described, and the accepted definition/call dialects enumerated, an agent has everything needed to invoke this correctly. Nothing material is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by spelling out the alternative shapes each definition and call object may take, which materially helps an agent construct a valid `allowed_functions` entry or unwrap a `{function:{…}}` call.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (validate) and resource (LLM function call / tool_use output), then enumerates exactly what is checked: name in allowed list, argument types/enums/ranges, extra/missing args. This clearly separates it from siblings like validate_mcp_response and validate_agent_trajectory.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the scope explicit — validating function-call output against definitions — which implies when to reach for it versus response/trajectory validators. It does not, however, name those sibling tools or state exclusions, so it stops short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_ci_workflowA
Read-onlyIdempotent
Inspect

Generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA: "eval_contract" runs a .ia-eval.yaml through ia-qa-com/eval-action@v1 (needs a provider key as a repo secret), "cli_checks" runs deterministic tools via npx @ia-qa/cli and fails the build on their verdict, "selector_drift" gates E2E selectors with @ia-qa/self-healing. Deterministic template, no LLM. Returns the YAML, the secrets to create and the next steps. Pair with generate_eval_yaml for the contract.

NameRequiredDescriptionDefault
cronNoFor schedule (default '0 6 * * 1').
gateNoDefault eval_contract. both = CLI checks + eval; all = CLI checks, drift, eval.
providerNoDecides which repo secret is wired (default groq).
triggersNoDefault push + pull_request.
cli_toolsNoTools to gate on, e.g. ["secret_scan","prompt_injection_scan"].
min_scoreNoOverride the contract min_score (0-100).
app_base_urlNo[selector_drift] Default http://127.0.0.1:3000; must match .ia-qa/config.json.
fail_on_failNoDefault true; false reports without gating.
node_versionNoDefault "20".
contract_pathNoDefault evals/smoke.ia-eval.yaml.
workflow_nameNo
app_start_commandNo[selector_drift] Boots the app, e.g. "npm run start:ci". Never guessed: omitted, the step is a TODO.

Output Schema

NameRequiredDescription
gateNo
pathNo
yamlNo
notesNo
next_stepsNo
secrets_requiredNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds useful behavioral context beyond annotations: it is a deterministic template with no LLM, requires a provider key as a repo secret, and returns YAML, secrets to create, and next steps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and then efficiently explains the gate modes, deterministic behavior, return values, and pairing recommendation. Every clause contributes useful selection or invocation context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 12 optional parameters, 92% schema description coverage, annotations, and an output schema, the description is complete enough. It covers the tool's purpose, main modes, secret requirement, output expectations, and sibling pairing without redundantly documenting every parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 92%, so the schema does most parameter-level work. The description still adds meaning by explaining the gate modes, noting that provider determines which repo secret is wired, and clarifying that app_start_command is never guessed and becomes a TODO if omitted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA. It distinguishes the three gate modes (eval_contract, cli_checks, selector_drift) and names a sibling tool, generate_eval_yaml, so the agent can differentiate it from alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear context for when to use the tool and how gate modes map to CI behaviors, and it explicitly says to pair it with generate_eval_yaml for the contract. However, it does not name when to use this versus run_eval_contract or other eval-running siblings, so exclusions are implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_curlA
Read-onlyIdempotent
Inspect

Generate a curl command from request parameters. Supports GET/POST/PUT/DELETE, custom headers, JSON body, and form data. Useful for documentation, sharing, and debugging API calls.

NameRequiredDescriptionDefault
urlYesRequest URL (must be http/https)
bodyNoRaw request body string
methodNoHTTP method (default: GET)
headersNoRequest headers as key-value object
verboseNoAdd -v for verbose output (default: false)
body_jsonNoJSON body (auto-adds Content-Type: application/json)
follow_redirectsNoFollow redirects with -L flag (default: true)

Output Schema

NameRequiredDescription
urlNo
curlNo
methodNo
header_countNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/idempotentHint/non-destructive and openWorldHint=false, so the safety profile is covered; the description adds that it only synthesizes a command string for the listed methods rather than executing anything. It still doesn't explicitly say no request is sent, which is the one assumption an agent might get wrong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three front-loaded sentences with zero filler: purpose, capabilities, then use cases. Slightly more efficient than needed but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description rightly does not explain the returned curl string. All required elements are covered; only the 'form data' claim and the no-execution guarantee would need tightening for a top score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so values, defaults, and flags (verbose, follow_redirects) are already documented. The description adds little beyond the schema and even mentions 'form data', which has no corresponding documented parameter — body/body_json cover raw and JSON payloads. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and concrete artifact: 'Generate a curl command from request parameters.' No sibling tool offers the same output, and the description immediately names the supported surface (methods, headers, body, form data), so an agent can tell what it produces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear scenarios where this applies ('documentation, sharing, and debugging API calls') but does not say when not to use it or point at an alternative (e.g., cors_test, http_status_lookup). For a fairly unique tool the lack of exclusions is a minor gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_eval_yamlA
Read-only
Inspect

Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do. Uses Groq openai/gpt-oss-20b (server-side, no API key needed). Returns ready-to-run YAML for the LLM Test Runner (run_eval_contract). Picks appropriate evaluators (cosine_similarity, contains_check, hallucination_check, etc.) based on the task type.

NameRequiredDescriptionDefault
task_typeNoOptional task type hint to guide evaluator selection.
descriptionYesPlain-language description of what the LLM under test should do. Be specific: describe inputs, expected behaviour, and constraints.
system_promptNoOptional system prompt of the LLM under test. Helps generate more accurate test cases.
scenario_countNoNumber of scenarios to generate (default: 5). Covers happy path + edge cases + adversarial.

Output Schema

NameRequiredDescription
yamlNo
task_typeNo
model_usedNo
scenario_countNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint=true, destructiveHint=false), so the description earns credit for adding deployment context the annotations don't: it runs server-side via Groq openai/gpt-oss-20b with no API key, and it auto-selects evaluators (cosine_similarity, contains_check, hallucination_check). That explains the openWorldHint dependency. It does not mention latency, rate limits, or that output varies run-to-run despite idempotentHint=false.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the verb and artifact, then progressively adding model, return value, and evaluator behavior. No filler and no repetition beyond the necessary naming of the consumer tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return-value details are unnecessary, and annotations handle the safety profile; the description covers what's left — external model, no auth, downstream use. Minor gap: nothing about failure modes (e.g. what happens with a vague description) or how deterministic the generated YAML is.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are fully documented in the schema itself, including defaults and enum values. The description adds one genuinely useful link — that task_type drives evaluator selection — but gives no guidance on system_prompt or scenario_count trade-offs. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and artifact: generate a complete .ia-eval.yaml evaluation contract from a plain-language description. It also names the downstream consumer (run_eval_contract / LLM Test Runner), which separates it cleanly from siblings like generate_test_cases or prompt_test_suite that produce different artifacts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the usage context clear: you have a plain-language spec of an LLM's behavior and want a runnable eval contract. Naming run_eval_contract as the follow-up step is a useful routing signal. It stops short of explicit exclusions — it never says when to prefer generate_test_cases, prompt_test_suite, or json_schema_generate instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_hmacA
Read-onlyIdempotent
Inspect

Compute an HMAC signature for a message using a secret key. Supports SHA-256 (default), SHA-512, SHA-1, and MD5. Used for API request signing, webhook verification (GitHub, Stripe, Twilio), and JWT validation.

NameRequiredDescriptionDefault
secretYesSecret key
messageYesMessage to sign
encodingNoOutput encoding (default: hex)
algorithmNoHash algorithm: sha256 (default), sha512, sha1, md5

Output Schema

NameRequiredDescription
hmacNo
encodingNo
algorithmNo
message_lengthNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is fully covered by structured data. The description adds that the operation is keyed and lists supported algorithms with SHA-256 as default, but says nothing about determinism guarantees, output format, or how invalid algorithms are handled — modest added value given the annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences: the core action first, then supported algorithms with the default, then concrete use cases. Every sentence carries distinct information and nothing is repeated for padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a pure deterministic computation with full annotation coverage, 100% schema coverage, an examples block, and a declared output schema, the description supplies everything an agent needs: what it computes, the algorithm options, and the realistic use cases. Return-value details are correctly left to the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents secret, message, encoding (hex default), and algorithm with its defaults and options. The description restates the algorithm list and default without adding format, length, or constraint guidance, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Compute an HMAC signature for a message using a secret key' — which distinguishes it from plain hashing siblings like hash_text by making the secret-key requirement explicit. It does not name any sibling tool directly, so differentiation relies on the reader inferring the keyed-vs-unkeyed distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists scenarios (API request signing, webhook verification for GitHub/Stripe/Twilio, JWT validation), which implies when the tool is appropriate. However, it never states when NOT to use it or points to alternatives such as hash_text for unkeyed digests or decode_jwt for JWT inspection, leaving the routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_html_reportA
Read-onlyIdempotent
Inspect

Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, evaluator score bars, and a radar chart. Returns the HTML as a string.

NameRequiredDescriptionDefault
resultsYesThe JSON object returned by run_eval_contract()

Output Schema

NameRequiredDescription
htmlNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so the safety profile is covered. The description adds meaningful behavior beyond that: it is a pure transformation whose output is a self-contained HTML string, which tells the agent not to expect a file write. It does not mention output size or that assets are inlined rather than linked.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the transformation and then the return value; the feature list is dense but each item adds selection value. Slightly list-heavy but no filler sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one required object parameter, full schema coverage, rich annotations, and an output schema in place, the description covers what an agent needs to call it correctly. Only marginal gaps remain (output size, whether the result object shape is validated).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter's schema text ('The JSON object returned by run_eval_contract()') matches the description's wording, so the description adds no new syntax or structural detail. Baseline 3 applies since the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (convert) and a precise input resource (run_eval_contract() LLM Test Runner JSON result) plus the exact output artifact (self-contained dark-themed HTML report). The enumerated report contents (Pass/Fail badges, Input/Output/Ground-Truth panels, score bars, radar chart) make it unmistakable versus siblings like ab_test_report or format_table.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is explicit: it consumes the JSON returned by run_eval_contract(), which tells the agent this is the step that follows that tool. No exclusions or alternative reporting tools (e.g. ab_test_report) are named, so the agent must infer when a different report generator is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_json_ldA
Read-onlyIdempotent
Inspect

Generate a ready-to-paste snippet for GEO / structured data optimization. Supported types: WebSite, FAQPage, Article, Person, Organization, SoftwareApplication, HowTo.

NameRequiredDescriptionDefault
typeYesSchema @type: "WebSite", "FAQPage", "Article", "Person", "Organization", "SoftwareApplication", "HowTo"
fieldsNoSchema fields as key-value pairs (name, url, description, author, datePublished, etc.)
faq_itemsNoFor FAQPage/HowTo: array of { question, answer } objects

Output Schema

NameRequiredDescription
nameNo
schemaNo
snippetNo
acceptedAnswerNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the generator's safety profile is known. The description adds that output is a 'ready-to-paste' snippet (return-format context), but doesn't say whether output is validated against schema.org or how missing/unsupported fields are handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly-packed sentences with the action and output form front-loaded, followed by the supported-type list. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and 100% parameter coverage, the description only needs to frame purpose and supported types, which it does. Complete enough for an agent to call correctly, with minor room for noting type-to-parameter mapping (e.g., faq_items only for FAQPage/HowTo).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline applies. The description repeats the supported type values that the schema already enumerates for the 'type' parameter, adding no new semantics for 'fields' or 'faq_items' beyond what the schema documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Generate) and resource (ready-to-paste JSON-LD script snippet) plus the domain purpose (GEO/structured data optimization). An agent can immediately tell it produces structured-data markup, though it doesn't distinguish itself from nearby generators like json_schema_generate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for GEO / structured data optimization' implies when to use it, but there are no explicit conditions, exclusions, or named alternatives among the many sibling generators. Usage is inferrable rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_passwordA
Read-only
Inspect

Generate a cryptographically secure random password using crypto.randomBytes. Configurable length (4–128), uppercase letters, digits, and symbols. Use when resetting user passwords, seeding test accounts, or generating API secrets.

NameRequiredDescriptionDefault
lengthNoPassword length (4–128, default: 16)
numbersNoInclude digits (default: true)
symbolsNoInclude symbols like !@#$ (default: false)
uppercaseNoInclude uppercase letters (default: true)

Output Schema

NameRequiredDescription
lengthNo
passwordNo
charset_sizeNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/non-destructive/not-idempotent, and the description reinforces the key trait by naming the cryptographic randomness source. The length bounds (4–128) are additional useful context; it doesn't say what happens on out-of-range input, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and implementation, followed by configuration and usage. Zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be spelled out, and annotations cover the safety profile. The remaining gap is edge-case behavior (invalid length handling, character-class guarantees), which is minor for this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters, their defaults, and the 4–128 range are already documented. The description restates the option list and range without adding syntax or behavioral nuance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Generate a ... random password') and even names the entropy source (crypto.randomBytes). No sibling tool overlaps, so an agent can select it unambiguously.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the three scenarios for using it: resetting user passwords, seeding test accounts, and generating API secrets. No exclusions or alternatives are given, but no near-equivalent sibling exists to route away from.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_slugA
Read-onlyIdempotent
Inspect

Convert any string into a URL-friendly slug: lowercase, ASCII-normalized (é→e), special characters removed, spaces replaced with hyphens. Use for generating SEO-friendly URL paths, file names, or identifier keys from user-provided titles or labels.

NameRequiredDescriptionDefault
inputYesString to slugify
separatorNoSeparator character (default: "-")

Output Schema

NameRequiredDescription
slugNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description adds genuine semantic detail beyond annotations – the exact normalization pipeline (lowercase, ASCII-fold, strip special chars, hyphenate spaces) – which tells the agent precisely what the output will look like, though it says nothing about edge cases like empty results or length limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, with the core transformation front-loaded before the use-case guidance. Every clause contributes distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple pure-function tool with an output schema present, a complete parameter schema, and full annotations, the description gives everything an agent needs to invoke it correctly. Return-value explanation is rightly omitted since the output schema exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both 'input' and 'separator' are documented already. The description's mention of spaces being replaced with hyphens loosely relates to the separator parameter but adds no real new semantics beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Convert') and resource ('any string into a URL-friendly slug') and then enumerates the exact transformation rules. An agent can distinguish it from siblings like case_convert or url_encode purely from the description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the use cases ('SEO-friendly URL paths, file names, or identifier keys from user-provided titles or labels'), giving clear context for when to reach for it. It stops short of naming when NOT to use it or pointing to alternatives, so it does not reach the top tier.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_test_casesA
Read-onlyIdempotent
Inspect

Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description. Declared constraints drive the boundaries: a length or numeric bound ("[8-64]", "min 8 chars", "at least 8 characters" — read from inputs, and from the feature prose when a sentence names exactly one field) yields the last accepted value AND the first rejected one; a format (email/url/uuid, from the type, the field name or the wording) yields malformed-value cases. A bound nobody declared is labelled as this tool's assumption, not as expected behaviour. gherkinFormat renders every case (cap 200, stated in the output) and gherkinScenarioCount lets you check it against totalCases.

NameRequiredDescriptionDefault
inputsNoOptional: list of input parameters (one per line, e.g. "email: string [required]", "password: string [required, min 8 chars]", "age: number [18-99]")
featureYesFeature or function to test. Be specific: describe inputs, expected behaviour, context. Constraints stated here ("password must be at least 8 characters") are used when the sentence names exactly one field.

Output Schema

NameRequiredDescription
featureNo
test_casesNo
parsedInputsNo
gherkinFormatNo
gherkinScenarioCountNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description is consistent with them while adding genuinely non-obvious behavior: undeclared bounds are flagged as the tool's own assumptions rather than expected behaviour, and gherkinFormat output is capped at 200 cases with the cap stated in the output. That assumption-labelling is a meaningful disclosure an agent would otherwise have to discover by experiment. It stops short of describing failure modes or what happens with an empty/ambiguous feature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is correctly front-loaded in sentence one, and most later content earns its place. But the final sentence drops `gherkinFormat` and `gherkinScenarioCount` without marking them as output fields, which reads as if they were parameters that do not exist in this schema, and the nested parentheticals (with quoted regex-ish examples) make the middle dense to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and both parameters are fully documented in the schema. The description supplies the interpretive rules an agent needs (constraint sources, assumption labelling, case cap) for a fairly complex generator. Minor gaps remain: no guidance on how large/rich the feature description must be, and the gherkin output fields are referenced without being identified as outputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes beyond it by explaining how `feature` prose is parsed for constraints (only when a sentence names exactly one field) and how `inputs`-declared bounds like "[8-64]" produce last-accepted and first-rejected values. That is real semantic guidance the schema does not contain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence gives a specific verb and resource — "Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description" — and the parenthetical enumerates the case categories, which is more than a restatement of the name. It does not distinguish itself from near-neighbours like jira_to_test_suite or prompt_test_suite, so an agent must infer the boundary rather than read it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the detailed mechanics (constraints in `inputs` or in the feature prose drive boundaries; undeclared bounds become labelled assumptions), but there is no explicit "use this when…" statement and no named alternative. The one implicit scoping rule — prose constraints count only "when a sentence names exactly one field" — is useful but is about interpretation, not tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_uuidA
Read-only
Inspect

Generate one or more cryptographically random UUID v4 identifiers. Use this when you need unique IDs for test fixtures, database records, session tokens, or any scenario requiring a guaranteed-unique string. Returns up to 100 UUIDs in one call.

NameRequiredDescriptionDefault
countNoNumber of UUIDs to generate (1–100, default: 1)

Output Schema

NameRequiredDescription
countNo
uuidsNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the safety hints in the annotations, the description provides two critical behavioral facts: that the identifiers are 'cryptographically random' and that it 'Returns up to 100 UUIDS in one call.' This maximum limit is vital operational context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is impeccably structured, moving logically from definition to use cases, and ending with a key performance constraint (the 100-limit) without unnecessary filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the strong input schema and comprehensive annotations, the description fills all remaining gaps by clarifying operational boundaries (UUID v4) and practical limits (100 maximum), making it complete for task execution.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the baseline is 3. The description adds value by explicitly framing the parameter `count` as the mechanism for requesting multiple UUID strings and reiterating the functional purpose (guaranteed-unique string).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is highly specific, stating it generates 'cryptographically random UUID v4 identifiers.' It immediately distinguishes itself from generic string utilities (like hashing or encoding tools among the siblings) by defining its exact function and use cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides excellent usage context by listing specific scenarios where the tool should be invoked ('test fixtures, database records, session tokens'). While it does not explicitly list alternatives not to use, its specialization makes the usage guidelines very clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_testing_guidelinesA
Read-onlyIdempotent
Inspect

Query the IA-QA methodology knowledge base. Returns structured testing guidelines, assertion strategies, thresholds, best practices, and relevant MCP tools for a given topic. Call without a topic to list all available topics. Topics: llm-unit-testing, rag-pipeline, prompt-stability, prompt-ab-testing, embedding-quality, eval-framework, semantic-testing, auto-testing, security, api-testing, ci-cd, multimodal, llm-data-security, agent-observability, pro-tips, learning-paths, golden-dataset, selector-drift, qa-recipes, playbooks. Not sure where to start testing an LLM, RAG pipeline or agent? Call without a topic (or with "start-here"): it maps what you are testing to the tools to call and the output field to gate CI on. A plain question such as "how do I test my RAG" also resolves to the right topic.

NameRequiredDescriptionDefault
topicNoThe testing topic to retrieve guidelines for. Omit to get the start-here map and the full list of available topics.

Output Schema

NameRequiredDescription
tipNo
topicNo
usageNo
keywordsNo
start_hereNo
available_topicsNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world), and the description adds real value beyond that: it discloses the return categories and the implicit routing/dispatch behavior for free-form questions. It stops short of describing result shape or size, but an output schema exists to carry that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The guidance sentences are front-loaded and purposeful, but the 19-item topic list duplicates the enum in the input schema verbatim, consuming most of the description's length without adding information. Trimming it would let the genuinely useful routing guidance stand out.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Annotations and an output schema mean the description needn't explain safety or return values, and it covers the main call patterns. However, the free-text-question promise conflicts with the strict enum, leaving an agent uncertain whether a natural-language topic will be accepted or rejected at validation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with an enum and its own description, so the schema already does the heavy lifting. The description adds the omit-to-list-all behavior, which is useful, but it also promises that free-text questions like "how do I test my RAG" resolve to a topic — something the enum constraint does not permit, creating ambiguity about what values are actually legal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ("Query the IA-QA methodology knowledge base") and enumerates what comes back: guidelines, assertion strategies, thresholds, best practices, and related MCP tools. This is clearly distinguishable from data-mutating siblings and from single-purpose guides like multimodal_eval_guide.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit call patterns: omit the topic to list everything, use "start-here" when unsure where to begin, and plain questions resolve to a topic. It also states the intent — mapping what you are testing to the tools to call and the CI gate field — so an agent knows exactly when to reach for it versus a direct tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

guardrail_testB
Read-onlyIdempotent
Inspect

Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex. Returns pass/fail per rule.

NameRequiredDescriptionDefault
rulesYesArray of guardrail rules to check
responseYesThe LLM response to test

Output Schema

NameRequiredDescription
passNo
ruleNo
labelNo
valueNo
detailNo
failedNo
passedNo
resultsNo
all_passedNo
total_rulesNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile needs no restating. The description adds the response contract ("Returns pass/fail per rule"), which is real value, but says nothing about how invalid or unrecognized rule types are handled, or whether evaluation short-circuits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the core action and closed with the return shape; nothing is padded. The single flaw is that the inline rule enumeration is long enough to be misleading rather than merely terse, given the enum mismatch.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the return values are covered, and the description correctly conveys the input contract at a high level. However, for a tool whose whole difficulty is authoring valid rules, the description neither enumerates the actual supported rule types nor clarifies the string-vs-object rule shape shown in the example, leaving the agent to reverse-engineer the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds little beyond restating rule categories. Worse, its enumeration is partly wrong – "language" is not in the type enum, and fully half the enum (min_length, starts_with, ends_with, word_count_max/min) goes unmentioned, while the example's bare-string shorthand "no_pii" is nowhere explained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Test an LLM response against a set of guardrail rules") and enumerates the constraint categories, so the agent knows this is a rule-based response validator rather than a scanner. It does not differentiate against adjacent siblings like shield_analyze, prompt_injection_scan or toxicity_scan, and two enumerated categories ("language", "forbidden patterns") have no matching rule type in the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the phrasing "Test an LLM response" – the agent can infer this is for validating generated text against explicit rule sets. There is no statement of when to prefer this over the many sibling scanners, nor any prerequisite or exclusion (e.g. what to do with an unknown rule type).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hallucination_check
Read-onlyIdempotent
Inspect

Lexical hallucination check: verifies an LLM answer's words, numbers and polarity against the provided source/context. Fast, deterministic, no API key needed. Each answer sentence is aligned to its best-matching source sentence, so a number only counts as support when it sits on the SAME statement ("founded in 1998" is not grounded by "sold 1998 units"), and a negation or antonym flip against that sentence returns verdict "contradicted" — the corrupted-fact hallucination that reuses source vocabulary. Limitations: still lexical — it cannot follow a paraphrase, a synonym, or multi-sentence reasoning, so a "well_grounded" verdict means "nothing lexical found", never "verified true". Entailment: run_semantic_tests (NLI/embedding) or a calibrated judge.

NameRequiredDescriptionDefault
casesNoBatch instead of answer + context: one result per case, and the worst gate overall.
answerNoThe LLM answer to verify (with context).
strictNoEvery sentence must be supported (default false).
contextNoThe source text that should ground it.

Output Schema

NameRequiredDescription
detailNo
messageNo
numbersNo
overlapNo
verdictNo
analysisNo
entitiesNo
groundedNo
sentenceNo
total_wordsNo
matched_wordsNo
contradictionsNo
grounded_countNo
unbound_claimsNo
grounding_scoreNo
total_sentencesNo
ungrounded_countNo
unsupported_claimsNo
hash_textA
Read-onlyIdempotent
Inspect

Compute a cryptographic hash of a text string. Use when you need to verify data integrity, generate content fingerprints, hash passwords (prefer SHA-256+), or produce a fixed-length digest of any input. Supports SHA-256 (default), SHA-512, SHA-1, and MD5.

NameRequiredDescriptionDefault
inputYesText to hash
algorithmNoHash algorithm: sha256 (default), sha512, sha1, md5

Output Schema

NameRequiredDescription
hashNo
algorithmNo
input_lengthNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the operation as read-only, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds useful context about supported algorithms and a security recommendation to prefer SHA-256+ for passwords, which goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core operation and followed by use cases and supported algorithms. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter utility with full schema coverage, an output schema, and comprehensive annotations, the description provides all the context an agent needs to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already documents both the input and the algorithm parameter with its default and options. The description mostly repeats the algorithm list; its only added semantic value is the password recommendation for SHA-256+.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Compute a cryptographic hash of a text string.' This is unambiguous and distinct from sibling utilities like base64_encode or generate_uuid.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear usage contexts: integrity verification, content fingerprints, password hashing, and fixed-length digests. It does not give explicit when-not-to-use or point to sibling tools, but the domain is narrow enough that alternatives are largely unnecessary.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

html_to_markdownA
Read-onlyIdempotent
Inspect

Convert HTML to clean Markdown. Strips scripts, styles, nav, ads, and comments. Converts headings, lists, links, images, code blocks. Ideal for preparing web content as LLM context.

NameRequiredDescriptionDefault
inputYesHTML string to convert
strip_linksNoStrip link URLs, keep text only (default: false)

Output Schema

NameRequiredDescription
markdownNo
markdown_lengthNo
original_lengthNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the readOnly/idempotent/non-destructive annotations by naming exactly what is discarded (scripts, styles, nav, ads, comments) and what is preserved (headings, lists, links, images, code blocks) – the key fidelity trade-off an agent needs. Doesn't mention behavior on malformed HTML or whether output is lossy for tables, so not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, each carrying distinct information: the operation, the strip/convert behavior, and the intended use case. Front-loaded and waste-free.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering the safety profile, the description's remaining burden is behavioral, and it covers stripping and conversion well. A note on handling invalid or empty input is the only meaningful gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both `input` and `strip_links` are already described in the schema. The description mentions link conversion but never references the `strip_links` option, adding no meaning beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource pair ("Convert HTML to clean Markdown") plus an explicit direction of conversion, which cleanly separates it from reverse-direction siblings like strip_markdown, escape_html and unescape_html. An agent can identify the operation without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Ideal for preparing web content as LLM context" gives a concrete usage context that maps to siblings like build_rag_prompt and optimize_prompt_tokens. No when-not-to-use guidance or named alternatives, so it stops short of 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

http_status_lookupA
Read-onlyIdempotent
Inspect

Look up detailed information about any HTTP status code: class, name, description, cacheability, typical causes, and handling best practices. Covers every code in the IANA HTTP Status Code Registry (1xx-5xx, including 226, 425, 451, 508, 511 and the WebDAV codes) with its defining RFC; anything outside the registry is reported as registered: false rather than described. cacheable means heuristically cacheable by default per RFC 9110 §15.1.

NameRequiredDescriptionDefault
codeYesHTTP status code (e.g. 200, 404, 429, 503)

Output Schema

NameRequiredDescription
codeNo
descNo
nameNo
classNo
cacheableNo
registeredNo
descriptionNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, so the safety profile needs no restating. The description instead adds genuinely non-obvious behavior: the IANA-registry boundary and the `registered: false` fallback for unknown codes, plus a precise definition of the `cacheable` field tied to RFC 9110 §15.1. Return-shape detail is deferred to the output schema, which is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the purpose and return fields, then the coverage boundary, then the one definitional clarification that would otherwise confuse an agent reading the `cacheable` field. Nothing is padded or repeated from annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter lookup with an output schema, an existing schema description, and full annotation coverage, the description supplies exactly the missing pieces: registry scope, out-of-registry behavior, and field semantics. Nothing an agent needs to call it correctly is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single `code` parameter already carries its own description and example (418), so the schema does the work. The description adds no format constraints beyond what the schema states (e.g. nothing about range validation or string-vs-number coercion). Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb plus resource ('Look up detailed information about any HTTP status code') with an explicit enumeration of what is returned (class, name, description, cacheability, causes, handling). No sibling tool overlaps this resource, so the agent can distinguish it immediately from the security-scanning siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The scope boundary is stated clearly: every code in the IANA registry is covered, and anything outside it comes back as `registered: false` rather than described — which tells the agent what happens for odd inputs like 999 or 0. There is no explicit 'use X instead' routing, but no sibling covers this resource, so the omission is minor.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

identify_callerA
Read-onlyIdempotent
Inspect

Returns what the server knows about the current MCP client: clientInfo captured during initialize, User-Agent, and any _meta fields sent with this request. Useful for debugging caller identification.

NameRequiredDescriptionDefault
_metaNoOptional self-identification. Keys: agent (string), model (string), version (string).

Output Schema

NameRequiredDescription
noteNo
sessionNo
meta_overrideNo
effective_agentNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is fully covered. The description adds provenance the annotations cannot: clientInfo is captured at initialize time while User-Agent and _meta come from the current request, which tells the agent these values may differ in origin. It does not mention whether the fields can be absent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the return value, with no repetition of schema details. The trailing usage sentence is short and earns its place as the only guidance present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, annotations are rich, and there is a single optional parameter, so the description only needs to establish purpose and provenance — which it does. Only the absence of any statement about missing/unavailable fields keeps it below 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the nested _meta properties (agent, model, version) are documented in the schema itself. The description only adds that _meta is 'sent with this request' and is echoed back, which is marginal extra meaning — baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Returns what the server knows about the current MCP client') and then enumerates exactly what comes back: clientInfo from initialize, User-Agent, and request _meta. No sibling tool overlaps with this capability, so an agent can identify it immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Useful for debugging caller identification' gives an implied use case but never says when not to use it or what alternative exists. There are no problematic alternatives in the sibling list, so the cost is low, but the guidance stops short of explicit conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

jira_to_test_suiteA
Read-only
Inspect

Turn a Jira ticket into a test suite: Gherkin scenarios, E2E steps, API tests, test data and the ambiguities to clarify. Pass the issue object, or Jira credentials to fetch it. It never invents: a thin ticket returns an empty suite with verdict INCOMPLETE_SPEC and the questions to ask. test_suite._gherkin_warnings is a deterministic syntax check. BYOK: your LLM key.

NameRequiredDescriptionDefault
issueNoThe ticket: key, summary, description, and optionally status, issue_type, priority, labels, comments (fetch_jira_issue returns this shape).
modelYese.g. "gpt-4o-mini", "claude-haiku-4-5-20251001", "gemini-3.5-flash".
api_keyYesYour LLM provider key.
issue_keyNoe.g. "PROJ-123", to fetch with the jira_* fields.
jira_emailNo
jira_tokenNo
max_tokensNoDefault 8192.
jira_base_urlNo
confluence_pagesNoPages from fetch_confluence_page, as documentation.

Output Schema

NameRequiredDescription
summaryNo
issue_keyNo
issue_urlNo
latency_msNo
model_usedNo
test_suiteNo
tokens_usedNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds substantial context beyond the annotations: it guarantees no invention ('It never invents'), names the INCOMPLETE_SPEC verdict and empty-suite behavior for thin tickets, and states that test_suite._gherkin_warnings is a deterministic syntax check. It also discloses the BYOK requirement ('your LLM key'). It does not address the credential-precedence question or what happens on LLM call failure, which keeps it below 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core capability, then supporting guarantees, in roughly five compact sentences with no filler. The telegraphic style ('BYOK: your LLM key.') is efficient, though the abrupt internal reference to test_suite._gherkin_warnings could be clearer.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter tool with nested objects, an output schema present, and a non-trivial LLM-backed pipeline, the description covers the essential contract: inputs (issue or credentials), the non-invention guarantee, the INCOMPLETE_SPEC outcome, and the BYOK key requirement. The remaining gaps are precedence between issue and issue_key and failure/credential-error behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67% with the key parameters (issue, model, api_key, issue_key, max_tokens, confluence_pages) documented in-schema; the description reinforces the issue-vs-credentials split and the BYOK nature of api_key. However, jira_email, jira_token, and jira_base_url have no schema descriptions and the description only gestures at them collectively as 'Jira credentials', leaving no guidance on precedence between issue and issue_key. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Turn a Jira ticket into a test suite') and enumerates concrete outputs: Gherkin scenarios, E2E steps, API tests, test data, and clarification questions. This clearly distinguishes it from nearby siblings like generate_test_cases, prompt_test_suite, and fix_gherkin, which are single-purpose generators or linters.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly documents the two entry paths: 'Pass the issue object, or Jira credentials to fetch it', and describes the degraded path ('a thin ticket returns an empty suite with verdict INCOMPLETE_SPEC'). It does not explicitly compare itself to the other test-generation siblings, so an agent must infer precedence.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_diff
Read-onlyIdempotent
Inspect

Compute a deep structural diff between two JSON values. Returns added, removed, and changed keys with dot-notation paths. Like git diff but for JSON objects, for API response regression testing. Arrays are not compared blindly by position: the same elements in a different order collapse to a single "reordered" change, and an array of records sharing a stable identity field (id, uuid, key, name…) is matched by that field, so paths read [id=42] and a moved record is not reported as N rewrites.

NameRequiredDescriptionDefault
afterYesThe modified JSON (after) — a JSON string, or the value itself.
beforeYesThe original JSON (before) — a JSON string, or the value itself.
max_depthNoMax nesting depth to recurse (default: 10)

Output Schema

NameRequiredDescription
addedNo
changesNo
removedNo
modifiedNo
identicalNo
total_changesNo
json_schema_generateA
Read-onlyIdempotent
Inspect

Infer a JSON Schema (draft-07) from a sample JSON value. Detects types, required fields, array item shapes, nested objects, and common string formats (email, uri, date, date-time, uuid). Returns a ready-to-use schema compatible with json_schema_validate. Use when you have a sample API response or LLM output and want to auto-generate a validation schema for CI/CD testing.

NameRequiredDescriptionDefault
inputYesThe sample JSON value to infer the schema from — a JSON string, or the value itself.
samplesNoMore samples of the same shape, merged with input: a field seen as null and as a date becomes ["string","null"]; one missing somewhere is not required.
required_allNoMark fields present in every observed object as required (default: true)

Output Schema

NameRequiredDescription
schemaNo
warningsNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is covered. The description adds real inference behavior (which constructs get detected, the draft-07 target, forward compatibility with the validator), but says nothing about edge cases such as heterogeneous samples, empty input, or how conflicts are resolved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core action, then detection scope, then the use case. Every sentence carries distinct information with no restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is unnecessary, and the description correctly covers behavior, scope and downstream use instead. Only the lack of guidance on degenerate or conflicting inputs keeps it below full marks for a pure-function inference tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters, including the subtle null/date merge behavior of 'samples' and the required_all default, are already documented. The description's mention of 'required fields' loosely echoes required_all but adds no syntax or format guidance beyond the schema — baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Infer a JSON Schema (draft-07) from a sample JSON value' — and names the sibling it complements ('compatible with json_schema_validate'), so an agent can separate it from json_schema_validate and mock_from_schema. The list of inferred features (types, required fields, array shapes, string formats) makes the output concrete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear triggering context: 'Use when you have a sample API response or LLM output and want to auto-generate a validation schema for CI/CD testing.' It links forward to json_schema_validate, but never states when not to use it or contrasts it with mock_from_schema, which also consumes schemas.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_schema_validateA
Read-onlyIdempotent
Inspect

Validate a JSON value against a JSON Schema (draft-07 subset). Supports type, required, properties, items, enum, const, pattern, format (email/uri/date/date-time/uuid), minimum/maximum, minLength/maxLength, minItems/maxItems, uniqueItems, additionalProperties, anyOf, allOf, oneOf, not, and local $ref (#/definitions/…, #/$defs/…). A schema using a keyword outside that list (patternProperties, if/then, dependencies, a remote $ref…) is never reported valid: the keyword is named in unevaluated_keywords. Returns all validation errors with dot-notation paths.

NameRequiredDescriptionDefault
valueYesThe JSON value to validate — a JSON string, or the value itself.
schemaYesThe JSON Schema — a JSON string, or the schema object itself.

Output Schema

NameRequiredDescription
validNo
errorsNo
error_countNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the read-only/idempotent/non-destructive safety profile, and the description goes well beyond them with a genuinely non-obvious behavior: schemas using unsupported keywords are never reported valid and the offending keyword is surfaced in `unevaluated_keywords`. It also discloses that all errors are returned with dot-notation paths, giving the agent a concrete picture of the response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence and the fail-closed caveat follows logically. The long keyword enumeration is dense but earns its place because it defines the supported subset precisely; it is the one part that could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain return values, yet it still flags the two semantically important response fields (`unevaluated_keywords` and error paths). Combined with 100% parameter coverage and clear annotations, an agent has everything needed to call it correctly and interpret conservative results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the two parameter descriptions already explain the 'JSON string or value itself' polymorphism. The description adds nothing further about parameter shapes or edge cases, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb+resource: validate a JSON value against a JSON Schema, with the dialect version (draft-07 subset) named. It is clearly distinguishable from siblings like json_schema_generate (produces schemas) or openapi_validate (different document type).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the applicable context clear by enumerating exactly which schema keywords are supported, which implicitly tells the agent when this tool will give a trustworthy answer. It does not, however, explicitly name alternatives such as openapi_validate or json_schema_generate, nor state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_to_csvA
Read-onlyIdempotent
Inspect

Convert a JSON array of objects to CSV format. Automatically detects columns from all object keys. Handles quoting and escaping per RFC 4180.

NameRequiredDescriptionDefault
inputYesThe array of objects to convert — a JSON string, or the array itself.
headersNoInclude header row (default: true)
delimiterNoColumn delimiter (default: ",")

Output Schema

NameRequiredDescription
csvNo
rowsNo
columnsNo
column_namesNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so safety is covered. The description adds real behavioral value — column detection across all object keys and RFC 4180 quoting/escaping — but stays silent on how nested objects, arrays, nulls, or missing keys are rendered, which is the main behavioral unknown for a converter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the conversion direction, then the column-detection rule and the escaping guarantee. Nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need no explanation, and annotations plus full schema coverage handle safety and parameters. The one gap is nested/array value handling and header behavior for heterogeneous keys, which an agent would want to know before calling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter (input, headers, delimiter) carries its own default and type documentation. The description adds no parameter-level detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Convert a JSON array of objects to CSV format'), making the direction of conversion unambiguous and implicitly the inverse of the sibling parse_csv. It stops short of naming that sibling explicitly, so the differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the name and the one-line purpose — an agent can tell this is the JSON-to-CSV direction. There is no explicit statement of when to prefer it over parse_csv or yaml_to_json, and no exclusions or preconditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_to_yamlA
Read-onlyIdempotent
Inspect

Convert a JSON object to clean, human-readable YAML. Handles nested objects, arrays, multiline strings, and special characters. No external dependencies.

NameRequiredDescriptionDefault
inputYesThe JSON to convert to YAML — a JSON string, or the value itself.
indentNoIndentation size in spaces (default: 2)

Output Schema

NameRequiredDescription
yamlNo
linesNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds real capability scope beyond that — nested objects, arrays, multiline strings, special characters — which tells the agent what input shapes will round-trip cleanly. It omits behavior on malformed JSON, which is the main remaining gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action, with no repetition of the schema. 'No external dependencies' is the weakest line — mildly useful operationally but not decision-relevant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return values need no explanation, and parameters are fully covered by the schema. The description supplies the capability scope an agent needs for a two-parameter converter; only error-handling behavior is unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both the 'input' (string, object, or array) and 'indent' (default 2) parameters are fully documented in structured fields, and an example is included. The description adds no further semantics, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Convert a JSON object to ... YAML') with an unambiguous direction, which implicitly distinguishes it from the inverse sibling yaml_to_json. It does not explicitly name the sibling, but no agent could confuse the two.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the tool's nature as a simple format converter; there is no statement of when to prefer it over alternatives such as format_json or yaml_to_json, nor any preconditions. For a self-evident utility this is adequate but leaves routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

latency_benchmarkA
Read-only
Inspect

Measure response time of one or more HTTP endpoints (GET/POST). Runs N iterations and returns min/max/avg/p95 latency. Useful for API and MCP server benchmarking.

NameRequiredDescriptionDefault
endpointsYesEndpoints to benchmark. Accepts a single URL string, an array of URL strings, or an array of {url, method?, body?, headers?, label?} objects.
iterationsNoNumber of iterations per endpoint (default: 3, max: 10)

Output Schema

NameRequiredDescription
resultsNo
iterationsNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/openWorld/non-idempotent, so the safety bar is lower; the description still adds real value by stating that N requests are fired per endpoint and which statistics are computed. It omits timeouts, error handling on failed endpoints, and the auth/header requirements for targets. Note a mild tension worth surfacing: readOnlyHint=true but POST bodies are supported and repeated, so external endpoints may see state changes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and the returned metrics, with no redundancy or filler. The benchmarking-purpose sentence is the only soft one and still conveys applicability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need no explanation, and the annotations carry the safety profile. Remaining gaps are operational details an agent would want — per-endpoint timeout, behavior when an endpoint errors, and whether requests are sequential — but nothing essential to invoking it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters (endpoints and iterations, including the accepted string/array/object shapes) are already fully documented in the schema. The description adds only the parenthetical GET/POST mention and no syntax, limits, or defaults beyond what the schema states; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Measure response time of one or more HTTP endpoints") plus the metrics returned (min/max/avg/p95), which clearly separates it from siblings like http_status_lookup or rate_tool. It does not explicitly name or exclude any sibling, so it stops short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The trailing "Useful for API and MCP server benchmarking" implies the use case but gives no when-not conditions, prerequisites (auth headers, permissions), or alternative tools for related checks. The agent must infer that this is for timing rather than correctness or security testing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

levenshtein_distanceA
Read-onlyIdempotent
Inspect

Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings. Supports batch comparison. Useful for fuzzy string matching, deduplication, and test result comparison.

NameRequiredDescriptionDefault
aNoFirst string (single-pair mode)
bNoSecond string (single-pair mode)
batchNoBatch of {a,b} pairs (max 50)
case_insensitiveNoIgnore case differences (default: false)

Output Schema

NameRequiredDescription
aNo
bNo
modeNo
countNo
resultsNo
distanceNo
similarityNo
operations_neededNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds genuinely new behavior beyond that: batch comparison is supported and a normalized similarity ratio is returned alongside the raw distance. It stops short of stating any size or performance caveat for large strings.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences: capability first, then a behavioral note, then use cases. Front-loaded and waste-free, though the use-case sentence is the softest and borders on generic filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and all four parameters are documented in the schema. The remaining gap is that no required parameters exist, yet the description never says which inputs to supply for single-pair versus batch mode, leaving a plausible mis-invocation path.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including mode hints ("single-pair mode") and the batch cap of 50, so the baseline is 3. The description only reinforces the existence of batch mode and does not clarify the key ambiguity that zero parameters are required or how a/b interact with batch.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: computes Levenshtein edit distance plus a normalized similarity ratio, with batch support. It is clear what the tool does, but it never names or contrasts the closely-related siblings (similarity_score, diff_text, embedding_similarity, text_stats), so an agent cannot fully disambiguate on purpose alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Useful for fuzzy string matching, deduplication, and test result comparison" gives concrete when-to-use contexts. There are no when-not conditions and no mention of the alternative similarity/diff tools, so selection between siblings still relies on the agent's own knowledge.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lint_commit_messageA
Read-onlyIdempotent
Inspect

Validate a git commit message against the Conventional Commits spec (feat, fix, docs, style, refactor, test, chore, ci, perf, build). Returns compliance score, breaking change detection, and actionable suggestions.

NameRequiredDescriptionDefault
strictNoEnforce strict rules: max 72-char subject, imperative mood check (default: false)
messageYesGit commit message to validate

Output Schema

NameRequiredDescription
typeNo
scopeNo
scoreNo
validNo
checksNo
subjectNo
has_bodyNo
is_breaking_changeNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, lowering the burden on the description. The description adds value by stating it returns a compliance score, breaking change detection, and actionable suggestions, though it does not elaborate on strict-mode behavior beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no wasted wording. The parenthetical list of types is compact and directly supports the stated spec.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With full schema coverage, annotations covering safety, and an output schema present, the description is complete enough for an agent to call the tool correctly. It states the validation target, recognized types, and return contents.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both the message and strict parameters. The description lists recognized commit types but adds no meaning about the strict parameter or syntax beyond what the schema provides, making baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (validate) and resource (git commit message) and names the Conventional Commits spec plus recognized types. It is clearly distinguishable from sibling validation tools like validate_email or json_schema_validate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The usage context is implied: validate a commit message against Conventional Commits. However, the description does not explicitly state when to use this tool versus alternatives or when not to use it, so guidance remains minimal.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_llm_modelsA
Read-onlyIdempotent
Inspect

List all LLM models available on ia-qa.com with their provider, API endpoint, and capabilities. Filter by provider name (e.g. "Groq", "HuggingFace", "OpenAI") or return the full catalog. Use this to discover which models are available before calling an LLM API, or to compare providers.

NameRequiredDescriptionDefault
providerNoFilter by provider name (case-insensitive). E.g. "Groq", "HuggingFace", "OpenAI", "Anthropic", "Google", "DeepSeek", "xAI", "Ollama". Omit for full catalog.

Output Schema

NameRequiredDescription
totalNo
filterNo
modelsNo
providersNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world behavior, so the safety profile is covered. The description adds useful specifics beyond that: the catalog includes endpoints and capabilities, filtering is case-insensitive, and omitting the filter returns everything. It stops short of noting any rate limits or catalog freshness, which is acceptable for a simple read tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences with the core purpose front-loaded, followed by the filtering behavior and then usage guidance. Minor redundancy between the description's provider examples and the schema's identical examples, but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complexity is low (one optional parameter, output schema present, rich annotations), and the description covers purpose, filtering behavior, and intended usage. The only gap is sibling disambiguation against model_info and compare_models, which an agent would still have to resolve on its own.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single provider parameter already carries a full description with an example list. The description largely restates that same information (provider examples, omit for full catalog) without adding syntax or matching behavior beyond 'case-insensitive', which the schema also states. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List all LLM models available on ia-qa.com') and enumerates what comes back: provider, API endpoint, and capabilities. It does not differentiate itself from potentially overlapping siblings such as model_info, llm_fit_finder, or compare_models, which keeps it out of 5 territory.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear usage context: 'Use this to discover which models are available before calling an LLM API, or to compare providers.' It also states the filter-or-full-catalog choice explicitly. However, it names no alternatives or exclusions, so an agent choosing between this and model_info or compare_models gets no routing help.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_local_testsA
Read-onlyIdempotent
Inspect

Discover .ia-eval.yaml LLM test suite files in the project directory. Scans CWD and standard sub-directories (evals/, tests/, contracts/). Returns file paths ready to pass to run_eval_contract.

NameRequiredDescriptionDefault
dirNoDirectory to scan (defaults to server CWD)

Output Schema

NameRequiredDescription
dirNo
countNo
filesNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so safety is covered. The description adds useful context beyond that: it scans CWD plus evals/, tests/, contracts/ and returns paths – specifics the annotations do not convey. Minor gaps on recursion depth or hidden-dir handling keep it at 4.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the resource and pattern, then scan scope, then output purpose. No wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return values needn't be explained, and annotations cover the safety profile. The description fully equips an agent to call this for file discovery before run_eval_contract.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single dir parameter has 100% schema description coverage ('Directory to scan, defaults to server CWD'), so the schema already carries the semantics. The description only implies scanning behavior without adding syntax, path format, or override specifics, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (discover) and resource (.ia-eval.yaml LLM test suite files) with the exact file pattern, and explicitly names the sibling run_eval_contract as the consumer so an agent can distinguish the discovery step from the execution step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Returns file paths ready to pass to run_eval_contract' clearly implies the find-then-run workflow and positions this against that sibling. It doesn't explicitly state when NOT to use it or other alternatives, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_fit_finder
Read-onlyIdempotent
Inspect

Shortlist LLMs on facts, no ranking: cloud models with the features you need within a monthly budget, cheapest first; or local models that fit your VRAM, largest first, with the ollama command. A shortlist is not a choice: only your own cases show which model fits.

NameRequiredDescriptionDefault
modeNocloud (API models) or local (Ollama/self-hosted). Default: cloud
top_nNoNumber of models to return (default: 5)
vram_gbNoGPU VRAM in GB (only for mode=local). Default: 16
featuresNoRequired features: vision, function_calling, json_mode, streaming, reasoning
use_caseNoOnly "reasoning" filters (requires the feature); chatbot | code | rag | summarization | classification | agents | multilingual rank nothing
max_budgetNoMaximum monthly budget in USD (based on tokens_per_day)
quantizationNoQuantization (only for mode=local): Q4_K_M | Q8_0 | FP16. Default: Q4_K_M
tokens_per_dayNoEstimated daily token volume (default: 100000)

Output Schema

NameRequiredDescription
modeNo
noteNo
orderNo
resultsNo
vram_gbNo
use_caseNo
prices_as_ofNo
quantizationNo
tokens_per_dayNo
total_matchingNo
llm_format_checkA
Read-onlyIdempotent
Inspect

Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex. Essential for structured output testing.

NameRequiredDescriptionDefault
outputYesThe LLM output to validate
regex_patternNoCustom regex pattern (only when expected_format is "regex")
expected_formatYesExpected format

Output Schema

NameRequiredDescription
validNo
checksNo
failedNo
passedNo
total_checksNo
expected_formatNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so safety behavior is covered. The description adds the meaningful behavioral detail that validation covers nine distinct format families, but says nothing about strictness, what a failure result looks like, or how partial/malformed matches are treated beyond what the output schema presumably conveys.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the core action front-loaded and the format enumeration carrying real information rather than padding. Nothing could be removed without losing content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full parameter coverage, the description only needs to establish scope, which it does. The one gap is sibling disambiguation among the several other LLM-output validators, which matters in this crowded toolset.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters are already documented, and the top-level example demonstrates usage. The description's format list mirrors the expected_format enum without adding semantics for regex_pattern (e.g., anchoring, flags) that the schema does not already give, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Validate that an LLM output matches an expected format') and enumerates the supported formats, so an agent knows exactly what it checks. It does not differentiate itself from nearby siblings like llm_output_validator, llm_json_schema_check, or function_call_validate, which an agent could easily confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Essential for structured output testing' implies the usage context but never states when to pick this over llm_output_validator or llm_json_schema_check, nor any exclusions. The scenario is inferable but the routing decision is left to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_generateA
Read-only
Inspect

Generate text with an open-source model hosted on Groq, on the server's key — no API key needed. Models: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default "openai/gpt-oss-20b"). HuggingFace models are not served on the server key. A reasoning model that spends max_tokens thinking is retried once at 4096. Rate-limited.

NameRequiredDescriptionDefault
modelNoModel ID (default: "openai/gpt-oss-20b"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b. Other ids from list_llm_models, HuggingFace ones included, are BYOK-only and will be rejected.
promptYesThe user prompt / instruction to send to the model
systemNoOptional system prompt to set context or persona
max_tokensNoMaximum tokens to generate (default: 2048, max: 4096)
temperatureNoSampling temperature 0.0–1.5 (default: 0.7)

Output Schema

NameRequiredDescription
modelNo
usageNo
contentNo
providerNo
latency_msNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: discloses the keyless server-key model, a whitelist that rejects unlisted/HuggingFace ids, rate limiting, and a non-obvious retry behavior for reasoning models that exhaust max_tokens (retried once at 4096). No contradiction with readOnlyHint=true, openWorldHint=true, idempotentHint=false.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the capability and key constraint, then constraints, then failure modes — four dense sentences with little waste. The model enumeration duplicates the model parameter's schema description, which is the only redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-param generation tool with full schema coverage, an output schema, and annotations covering safety, nothing material is missing: cost/auth model, allowed models, limits, and retry behavior are all disclosed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so baseline is 3; the description adds real meaning by tying max_tokens to the reasoning-model retry at 4096 and by reinforcing that the model id must be on a server-keyed whitelist. It does not add syntax or interaction detail for temperature or system, so it isn't a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Generate text') plus the resource and hosting model ('open-source model hosted on Groq, on the server's key'), and enumerates the exact models available. An agent knows this is free server-keyed text generation, distinct from evaluation/analysis siblings like compare_models or run_semantic_tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear selection condition ('no API key needed' / server key available) and an explicit exclusion ('HuggingFace models are not served on the server key'), which routes the agent elsewhere for BYOK models. It stops short of naming which sibling to use instead for that case, but the constraint is actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_json_schema_checkA
Read-onlyIdempotent
Inspect

Validate that an LLM JSON output matches a JSON Schema definition. Tests required fields, types, enums, nested objects, and arrays. Critical for function-calling and structured output testing.

NameRequiredDescriptionDefault
outputYesThe LLM JSON output (raw string, will be parsed)
schemaYesJSON Schema (draft-07 subset) to validate against

Output Schema

NameRequiredDescription
validNo
errorsNo
error_countNo
parse_errorNo
parsed_typeNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so the safety profile is covered. The description adds the useful detail that the output is a raw string that gets parsed and enumerates which schema constructs are checked, but says nothing about failure behavior (throw vs. return error list) or what constitutes a pass/fail result beyond the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action before the enumerated checks and use case. Nothing is redundant, though the third sentence is more marketing than operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and both parameters are fully documented in the schema. What remains missing is sibling differentiation against a crowded field of JSON/LLM validators, which is the main gap for an agent choosing among them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented including 'raw string, will be parsed' and 'draft-07 subset'. The description adds no format, escaping, or subset-limit detail beyond what the schema states, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Validate that an LLM JSON output matches a JSON Schema definition', plus the concrete checks it performs (required fields, types, enums, nested objects, arrays). However, it never distinguishes itself from close siblings like json_schema_validate, llm_output_validator, or llm_format_check, so an agent cannot tell which to pick from the description alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Critical for function-calling and structured output testing' implies when the tool is relevant, but there is no explicit when-to-use vs. when-not, and no named alternative among the many overlapping validation siblings. The routing decision is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_output_validatorA
Read-onlyIdempotent
Inspect

Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safety patterns. Essential for QA testing LLM-powered features.

NameRequiredDescriptionDefault
outputYesThe LLM output text to validate
max_lengthNoMaximum character length for the output
min_lengthNoMinimum character length for the output
check_safetyNoCheck for PII patterns (emails, phones, SSN), profanity signals, and prompt leakage
must_includeNoComma-separated strings that MUST appear in the output
expected_formatNoExpected output format
must_not_includeNoComma-separated strings that must NOT appear (e.g. "TODO, FIXME, undefined, NaN")
check_json_schemaNoIf expected_format is JSON, provide required keys as comma-separated list to validate the structure
expected_languageNoExpected language of the output (en, fr, es, de…). Checks for common words.

Output Schema

NameRequiredDescription
totalNo
checksNo
failedNo
passedNo
verdictNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true and destructiveHint=false, so the safety profile is covered by structured data. The description adds useful scope detail (that it checks safety patterns and language detection) but says nothing about failure behavior — whether it stops at the first failed check, returns per-check results, or how severity is reported.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences, front-loaded with the verb and the resource, followed by a compact capability list. The closing 'Essential for QA testing' sentence is somewhat promotional filler but does establish context, so it only mildly dilutes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and annotations cover the read-only/idempotent profile. The description accounts for all nine parameters at a category level and is adequate for calling the tool, though it omits failure semantics and sibling differentiation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter carries its own description, including the comma-separated semantics of must_include and must_not_include and the enum for expected_format. The description merely echoes those categories at a high level, adding no syntax or interaction details (e.g. how check_json_schema relates to expected_format) beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Validate an LLM response') and enumerates the check families it covers: format, content rules, length, language, and safety. However, it never distinguishes itself from very close siblings such as llm_format_check, llm_json_schema_check, or response_quality_score, so an agent cannot tell which of these to reach for.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Essential for QA testing LLM-powered features' implies the context of use but gives no explicit when-to-use/when-not, no prerequisites, and no named alternatives. With several overlapping validator tools in the sibling list, this leaves routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lorem_ipsumA
Read-only
Inspect

Generate Lorem Ipsum placeholder text for UI mockups, design prototypes, or test data population. Configurable paragraphs (1–10), sentences per paragraph (1–20), and approximate words per sentence (3–30).

NameRequiredDescriptionDefault
paragraphsNoNumber of paragraphs to generate (1–10, default: 1)
words_per_sentenceNoApproximate words per sentence (3–30, default: 10)
sentences_per_paragraphNoSentences per paragraph (1–20, default: 5)

Output Schema

NameRequiredDescription
paragraphsNo
paragraph_countNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=false, so the safety profile is fully covered and the description need not repeat it. What the description omits is a behavioral detail annotations only imply: that output is regenerated non-deterministically each call (consistent with idempotentHint=false), so identical inputs may not yield identical text. Useful additional context, but thin.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste: the purpose and use cases lead, and the configurability note follows. Nothing is padded or repeated unnecessarily beyond the compact range restatement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description correctly need not explain return values, and annotations cover safety. The only meaningful omission is non-determinism of the generated text, which matters for a tool whose whole output is synthesized content.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter documents its own range and default. The description merely restates those ranges (1–10, 1–20, 3–30) without adding meaning such as how 'approximate' words per sentence is actualized or what the default is, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Generate Lorem Ipsum placeholder text') plus concrete use cases, which is far more informative than the bare name. It does not, however, distinguish itself from mock_from_schema, the closest sibling that also produces test/mock data, leaving a real selection ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear contexts for use (UI mockups, design prototypes, test data population), so an agent knows the intended scenario. It stops short of any when-not guidance or pointing at mock_from_schema for schema-driven mock data, so the boundary between the two is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_schema_lintA
Read-onlyIdempotent
Inspect

Lint one MCP tool definition with the same rules as mcp_server_health_check: errors break the MCP spec (FAIL), documentation_issues are what its strict mode refuses — a parameter without description or type (FIX, never an A) — and suggestions are style only (snake_case) and change nothing.

NameRequiredDescriptionDefault
tool_definitionYesMCP tool definition object with name, description, inputSchema

Output Schema

NameRequiredDescription
gradeNo
errorsNo
warningsNo
error_countNo
quality_scoreNo
warning_countNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnly, idempotent, non-destructive), so the description's value is in explaining the severity taxonomy: errors break the spec, documentation_issues are FIX-level and never an A, suggestions are style-only and change nothing. That tells the agent how to interpret the result, which is real added context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with no wasted preamble, and the core action ('lint one MCP tool definition') is front-loaded. The em-dash nesting of the three severity levels is slightly hard to parse but every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering the safety profile, the description only needs to explain the severity semantics — which it does for all three levels. Nothing essential for correct invocation or interpretation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and there is a single required parameter, so the schema already documents tool_definition. The description adds nothing about the parameter itself (e.g. that it must be an object with name/description/inputSchema), leaving that to the schema's example.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (lint) and resource (one MCP tool definition), which cleanly separates it from the server-wide mcp_server_health_check named in the text. The scope word 'one' is the key differentiator, though it is embedded in a clause about rules rather than stated as its own contrast.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The reference to mcp_server_health_check implies the split (one definition vs a whole server), but the description never says explicitly 'use this for a single tool, use that for a server.' The agent must infer the selection rule from 'one ... with the same rules as'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_server_evaluateA
Read-only
Inspect

Check that a live MCP server over Streamable HTTP speaks the protocol, 2026-07-28 or earlier: handshake or server/discover, tools/list and its definitions, error codes, one tools/call, structuredContent against outputSchema, latency. Returns PASS/FIX/BLOCK with a 0-100 score, or INCONCLUSIVE (score null) when authorization is required. The verdict covers the PROTOCOL, never whether tools answer correctly: what_this_does_not_prove lists what was not measured. Called: only test_tool_name, or a tool annotated readOnlyHint: true. A refusal of invented arguments is not scored (tool_behaviour: refused_placeholder); pass test_tool_args to exercise the tool. Private addresses and stdio are out of reach: paste tools/list into mcp_server_health_check.

NameRequiredDescriptionDefault
urlYesPublic URL of the MCP endpoint (e.g. https://mcp.example.com/mcp). An origin alone (https://www.ia-qa.com) tries /mcp, then /.
test_tool_argsNoReal arguments, sent to the tested server. Without them, required ones are invented.
test_tool_nameNoTool to call. Default: a read-only one, preferring no required arg.

Output Schema

NameRequiredDescription
scopeNo
scoreNo
checksNo
serverNo
latencyNo
verdictNo
endpointNo
next_stepsNo
tool_behaviourNo
what_this_provesNo
what_this_does_not_proveNo

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses the verdict vocabulary (PASS/FIX/BLOCK with 0-100 score, INCONCLUSIVE with null score when authorization is required), the scope limit that the verdict covers the protocol and never tool correctness, and the presence of a what_this_does_not_prove field. It also explains that a refusal of invented arguments is not scored (tool_behaviour: refused_placeholder), which prevents misreading a FIX result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose and scoping before the verdict semantics, and nearly every clause carries information. The single dense paragraph stacks many semicolon-separated constraints, which costs some scanability for an agent parsing it quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a live-network probe tool with an output schema present, the description supplies everything an agent needs: what is tested, what is not, the result vocabulary, the authorization edge case, and the safety constraint on which tool gets called. No gap remains that would cause a misinvocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already defines all three parameters, but the description still adds real meaning: why test_tool_args matters (without them required args are invented) and what happens with a bare origin URL. The read-only restriction on test_tool_name is reinforced here rather than left only in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — checks that a live MCP server over Streamable HTTP speaks the protocol (2026-07-28 or earlier) — and enumerates exactly what is probed: handshake/server/discover, tools/list, error codes, one tools/call, structuredContent vs outputSchema, latency. It also draws a sharp line against its nearest siblings, naming mcp_server_health_check for private addresses/stdio.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit calling constraint (only test_tool_name, or a tool annotated readOnlyHint: true) and tells the agent how to handle refused placeholder arguments by passing test_tool_args. It also states the out-of-scope conditions (private addresses, stdio) and routes those cases to mcp_server_health_check as the alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_server_health_checkA
Read-onlyIdempotent
Inspect

Check an MCP server's tool definitions without contacting it — for a server not deployed yet, a stdio server, or a CI gate. Paste the answer to tools/list (raw JSON-RPC or its result), a bare array of tools, one tool definition, or a manifest carrying name/description; input_shape says which one was read. Validates every tool against the MCP spec (name 1-128 chars of A-Z a-z 0-9 _ - .; camelCase and a tool with no parameters are valid; inputSchema of type object; required fields listed in properties) and, with strict, documentation (description ≥ 20 chars, every parameter with a description and a type). verdict: FAIL when a tool breaks a rule or no tool is found, WARN when only a manifest's server name/description is missing, PASS otherwise. A tools/list carries no server name, so those two checks are not scored on it (the name comes from initialize). Gate on verdict or on checks[tool_quality].pass.

NameRequiredDescriptionDefault
strictNoAlso require documentation: every description ≥ 20 characters, every parameter with a description and a type.
manifestYesJSON text: the answer to tools/list (raw JSON-RPC or its result), an array of tools, one tool definition, or a manifest with name/description/tools.

Output Schema

NameRequiredDescription
statsNo
totalNo
checksNo
failedNo
passedNo
verdictNo
toolIssuesNo
input_shapeNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent annotations: it enumerates the exact validation rules applied (name charset and length, inputSchema type object, required listed in properties), what strict adds, and the three-way FAIL/WARN/PASS verdict semantics. It even discloses a subtle scoring caveat — that a tools/list carries no server name, so those two checks are skipped.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then inputs, then rules, then verdict semantics — every sentence carries information. It is a single dense block with no line breaks or list structure, which makes the rule enumeration harder to scan than it needs to be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema handling return values and annotations covering the safety profile, the description supplies exactly the remaining gaps: accepted input forms, validation rules, strict-mode behavior, and verdict thresholds. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both `manifest` and `strict` are already documented in the schema, and the description largely restates them. It adds no syntax or format detail beyond what the schema provides, so this sits at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Check an MCP server's tool definitions") plus the defining constraint "without contacting it," which cleanly separates it from live-evaluation siblings like mcp_server_evaluate and from the narrower mcp_schema_lint. The enumerated accepted input shapes make the scope unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives three concrete when-to-use scenarios (server not deployed yet, stdio server, CI gate) and tells the agent how to gate on the result (verdict or checks[tool_quality].pass). It lacks an explicit "not for X, use Y instead" exclusion against siblings, so it stops short of the top band.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

merge_jsonA
Read-onlyIdempotent
Inspect

Deep merge two JSON objects. Supports three array strategies: replace (default), concat, or unique (dedup concat). Nested objects are recursively merged — override takes precedence for primitives.

NameRequiredDescriptionDefault
baseYesThe base JSON object (merged into) — a JSON string, or the object itself.
overrideYesThe override JSON object (takes precedence) — a JSON string, or the object itself.
array_strategyNoArray merge strategy: replace (default), concat, or unique

Output Schema

NameRequiredDescription
mergedNo
new_keysNo
total_keysNo
overridden_keysNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false. The description adds valuable merge semantics: recursive nested merging, override precedence for primitives, and three array strategies with 'replace' as the default. It stops short of describing error behavior for invalid JSON.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with the core action. Every sentence adds useful semantic detail with no repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a pure merge utility with full schema coverage, clear annotations, and an output schema. The description covers the important merge and array-handling behavior an agent needs before calling it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description still adds meaning beyond the schema by clarifying that nested objects are recursively merged and that override takes precedence for primitive values, plus that 'unique' means deduplicated concat.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Deep merge two JSON objects.' The operation is unambiguous and distinct from neighboring JSON utilities such as json_diff, flatten_json, and transform_json_array.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains mechanics but gives no explicit when-to-use guidance, prerequisites, or alternatives. It does not say when to choose merge_json over sibling JSON tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metamorphic_checkA
Read-onlyIdempotent
Inspect

Reference-free stability check: does an assistant's answer stay the same when the QUESTION is transformed (case, typo, paraphrase, reorder, translation)? You bring the outputs; no model is called (tfidf mode is deterministic and free). Default thresholds: case .95, typo .90, paraphrase .80, reorder .80, translation .75 (embeddings only), specialization .60 (advisory, never gated). Returns PASS / FAIL / INVALID — INVALID when the BASE answer is a refusal or too short, so a model that refuses everything never scores 1.0. Pair with run_semantic_tests for correctness.

NameRequiredDescriptionDefault
baseYesThe original run.
modeNotfidf (default, lexical: gate on case/typo, read paraphrase as a trend) or embeddings (OpenAI, needs api_key; required for translation).
api_keyNoOpenAI key, embeddings mode only.
variantsYesAnswers to transformed versions of the question.
thresholdsNoPer-relation overrides, 0-1. Calibrate on your own data before gating.
require_allNoDefault true: every gated variant must pass. Keep it for a gate — false lets one easy typo row hold the run at PASS while a paraphrase fails.
baseline_guardNoFloor on the BASE answer; failing it returns INVALID.

Output Schema

NameRequiredDescription
modeNo
summaryNo
verdictNo
weakestNo
baselineNo
variantsNo
warningsNo
thresholdsNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnly/idempotent/non-destructive and the description is consistent, adding context annotations cannot carry: no model invocation, deterministic vs paid mode, default thresholds, and the crucial INVALID rule when the BASE answer is a refusal or too short — which pre-empts a gaming failure mode. That is real behavior disclosure, not restatement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core question it answers, and every sentence carries information (modes, thresholds, verdict semantics, pairing guidance). It is dense with parentheticals and slightly run-on, which costs a little scannability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with nested objects and an output schema, the description covers everything an agent needs to decide and call: input expectations, mode trade-offs, threshold defaults, verdict meanings, and baseline guard behavior. Return-value detail is correctly delegated to the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description goes further by stating the actual default threshold values per relation (case .95, typo .90, paraphrase .80, reorder .80, translation .75, specialization .60 advisory) and the rationale behind require_all=true, which the schema does not provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific operation (reference-free stability check) on a specific resource (an assistant's answer under question transformations) and enumerates the transformations. It explicitly separates itself from run_semantic_tests (correctness) and implicitly from similarity/consistency siblings, so an agent can route without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Says what you must supply ('You bring the outputs; no model is called'), how the modes differ (tfidf deterministic and free vs embeddings needing api_key), and names the complement tool for correctness. It lacks an explicit 'do not use this when…' exclusion, so it stops short of full alternatives guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

minify_jsA
Read-onlyIdempotent
Inspect

Minify a JavaScript snippet, function, class, or module up to 50 KB using Terser. Returns minified code and byte savings. Use when embedding scripts in HTML templates, report payloads, or injecting inline code programmatically.

NameRequiredDescriptionDefault
codeYesJavaScript code to minify (max 50kb)

Output Schema

NameRequiredDescription
minifiedNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly, idempotent, non-destructive, closed-world. The description adds valuable context: the 50 KB cap, the Terser engine, and that it returns minified code plus byte savings. It does not discuss failure modes (e.g., invalid JS syntax) — a minor gap against an otherwise informative entry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: capability first, return contract second, usage guidance last. Zero filler; every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite an available output schema and rich annotations, the description still communicates the return shape, size limit, and use contexts, leaving no information gap for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter, fully documented in the schema (100% coverage), so the baseline is high. The description adds meaning by enumerating what 'code' can be (snippet, function, class, module) and reiterating the 50 KB limit, giving slightly more than the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (minify) and resource (JavaScript snippet/function/class/module), names the underlying engine (Terser), and gives a size ceiling of 50 KB. An agent can immediately distinguish this from siblings like format_json or html_to_markdown.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly lists when to use it: embedding scripts in HTML templates, report payloads, or injecting inline code. No explicit exclusions or named alternatives, but no sibling in the list is a direct substitute, so the context is quite clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mock_from_schema
Read-only
Inspect

Generate realistic mock data from a JSON Schema. Supports all common types (string, number, integer, boolean, array, object, null), format hints (email, date, date-time, uri, uuid), enum, const, and nested schemas.

NameRequiredDescriptionDefault
seedNoOptional seed string for deterministic output (uses first char codes)
countNoNumber of mock objects to generate (default: 1, max: 20)
schemaYesThe JSON Schema to generate from — a JSON string, or the schema object itself.

Output Schema

NameRequiredDescription
countNo
resultsNo
model_infoA
Read-onlyIdempotent
Inspect

Get detailed specs for an AI model: context window, pricing per 1K tokens, knowledge cutoff, provider, multimodal support, reasoning capabilities, and feature list. Covers 30+ models from OpenAI, Anthropic, Google, DeepSeek, Meta, Mistral, Cohere, xAI.

NameRequiredDescriptionDefault
modelYesModel name (e.g. "gpt-4o", "claude-3.5-sonnet", "gemini-3.1-pro-preview")

Output Schema

NameRequiredDescription
modelNo
pricing_per_1kNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and openWorldHint=false, so the safety profile is fully covered and the description's '30+ models' line usefully reinforces that this is a closed, static dataset. It does not disclose what happens on an unknown/ambiguous model name (error vs. nearest match) or whether the data can be stale, which are the remaining behavioral unknowns for a lookup tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the verb and resource, then the return fields, then coverage. The provider roll-call is slightly padded but does convey breadth of catalog in few words; nothing is wasted or buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present the description needn't explain return values, and it correctly spends its words on scope and catalog coverage instead. Missing only edge-case behavior for unrecognized model names, which is a minor gap for a static reference lookup.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single 'model' parameter already carries examples, so the schema does the work. The description adds no syntax rules (exact-match vs. alias handling, whether provider prefix like 'openai/gpt-4o' is accepted), so it does not rise above the baseline for a fully documented parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

A specific verb+resource ('Get detailed specs for an AI model') followed by an enumerated field list (context window, pricing, cutoff, provider, multimodal, reasoning, features) makes the payload unmistakable. It also scopes the catalog ('30+ models from OpenAI, Anthropic, Google...'), which implicitly separates it from a live/single-provider tool, but it never names the closest siblings (list_llm_models, compare_models, context_window_check), so differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the lookup to call when it needs static facts about a named model. There is no when-to-use/when-not statement, no mention of list_llm_models for discovery or compare_models for side-by-side comparison, and no guidance on what to do when a model isn't in the static catalog.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

multimodal_eval_guideA
Read-onlyIdempotent
Inspect

Multimodal evaluation: action=guide (default) interprets a CLIP/FID/VQA score against reference thresholds; clip_score, fid_score, vqa_accuracy compute the metric (HuggingFace, or your VLM key for VQA); pipeline runs all three.

NameRequiredDescriptionDefault
fidNo[pipeline] {real_images, generated_images}
vqaNo[pipeline] Same inputs as vqa_accuracy.
clipNo[pipeline] {image_url, text}
textNo[clip_score] Text to compare with the image.
modelNo[vqa_accuracy] Default gpt-4o.
scoreNo[guide] A score to interpret.
actionNo
metricNo[guide]
api_keyNo[vqa_accuracy] Key for the model's provider.
image_urlNo[clip_score, vqa_accuracy] Public image URL.
test_casesNo[vqa_accuracy]
real_imagesNo[fid_score] URLs.
image_base64No[clip_score, vqa_accuracy]
system_promptNo[vqa_accuracy]
image_mime_typeNo[clip_score, vqa_accuracy]
generated_imagesNo[fid_score] URLs.

Output Schema

NameRequiredDescription
errorsNo
metricsNo
resultsNo
web_toolNo
best_practicesNo
comparison_tableNo
score_interpretationNo

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description does add real behavioral context beyond the annotations: guide is the default action, metrics are computed via HuggingFace, and VQA requires a provider key supplied through api_key. However, it also describes outbound calls to HuggingFace and an external VLM provider ('your VLM key for VQA'), which conflicts with openWorldHint=false declaring no external entity interaction. The readOnlyHint=true claim is defensible (no local mutation) but the open-world claim is not.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences cover all five actions with zero filler, and the default action plus its meaning is front-loaded so a careless caller still lands on the safe path. The parenthetical dependency note is terse and information-dense rather than decorative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 16-parameter multiplexed tool with nested objects, the description supplies the action routing model that is the real cognitive load, and an output schema exists so return values need not be described. What is missing is per-action input requirements (e.g. that clip_score needs image_url/image_base64 plus text) and any note that pipeline is a heavier call than the individual metrics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 94%, so the baseline is 3, and the description earns above that by explaining the action-level semantics that the bracketed tags ('[guide]', '[pipeline]') cannot convey on their own: which action consumes score/metric, that pipeline bundles the three metric payloads, and that VQA needs an api_key. It does not state which parameters become required for each action, leaving that to be inferred from the tags.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (multimodal evaluation) and then enumerates what each of the five actions does: 'guide' interprets a CLIP/FID/VQA score, the three metric actions compute a metric, and 'pipeline' runs all three. That is enough for an agent to know which action covers which job without opening the schema. It stops short of differentiating itself from siblings such as run_vlm_test_suite or hallucination_check, which also touch multimodal quality assessment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: 'guide (default) interprets a ... score' tells the agent that an existing score should go through guide, while the metric actions compute one. There is no explicit when-not, no prerequisite statement for pipeline (e.g. that it needs clip/fid/vqa payloads together), and no routing advice versus the VLM test-suite siblings. An agent can infer the split, but it must infer it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

needle_haystack_generateA
Read-only
Inspect

Generate a "needle in a haystack" test: embeds a target fact into a large block of filler text at a specified position. Use this to test LLM context window retrieval accuracy. Returns the full haystack, the question to ask, and metadata. No API key needed.

NameRequiredDescriptionDefault
needleYesThe fact to hide (e.g. "The secret code is ALPHA-42")
tokensNoTarget haystack size in tokens (default: 5000, max: 100000)
positionNoWhere to insert the needle: "start", "middle", "end", "random" (default: "middle")middle
questionYesThe question to ask the LLM (e.g. "What is the secret code?")

Output Schema

NameRequiredDescription
needleNo
haystackNo
positionNo
questionNo
insert_blockNo
total_blocksNo
estimated_tokensNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, non-destructive, closed-world behavior, so the description does not need to re-litigate safety. It adds useful operational context such as returning the full haystack and requiring no API key, but it does not explain why the tool is marked non-idempotent or how randomness affects output when position is 'random'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and compact: it names the operation, summarizes the mechanism, states the use case, and notes the return shape without wasted words. The final note about no API key is brief and operationally relevant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With full schema coverage, an output schema, and rich annotations, the description provides enough context for an agent to select and invoke the tool correctly. The only meaningful gap is the lack of explanation around non-idempotent behavior or random-position variability.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameter meanings, token maximum, position enum, and defaults are already fully documented in the schema. The description mentions that the needle is inserted at a specified position, but adds no syntax, constraints, or format guidance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: generating a needle-in-a-haystack test by embedding a target fact into filler text. It clearly conveys the mechanism and purpose, but it does not explicitly differentiate the tool from nearby siblings such as lorem_ipsum, context_window_check, or prompt_test_suite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear intended use: testing LLM context window retrieval accuracy. That is enough to know when to reach for it, but there is no explicit guidance on when not to use it or which alternative to use for adjacent testing needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

normalize_vectorA
Read-onlyIdempotent
Inspect

L2-normalize a float vector (produce a unit vector with norm=1). Required by many vector DBs (Pinecone, Qdrant cosine). Supports batch normalization of up to 1000 vectors.

NameRequiredDescriptionDefault
batchNoBatch of vectors to normalize (overrides vector)
vectorNoSingle vector to normalize

Output Schema

NameRequiredDescription
modeNo
normNo
countNo
indexNo
vectorNo
resultsNo
dimensionNo
norm_afterNo
normalizedNo
norm_beforeNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description adds genuinely non-schema information: the 1000-vector batch cap, which an agent cannot infer from the input schema. Return format is left to the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, each earning its place: what it does, why it matters, and the batch capability. The core operation is front-loaded before supporting context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need no explanation, and annotations cover safety. The one gap is that no parameter is marked required, yet the tool needs at least one of vector or batch; the description never states that precondition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already explains both `vector` and `batch`, including that batch overrides vector. The description's mention of batch normalization adds no syntax or precedence detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb+resource (L2-normalize a float vector) and immediately defines the output (unit vector with norm=1). This is unambiguous against siblings like vector_quantize, vector_stats, and vector_similarity, which perform clearly different operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Required by many vector DBs (Pinecone, Qdrant cosine)" gives concrete context for why and when an agent would reach for this tool. It stops short of explicit exclusions or naming an alternative sibling, so it is clear context rather than full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

normalize_whitespaceA
Read-onlyIdempotent
Inspect

Normalize whitespace: trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces. Useful for cleaning code, configs, and text before processing.

NameRequiredDescriptionDefault
inputYesText to normalize
trim_fileNoTrim leading/trailing blank lines (default: true)
trim_linesNoTrim trailing whitespace from each line (default: true)
line_endingNo"lf" (default), "crlf", or "cr"
tab_to_spacesNoConvert tabs to N spaces (omit to keep tabs)
collapse_blanksNoCollapse runs of blank lines down to max_blank_lines (default: true)
max_blank_linesNoBlank lines to keep when collapsing, 0-10 (default: 2)

Output Schema

NameRequiredDescription
resultNo
line_endingNo
original_lengthNo
normalized_lengthNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds real value by enumerating the specific mutations applied, though it omits that these are individually toggleable and what the defaults are (those live in the schema).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste, with the concrete operations front-loaded and the usage note trailing. Nothing is repeated from structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full param descriptions, the definition is essentially complete. The only shortfall is that the interaction between options (e.g. collapse_blanks + max_blank_lines, trim_file vs trim_lines) is left entirely to the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description touches on tab conversion and line endings but adds no syntax detail beyond the schema – notably it says "convert tabs to spaces" while the schema specifies an N count and that omitting it keeps tabs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (normalize) and resource (whitespace) and enumerates the exact transformations: trailing spaces, blank lines, line endings, tabs. An agent can distinguish this from siblings like sort_lines, strip_markdown, or format_json without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Useful for cleaning code, configs, and text before processing" gives a usage context, but it names no alternatives and no exclusions. With siblings such as strip_markdown or sort_lines that also mangle text, the agent gets implied but not explicit selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

number_base_convertA
Read-onlyIdempotent
Inspect

Convert numbers between bases: decimal, binary, octal, hexadecimal, or any base 2–36. Auto-detects 0x, 0b, 0o prefixes.

NameRequiredDescriptionDefault
inputYesNumber to convert (e.g., "255", "0xFF", "0b1010", "0o77")
to_baseNoTarget base 2–36 (omit to get all common bases)
from_baseNoSource base 2–36 (auto-detects prefix if omitted)

Output Schema

NameRequiredDescription
octalNo
binaryNo
resultNo
decimalNo
to_baseNo
from_baseNo
hexadecimalNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, so safety is covered. The description adds real behavior beyond that: prefix auto-detection for 0x/0b/0o and the implicit default of returning all common bases when to_base is omitted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, zero filler, with the supported-scope clause front-loaded ahead of the auto-detect detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and the base-range plus prefix behavior round out the core contract. The remaining gap is the unprefixed-input assumption and out-of-range/invalid-input behavior, which an agent would need to guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both to_base/from_base descriptions already state the 2–36 range and prefix auto-detection, so the description largely restates the schema. It adds no clarification for the ambiguous unprefixed case (e.g. whether '11' is decimal or binary).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Convert') plus resource ('numbers between bases') and enumerates the supported base families and the 2–36 range. An agent can distinguish this from siblings like base64_encode or color_convert purely from the text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the narrow domain, but there is no explicit when-to-use or when-not framing. Notably, it does not rule out the most plausible sibling confusion — base64_encode/base64_decode, which also 'convert' representations of data — so the agent must infer the boundary itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

openapi_validateA
Read-onlyIdempotent
Inspect

Validate the structure of an OpenAPI 3.x specification (JSON or YAML). Checks required top-level fields (openapi, info.title, info.version, paths), validates each operation (responses, operationId uniqueness), detects undeclared $ref components, and flags missing 2xx responses. Returns a PASS/FAIL verdict, a 0–100 compliance score, and a list of errors and warnings with JSON-pointer locations. Use before publishing an API spec or generating SDK code.

NameRequiredDescriptionDefault
inputYesThe OpenAPI 3.x spec — a JSON string, a YAML string, or the already-parsed spec object.

Output Schema

NameRequiredDescription
scoreNo
statsNo
errorsNo
verdictNo
warningsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, non-destructive, and closed-world behavior, so the safety profile is covered. The description adds useful substance beyond that: the exact validation rules applied and the return shape (PASS/FAIL verdict, 0–100 score, errors/warnings with JSON-pointer locations). Minor gap: it doesn't say whether validation stops at the first failure or reports all issues.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core action and followed by the concrete checks, the return shape, and the use case. Every sentence contributes and nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a stateless single-input validator with a 100%-documented parameter and an output schema; the description covers what is validated, what is returned at a high level, and when to invoke it. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter's schema description already states it accepts a JSON string, YAML string, or parsed object. The description adds no parameter-level detail beyond that, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Validate the structure of an OpenAPI 3.x specification') and then enumerates exactly what is checked: required top-level fields, per-operation validation, undeclared $ref components, and missing 2xx responses. An agent can distinguish this from the sibling json_schema_validate purely on the OpenAPI-specific rule set.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear usage context: 'Use before publishing an API spec or generating SDK code.' That tells the agent when this tool is appropriate, but it names no exclusions or alternatives (e.g., json_schema_validate) for non-OpenAPI schema checking.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

optimize_prompt_tokensA
Read-onlyIdempotent
Inspect

Compress an LLM prompt by removing filler words, verbose phrases, duplicate sentences, and unnecessary whitespace. Returns optimized text with token savings breakdown. 100% deterministic, no API key needed.

NameRequiredDescriptionDefault
textYesThe prompt text to optimize
optionsNoToggle optimization steps (all true by default)

Output Schema

NameRequiredDescription
stepsNo
optimizedNo
tokens_afterNo
tokens_savedNo
percent_savedNo
tokens_beforeNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds genuinely non-obvious behavioral facts beyond that: the operation is fully deterministic, requires no API key or model call, and returns a token-savings breakdown alongside the optimized text.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: what it does, what it returns, and the operational constraint. Nothing is padded, and the capability statement is front-loaded before the return-value and determinism details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema is present, so the description need not explain the return shape, and it still summarizes it in one phrase. Combined with full schema coverage and rich annotations, the only real gap is the unexplained "instructions" toggle and the lack of sibling routing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, and the description's list of removal categories roughly maps onto the fillers/duplicates/whitespace toggles. However, the fourth toggle ("instructions") has no corresponding explanation anywhere, and the description doesn't clarify that optimization steps can be selectively disabled via the options object.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Compress an LLM prompt") and enumerates exactly what is removed: filler words, verbose phrases, duplicate sentences, whitespace. It does not name any sibling to contrast with, but the phrase "100% deterministic, no API key needed" implicitly distinguishes it from the LLM-backed tools in the same family.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the description makes clear this is for shortening a prompt before sending it to a model, but it gives no explicit when-to-use or when-not-to-use guidance. Siblings such as normalize_whitespace, strip_markdown, truncate_to_tokens, and count_tokens overlap in territory and the description never routes the agent between them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

parse_csvA
Read-onlyIdempotent
Inspect

Parse a CSV string into a JSON array of objects (or raw arrays). Full RFC 4180: quoted fields may contain the delimiter, embedded newlines (the Excel/Sheets multi-line cell), and doubled quotes. Custom delimiters supported. An unterminated quote is rejected with its position rather than parsed into corrupted rows. Use when processing spreadsheet exports, data imports, or structured text pipelines where the source is CSV. Supports up to 200 KB.

NameRequiredDescriptionDefault
inputYesCSV content to parse
headerNoTreat the first row as headers (default: true)
delimiterNoField delimiter character (default: ",")

Output Schema

NameRequiredDescription
rowsNo
columnsNo
headersNo
row_countNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive, closed-world semantics. The description adds genuinely new context: full RFC 4180 handling, quoted fields with embedded newlines/doubled quotes, unterminated-quote rejection with position info, and a 200 KB size cap. Only the exact error/return shape is left implicit, which the output schema covers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Five tight sentences, front-loaded with the core contract, then RFC details, then error behavior, then usage guidance. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a small, read-only 3-parameter tool with an output schema and full parameter coverage, the description supplies the key behavioral facts (RFC 4180 compliance, multi-line cells, error-on-unterminated-quote, size limit). Nothing critical is missing, though a note on how parse errors surface would close the last gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, including defaults for header and delimiter, so the schema carries the parameter documentation. The description only echoes custom delimiter support and never mentions the header flag, so no meaningful value is added beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (parse) and resource (CSV string) and immediately names the output form (JSON array of objects or raw arrays). This is clearly distinguishable from siblings like json_to_csv, which converts the other direction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage context: spreadsheet exports, data imports, structured text pipelines where the source is CSV. It does not explicitly rule out or point to alternatives (e.g., json_to_csv for the inverse direction), so it falls just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

parse_http_headersA
Read-onlyIdempotent
Inspect

Parse a raw HTTP headers block into a structured JSON object. Detects multi-value headers, masks Authorization values, and optionally audits for missing security headers (HSTS, CSP, X-Frame-Options, etc.).

NameRequiredDescriptionDefault
headersYesRaw HTTP headers (one "Name: Value" per line)
analyze_securityNoAudit for missing security headers (default: true)

Output Schema

NameRequiredDescription
parsedNo
securityNo
header_countNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is a read-only, idempotent, non-destructive, non-open-world operation. The description adds meaningful behavior beyond annotations: it detects multi-value headers and masks Authorization values, which is important for interpreting output. It does not cover permissions or error behavior, but for this tool the added behavioral context is solid.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loaded with the core action and followed by the most relevant behavioral details. There is no filler, and each clause adds useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete enough for a two-parameter parsing tool with a full schema and an output schema. The optional security audit is described with examples of checked headers. The only notable omission is routing guidance relative to similar security-header siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and both parameters are clearly documented in the schema. The description adds behavior around multi-value headers and security auditing but does not add syntax or constraints for the parameters beyond what the schema already provides. Baseline 3 is appropriate when the schema carries parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: parsing a raw HTTP headers block into structured JSON. It also names notable behaviors (multi-value detection, Authorization masking, optional security-header audit). It does not explicitly differentiate itself from closely related siblings such as security_headers_check or web_security_audit, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: it is for raw HTTP header blocks. The description does not state when to choose this over security_headers_check or web_security_audit, nor does it give exclusions or routing guidance. That is adequate but leaves the agent to infer sibling selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

post_jira_comment
Destructive
Inspect

Post the output of jira_to_test_suite as a formatted comment on the source Jira ticket. Converts Gherkin, E2E steps, API tests, and ambiguities into Atlassian Document Format (ADF). STATEFUL — creates a comment on the issue.

NameRequiredDescriptionDefault
issue_keyYesJira issue key, e.g. "PROJ-123"
jira_emailYesAtlassian account email
jira_tokenYesAtlassian API token
test_suiteYesThe test_suite object from jira_to_test_suite result
jira_base_urlYesAtlassian base URL

Output Schema

NameRequiredDescription
successNo
comment_idNo
comment_urlNo
pr_gatekeeper
Read-onlyIdempotent
Inspect

Compound quality gate for pull requests. Runs three sequential checks: (1) secret detection — scans diff for API keys, tokens, passwords matching 16 regex patterns; (2) bug analysis — heuristic scan for eval(), innerHTML, empty catch, console.log, TODO/FIXME; (3) commit message linting against Conventional Commits spec. Returns gate verdict (PASS/WARN/BLOCK), blockers, and actionable warnings.

NameRequiredDescriptionDefault
diffYesUnified git diff (output of `git diff HEAD`)
contextNoOptional: PR title or description for richer bug analysis
commit_messageYesThe commit message to lint (e.g. "feat(auth): add OAuth2 login")

Output Schema

NameRequiredDescription
flagsNo
scoreNo
checksNo
verdictNo
prompt_injection_scanA
Read-onlyIdempotent
Inspect

Scan user input or prompts for prompt injection patterns. Pattern-based: it recognises known phrasings and structures, not intent — an uncovered paraphrase or language reads clean, so clean is not proof of safety. Instruction overrides are covered in 11 languages (EN, FR, ES, DE, IT, PT, NL, RU, ZH, JA, AR), also after removing invisible characters, lookalike letters, spacing and leetspeak (marked obfuscated). Detects overrides, jailbreaks, role manipulation, encoding tricks, chat-template delimiters and fake role headers, template injection ({{…}}, ${…}) and requests to repeat the preceding context. A match inside quoted or fenced text is reported one level lower and marked quoted, never suppressed; a quote introduced by a live imperative keeps its severity.

NameRequiredDescriptionDefault
inputYesThe user input or prompt to scan for injection patterns
sensitivityNoDetection sensitivity (default: medium)

Output Schema

NameRequiredDescription
notesNo
detectionsNo
risk_levelNo
sensitivityNo
input_lengthNo
detections_countNo
quoted_detectionsNo
injection_detectedNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, closed-world, so safety is covered; the description goes well beyond that by disclosing the detection method, language coverage, normalization steps (invisible chars, lookalikes, spacing, leetspeak), and the severity-downgrade rule for quoted/fenced matches. These are non-obvious behaviors an agent could not infer from the schema or annotations, and nothing contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the key false-negative caveat are front-loaded, and the remaining sentences are dense with substantive detection detail rather than filler. The 11-language enumeration and the long quoted-text sentence are slightly heavy but each carries real information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not explain return values, and what remains — detection scope, method, language/normalization coverage, and the quoted-match severity rule — is exactly the context needed to interpret results correctly. Nothing material for correct invocation or interpretation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both params (input, sensitivity enum) are already documented, justifying the baseline 3. The description adds no insight into what low/medium/high sensitivity actually changes in detection, and the `obfuscated`/`quoted` markers it mentions are output flags rather than parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (scan) and resource (user input or prompts) plus the exact threat class (prompt injection patterns), which cleanly separates it from siblings like toxicity_scan, secret_scan, and detect_secrets. The second and third sentences further narrow the scope to pattern-based override/jailbreak detection, so an agent knows exactly what class of check this performs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating context: it is for user input/prompts, it is pattern-based, and 'clean is not proof of safety' tells the agent how much weight to give a negative result. It stops short of explicitly naming alternatives (e.g. toxicity_scan or shield_analyze) or stating when not to use it, so it lands at 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_template_fillA
Read-onlyIdempotent
Inspect

Fill a prompt template with variables. Supports {{variable}} syntax and {{#if key}}...{{/if}} conditional blocks. Returns the filled prompt and lists unfilled variables.

NameRequiredDescriptionDefault
strictNoThrow error if any variable is not provided (default: false)
templateYesPrompt template with {{variable}} placeholders
variablesNoKey-value pairs to fill (e.g. {"name":"Alice","role":"engineer"})

Output Schema

NameRequiredDescription
resultNo
total_varsNo
filled_variablesNo
unfilled_variablesNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint, and openWorldHint. The description adds that it supports {{variable}} and {{#if}} syntax and returns both the filled prompt and unfilled variables, which is useful behavioral context beyond annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose, second adds syntax and return details. Front-loaded, no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be detailed, yet the description mentions them briefly. For a simple 3-parameter tool with annotations and full schema coverage, the description provides enough context to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all parameters, so baseline is 3. The description adds conditional block syntax for the template parameter, which is not in the schema, and mentions the return of unfilled variables—extra meaning that justifies a bump.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Fill) and resource (prompt template) with scope (with variables). It doesn't explicitly differentiate from similar siblings like build_rag_prompt or system_prompt_builder, but the core action is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage for template filling but offers no explicit when-to-use or when-to-prefer-alternatives guidance. No mention of prerequisites or exclusions, leaving context inferred from the function name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_test_suiteA
Read-onlyIdempotent
Inspect

Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria. Returns a test plan with scored rubric — use this as input for manual or automated LLM evaluation.

NameRequiredDescriptionDefault
max_tokensNoMax token budget for the test
temperatureNoTemperature to use
user_promptYesThe user prompt to send
check_safetyNoInclude safety/PII checks in the rubric
must_includeNoRequired content (comma-separated)
system_promptYesThe system prompt under test
expected_formatNoExpected output format
must_not_includeNoForbidden content (comma-separated)
expected_behaviorNoDescription of what the LLM should do (free text)
adversarial_promptsNoAuto-generate adversarial test variants (jailbreak, injection, edge cases)

Output Schema

NameRequiredDescription
rubricNo
categoriesNo
total_testsNo
instructionsNo
test_suite_nameNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is covered. The description adds meaningful context beyond that: the tool returns a scored rubric test plan and is a precursor to evaluation rather than an evaluator itself. It still doesn't say whether adversarial_prompts actually generates variants at call time or just annotates the plan, which is the one behavioral unknown left.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the first states the action and required inputs, the second states the return value and downstream use. Nothing is wasted and the essential information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 10 parameters, an output schema, and full annotation coverage, the description only needs to frame the tool's role and data flow, which it does. The remaining gap is routing: an agent can't tell from the description alone why it should pick this over run_semantic_tests or generate_test_cases, which matters given the crowded sibling set.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with per-property descriptions and an example object, so the schema carries the parameter burden. The description restates the three primary concepts (system prompt, user prompt, expected output criteria) but adds no syntax or interaction detail, such as how must_include/must_not_include feed the rubric or how adversarial_prompts changes the returned plan. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource: 'Define a test suite for a prompt,' plus the required inputs (system prompt, user prompt, expected criteria) and the artifact returned (a test plan with scored rubric). It is clearly a planning/authoring tool rather than an execution tool, which separates it from run_semantic_tests or guardrail_test, though it never names those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'use this as input for manual or automated LLM evaluation' implies a workflow position (build the plan first, then evaluate), which is useful guidance. But there is no explicit when-not clause and no named alternative for agents that want to actually run the suite rather than define it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rag_relevance_rankA
Read-onlyIdempotent
Inspect

Rank an array of text chunks by relevance to a query using TF-IDF scoring. Simulates retrieval ranking for RAG testing without needing embeddings or an API.

NameRequiredDescriptionDefault
queryYesThe user query
top_kNoReturn top K results (default: all)
chunksYesArray of text chunks to rank

Output Schema

NameRequiredDescription
rankNo
indexNo
queryNo
scoreNo
resultsNo
returnedNo
total_chunksNo
chunk_previewNo
keyword_overlapNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so safety and side-effect behavior are covered. The description adds meaningful behavioral context by naming the scoring algorithm (TF-IDF) and clarifying that it works offline without embeddings or API calls, though it omits details like sort order or score range.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no filler. The core purpose and the key differentiator (no embeddings/API) are front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given full schema coverage, rich annotations, and an output schema, the description does not need to explain return values. It adequately explains the method and intended context, though it could briefly mention output ordering or tie-breaking behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents 'query', 'chunks', and 'top_k' (including its default). The description adds no parameter-specific meaning beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Rank'), resource ('array of text chunks'), and the scoring method ('TF-IDF'). The phrase 'without needing embeddings or an API' implicitly distinguishes it from sibling tools like embedding_similarity and vector_similarity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a usage context ('Simulates retrieval ranking for RAG testing'), implying it is for testing rather than production retrieval. However, it never explicitly says when to use this versus alternatives such as bm25_score, embedding_similarity, or rerank_evaluate, nor does it state any exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rate_tool
Destructive
Inspect

Give honest usage feedback on an IA-QA MCP tool. Give a score (1-5) and a comment. Rate low (1-2) if the tool was wrong, irrelevant, or a poor fit; rate high (4-5) only if it genuinely solved your need. Maintainers read ratings, one vote per agent and tool a month: comments and per-tool scores are not published, only the overall average and vote count. Comments are screened: one carrying a credential is not stored, personal data becomes placeholders — describe what happened, not what a model should do. Skip routine successes: signal, not praise. Example: rate_tool({ tool_name: "format_json", score: 2, comment: "Tried to pretty-print a JSON5 file, it rejected trailing commas — not usable for my case." })

NameRequiredDescriptionDefault
scoreYesRating from 1 (poor) to 5 (excellent)
commentNoStrongly encouraged — explain what you were trying to do and whether the tool got you there. Be specific about what was missing, wrong, or a poor fit. This is the most valuable part of the rating. Up to 2000 chars are stored; go over and the response says so (truncated: true) — send the remainder as a second call rather than assuming it landed.
tool_nameYesName of the MCP tool to rate (e.g. "format_json", "shield_analyze")

Output Schema

NameRequiredDescription
okNo
scoreNo
commentNo
messageNo
rated_atNo
tool_nameNo
truncatedNo
stored_charsNo
redact_piiA
Read-onlyIdempotent
Inspect

Automatically detect and redact Personally Identifiable Information (PII) from text. Replaces emails, phone numbers, SSNs, IBANs, credit cards, IP addresses, and JWT tokens with [REDACTED_TYPE] placeholders. Safe to use before logging or sending to an LLM.

NameRequiredDescriptionDefault
inputYesText to redact PII from
typesNoComma-separated types to redact (default: all). Options: email, phone, ssn, iban, credit_card, ip_address, jwt
markerNoCustom replacement marker (default: "REDACTED"). Result: [REDACTED_EMAIL]

Output Schema

NameRequiredDescription
cleanNo
pii_foundNo
replacementsNo
redacted_textNo
total_redactionsNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description contributes meaningfully beyond the annotations: it discloses which data classes are replaced and the exact placeholder format ([REDACTED_TYPE]), which tells the agent the output is transformed in a predictable way. The readOnly/idempotent/non-destructive safety profile is already covered by annotations, and an output schema exists, so return shape is not the description's burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with zero filler: the core action comes first, then the enumerated coverage, then the safety/usage note. Every sentence carries information an agent can act on.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given rich annotations, an output schema, and 100% parameter coverage, the description fills the remaining gaps: what is redacted, how it is marked, and why/when to call it. Nothing needed for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real value by listing the redactable types (email, phone, ssn, iban, credit_card, ip_address, jwt) in the same vocabulary as the 'types' parameter, and by showing the marker output pattern ('[REDACTED_TYPE]') that clarifies the 'marker' parameter's effect.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pair and resource ('detect and redact PII from text') and enumerates the exact PII classes handled, so an agent knows precisely what it does. It stops short of explicitly distinguishing itself from sibling tools like detect_secrets or secret_scan, which also touch sensitive strings, so sibling differentiation is implied (redaction vs detection) rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Safe to use before logging or sending to an LLM' gives a concrete usage context that tells the agent when this tool is appropriate. It does not name an alternative tool or state when-not-to-use conditions, so it is clear context without explicit routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

regex_testA
Read-onlyIdempotent
Inspect

Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups. Use for validating user inputs, extracting structured data from text, or debugging regex patterns. Supports flags g, i, m, s, u, y. The match runs in an isolated thread with a 500 ms budget: a pattern that blows up (catastrophic backtracking, e.g. "(a+)+$") comes back as redos_detected:true — a real ReDoS verdict on your pattern — instead of hanging.

NameRequiredDescriptionDefault
flagsNoRegex flags: g (global), i (case-insensitive), m (multiline), s (dotAll) — default: ""
inputYesThe string to test against (max 50 KB)
patternYesRegular expression pattern (without delimiters)

Output Schema

NameRequiredDescription
noteNo
flagsNo
matchedNo
matchesNo
patternNo
verdictNo
elapsed_msNo
match_countNo
redos_detectedNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent annotations by disclosing the 500 ms isolated-thread budget and the exact failure mode: catastrophic backtracking returns redos_detected:true instead of hanging. That is precisely the kind of operational trait an agent cannot infer from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: capability first, use cases second, then the critical runtime caveat. No filler, and the ReDoS warning is placed where it will be read before invocation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be spelled out, yet the description still flags the redos_detected field that matters most. Combined with full parameter coverage and clear annotations, nothing an agent needs is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning by enumerating supported flags including u and y, which the schema's flags description omits (it lists only g, i, m, s). That genuinely extends the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Test a regular expression pattern against an input string') and immediately specifies the return shape (matches, index positions, named capture groups). This clearly separates it from siblings like extract_json_path or text_stats.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names three concrete use cases (validating user inputs, extracting structured data, debugging regex), giving a clear sense of when to reach for it. It does not, however, name any alternative tool or exclusion condition, so a 4 rather than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rerank_evaluateA
Read-only
Inspect

Evaluate RAG retrieval: rank passages against a query, compute Precision@k / Recall@k and a PASS/FAIL verdict from your relevance labels. Modes: (1) give every passage the score YOUR reranker produced — deterministic, offline, the mode to gate CI on; (2) no scores, no key — a local BM25 keyword baseline, never a judgement of a neural reranker; (3) api_key for an NVIDIA account that still has reranking (NVIDIA retired the hosted endpoints on 2026-05-18).

NameRequiredDescriptionDefault
queryYes
top_kNoDefault 3.
api_keyNoNVIDIA key, used only when no passage has a score; never stored.
passagesYes2 to 20 passages.
thresholdNoMinimum Precision@k to PASS (default 0.5).

Output Schema

NameRequiredDescription
kNo
modeNo
modelNo
rankedNo
verdictNo
thresholdNo
latency_msNo
recall_at_kNo
precision_at_kNo
total_passagesNo
has_ground_truthNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the operation as read-only and non-destructive, and the description adds useful traits: mode 1 is deterministic and offline and suitable for CI gating, while mode 2 is a keyword baseline that cannot judge a neural reranker. The retirement date for hosted NVIDIA endpoints is meaningful context beyond the annotations. It does not describe pagination or output shape, but an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action and metrics, then enumerates modes compactly; every clause carries information. The mode list is dense and slightly crowded into one paragraph, but no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Since an output schema exists, the description need not explain return values; what remains is exactly what an agent needs: what the tool does, which mode it is in based on the inputs, and which mode is safe for CI. The combination of modes, caveats, and example-annotated schema makes the definition self-sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 80% schema coverage, the baseline is 3, and the description adds meaningful meaning: the `score` value must be the caller's reranker output and applies to all passages or none, while `api_key` is needed only when no passage has a score. That's real mode-selection semantics beyond the schema's field text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

It names a specific verb and resource: rank passages against a query and compute Precision@k / Recall@k with a PASS/FAIL verdict. It also distinguishes its BM25 fallback mode from a neural reranker, which implicitly separates it from sibling tools like rag_relevance_rank and bm25_score. It does not name those siblings explicitly, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The three modes are enumerated with conditions: supply your own scores for deterministic offline CI gating, omit scores for a keyword baseline, or supply api_key. The description explicitly warns that mode 2 is not a judgment of a neural reranker and that NVIDIA retired hosted endpoints. It never explicitly says when to choose rag_relevance_rank instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

response_quality_score
Read-onlyIdempotent
Inspect

Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length. Returns a 0-100 score over the criteria actually measured — or total_score null with the reason when none is given, because relevance and correctness cannot be read off word overlap (a correct one-word answer shares no words with its question). Meaning against a reference answer: run_semantic_tests. Also returns unscored signals: question-term overlap, average sentence length, markdown.

NameRequiredDescriptionDefault
questionYesThe original question/prompt
responseYesThe LLM response to score
max_lengthNoMaximum character length. Scored as a proportional penalty beyond it.
expected_keywordsNoKeywords a correct answer must contain, as whole words (case-insensitive; a plural in -s/-es counts). Scored as coverage.

Output Schema

NameRequiredDescription
gradeNo
statsNo
reasonNo
signalsNo
breakdownNo
max_scoreNo
total_scoreNo
not_measuredNo
run_eval_contractA
Read-only
Inspect

Run an IA-EVAL test suite (.ia-eval.yaml contract): call the model per scenario; return passed (true, false, null if undecided) and a verdict (PASS, FAIL, PARTIAL). Expectations are assertions that fail a scenario whatever its score; evaluators give the score. Hosted: pass inline_contract. validate_only calls no model.

NameRequiredDescriptionDefault
api_keysNoYour keys per provider. Only Groq falls back to the server key; any other provider needs yours.
overridesNoOverride contract defaults
contract_pathNoA .ia-eval.yaml under the server's working directory (local use).
validate_onlyNoCheck the contract against the IA-EVAL schema and stop: no model call, no key, deterministic. Returns valid + issues instead of throwing. Gate a PR that edits a contract on valid.
inline_contractNoThe contract as an object: "metadata" ({name, version, model?, provider?}), "expectations", "evaluators" and "scenarios" ([{id, input, expectations?}]). generate_eval_yaml scaffolds one.

Output Schema

NameRequiredDescription
validNo
issuesNo
summaryNo
metadataNo
warningsNo
contract_pathNo
scenario_resultsNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations it discloses that a model is invoked per scenario, that null means undecided, the PASS/FAIL/PARTIAL verdict space, and that validate_only is deterministic with no model call or key. The distinction that expectations are hard assertions while evaluators produce the score is genuinely useful behavioral context; it does not, however, discuss cost, latency, or partial-run failure behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and dense throughout; every clause carries information. Some compression hurts readability ('Hosted:' is undefined shorthand, semicolon chaining obscures the passed/verdict relationship).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter, nested-object tool with a full output schema and annotations, the description covers the execution model, the two input paths, and the validation mode adequately. Minor gaps remain around what PARTIAL means and what happens when a scenario errors mid-run.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning by positioning inline_contract as the hosted path versus contract_path and by restating validate_only's no-key/no-model semantics. The api_keys fallback nuance and overrides fields are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Run an IA-EVAL test suite (.ia-eval.yaml contract)') and describes what happens per scenario (call the model, produce passed + verdict). It clarifies the two entry modes but does not name or distinguish itself from close siblings such as run_semantic_tests, prompt_test_suite, or run_vlm_test_suite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete routing rule between the two contract sources ('Hosted: pass inline_contract') and a clear when-to-use for validate_only ('gate a PR that edits a contract on valid'). It lacks explicit exclusions telling an agent when to prefer the sibling eval tools instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_pr_gate_pipelineA
Read-onlyIdempotent
Inspect

Review triage for a pull request. Takes a unified git diff (git diff HEAD) and returns: diff-lint findings with the lines that produced them, regression impact areas, a risk score 0–100 with the factors that built it (churn, files touched, sensitive paths, whether any test file changed, lint severities, impacted risk areas), generated test cases, and a PASS / CONDITIONAL / BLOCK recommendation. Advisory: the score measures properties of the diff, not the correctness of the change — it does not read the code semantically and does not replace a reviewer or a static analyser. See notAnalysed in the response.

NameRequiredDescriptionDefault
contextNoOptional PR title or description for richer analysis
git_diffYesUnified git diff (output of `git diff HEAD` or copied from GitHub diff view)

Output Schema

NameRequiredDescription
slaNo
highNo
topBugsNo
criticalNo
bugsFoundNo
riskLevelNo
riskScoreNo
disclaimerNo
impactAreasNo
inputFormatNo
notAnalysedNo
riskFactorsNo
changedFilesNo
severityLevelNo
testCasesGeneratedNo
mergeRecommendationNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly/idempotent/non-destructive/closed-world, and the description still adds substantive context: exactly which factors build the risk score (churn, files touched, sensitive paths, test-file changes, lint severities, impacted risk areas) and an explicit limitation that it does not read code semantically. It also redirects the reader to the notAnalysed field for unanalysed areas — genuinely beyond what the annotations say.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then the return set, then the caveat. The return enumeration is long but each item earns its place for tool selection; the one weak spot is that the output set is repeated almost verbatim against the output schema while the more decision-relevant caveat sits at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only analysis pipeline with a full output schema, the description covers input contract, output shape, decision semantics and known limitations. Missing only practical edges such as behaviour on malformed/large diffs or minimum diff size for meaningful output.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are documented there, so the schema does the heavy lifting. The description reinforces the required input's format (`git diff HEAD`) but adds nothing about the optional `context` parameter or about diff size/format failure modes, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource ('Review triage for a pull request') followed by an exhaustive enumeration of the artifacts returned: diff-lint findings, regression areas, risk score, generated tests, and a PASS/CONDITIONAL/BLOCK recommendation. It does not, however, distinguish itself from the sibling pr_gatekeeper or analyze_diff_bugs, which is the only real gap.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the input contract (`git diff HEAD`) and a clear advisory boundary — the score measures diff properties, not correctness, and does not replace a reviewer or static analyser. That is a useful when-not signal, but no alternative tool is named and no positive triggering condition (e.g. 'use before opening a PR') is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_semantic_testsA
Read-onlyIdempotent
Inspect

Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L. Two modes: tfidf (default, free, no API key) or embeddings (OpenAI text-embedding-3-small, BYOK, true semantic similarity). Returns per-case PASS/FAIL verdicts and an overall verdict. CI-ready: pipe the JSON verdict field to gate a build.

NameRequiredDescriptionDefault
modeNotfidf (default): fast, free, lexical. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key.
casesYesArray of (actual, expected) pairs to evaluate.
api_keyNoOpenAI API key — required only when mode is embeddings.
thresholdsNoPass/fail thresholds (defaults: cosine 0.75, rouge_l 0.5).
require_allNoIf true (default), all cases must pass for overall PASS. If false, at least one case passing returns PASS.

Output Schema

NameRequiredDescription
modeNo
totalNo
failedNo
passedNo
resultsNo
verdictNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive). The description adds meaningful context beyond them: the BYOK/API-key requirement for embeddings mode, that tfidf needs no key, and the per-case plus overall verdict behavior. Good added value though not exhaustive on edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with the core purpose, then modes, then output and CI use. Every sentence carries information with no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema, full annotation coverage, and 100% schema description coverage, the description provides everything else an agent needs — mode tradeoffs, key requirement, and verdict output — to select and invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters including mode enum and thresholds. The description reinforces mode semantics and the API-key dependency, but adds little beyond the schema for thresholds or require_all, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — comparing actual vs expected text pairs via cosine similarity + ROUGE-L — plus the PASS/FAIL verdict output. Purpose is unambiguous, but it doesn't explicitly distinguish itself from sibling similarity tools like similarity_score, embedding_similarity, or compare_responses, so no sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains the two modes and when to pick each (tfidf default free/no key vs embeddings BYOK/true semantic), and notes the CI-ready gating use case. It stops short of naming alternative tools or stating when-not to use this one.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_vlm_test_suiteA
Read-only
Inspect

Test a vision model (OpenAI, Anthropic or Gemini) on one image: each test case asks a question and asserts on the answer (contains, not_contains, json_format, min_length, max_length, semantic_contains = TF-IDF cosine ≥ 0.4). Returns per-case PASS/FAIL, the pass rate, an overall PASS/WARNING/FAIL against threshold, and latency. BYOK: your key for the model's provider; the image is fetched only from a public address.

NameRequiredDescriptionDefault
modelYes
api_keyYesKey for the model's provider.
image_urlNoPublic image URL (or image_base64).
thresholdNoPass rate for an overall PASS (default 80).
test_casesYes
image_base64No
system_promptNo
image_mime_typeNoFor image_base64 (default image/jpeg).

Output Schema

NameRequiredDescription
modelNo
totalNo
failedNo
passedNo
resultsNo
verdictNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (readOnly, non-destructive, openWorld, non-idempotent), and the description adds real value on top: BYOK auth requirement, the public-address-only fetch restriction, and the shape of the result (per-case PASS/FAIL, pass rate, overall verdict, latency). It does not explain cost/rate implications of calling an external provider, which is the main residual gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences, front-loaded with the core action and the key behavioral facts; nothing is filler. The second sentence is a long run-on chain of clauses, which slightly hurts scannability but costs no information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so spelling out return values was optional, and the description covers auth, input constraints, and assertion semantics for a 3-required-of-8 parameter tool. Minor gaps remain around system_prompt's effect and base64 input handling, but an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With only 50% schema coverage, the description compensates well by defining assertion semantics that the schema only enumerates — notably that semantic_contains means TF-IDF cosine ≥ 0.4 — and clarifying that the threshold governs the overall PASS. It adds little for system_prompt, image_base64, or image_mime_type, keeping it short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: test a vision model (with named providers) on one image with question/assertion test cases. The 'one image' scope implicitly separates it from run_vlm_test_suite_batch, and the enumerated assertion types make the tool's job concrete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating context: BYOK, key must match the model's provider, and the image must be reachable at a public address. It never names an explicit alternative (e.g. run_vlm_test_suite_batch for multiple images, or prompt_test_suite for text-only), so routing is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_vlm_test_suite_batchA
Read-only
Inspect

Compare several vision models on the same image and test cases, in parallel (same assertions as run_vlm_test_suite). Returns per-model PASS/FAIL, pass rates, latency and a comparison table. BYOK: one key per model.

NameRequiredDescriptionDefault
modelsYes
api_keysYesModel id → API key.
image_urlNoPublic image URL (or image_base64).
thresholdNoPass rate for an overall PASS (default 80).
test_casesYes
image_base64No
system_promptNo
image_mime_typeNoFor image_base64 (default image/jpeg).

Output Schema

NameRequiredDescription
suitesNo
verdictNo
total_failedNo
total_passedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, non-destructive, openWorld, non-idempotent), so the bar is lower; the description still adds the parallel-execution trait and the BYOK auth requirement (one key per model), which is the main operational gotcha. It does not mention cost/latency implications of fanning out across up to six external providers, nor what happens when a model errors mid-batch.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and the parallelism trait, then the return shape, then the auth prerequisite. No filler or repetition of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't enumerate return fields, and it still names the headline results (PASS/FAIL, pass rates, latency, comparison table). Given a nested 8-parameter batch tool with BYOK auth, the main gap is failure/partial-success semantics and any batch-size or cost constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 50% across 8 parameters, and the description clarifies only the key mapping ('one key per model') and the shared image/test-case inputs, both of which the schema already conveys. It adds no meaning for threshold, system_prompt, image_mime_type, or the assertion_type/assertion_value pairing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (compare) and resource (vision models on the same image and test cases), and explicitly scopes it against the sibling run_vlm_test_suite by saying it uses the same assertions but runs models in parallel. An agent can distinguish this from compare_models/compare_responses without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The parenthetical '(same assertions as run_vlm_test_suite)' implicitly routes single-model runs to that sibling, and 'BYOK: one key per model' states the prerequisite for calling this. There is no explicit 'do not use when' clause or cost/rate-limit warning, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

sandbox_scenarioA
Read-onlyIdempotent
Inspect

Get a ready-made selector-drift test case with a known-correct answer, for testing this MCP server or an agent workflow end to end. Each scenario is a real DOM capture of a deliberately breakable app, taken before and after one specific UI change (a renamed label, two swapped buttons, a duplicated locator, an element moved behind a menu…), plus the verdict those two contracts MUST produce. Call with no arguments to list the scenarios; call with a scenario id to get "baseline" and "current" mappings. THE LOOP: pass baseline and current to diff_mappings, then compare its "verdict" and "counts" to this tool's "expected" — they must match exactly. A mismatch means this server's diff engine has drifted, not that your inputs are wrong. Deterministic and offline: the captures are committed fixtures, identical on every call. Try it live at https://www.ia-qa.com/devtools/sandbox

NameRequiredDescriptionDefault
scenarioNoScenario id. Omit to list every available scenario with its expected verdict. Ids: no-change, swap-label, add-testid, duplicate-role-name, remove-element, insert-sibling, rename-label, counter-label, move-behind-menu, add-element
include_htmlNoInclude the generated HTML of the mutated page (default false). Only useful if you want to render or re-capture it yourself; the loop does not need it.

Output Schema

NameRequiredDescription
htmlNo
pageNo
blurbNo
countNo
titleNo
currentNo
teachesNo
baselineNo
expectedNo
scenarioNo
mutationsNo
scenariosNo
how_to_run_the_loopNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by disclosing determinism and offline provenance ('committed fixtures, identical on every call'), which reinforces idempotence with concrete reason. It also adds diagnostic semantics not present in the annotations: a verdict mismatch indicates server drift, not bad inputs — context an agent could not infer elsewhere.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded, then the loop, then determinism — a sensible order with labeled segments ('THE LOOP:'). It runs slightly long and the trailing external URL is not needed for correct invocation, though the remainder earns its space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers everything needed for a two-parameter, read-only fixture tool: what it returns (baseline/current mappings), how to use it in a loop, expected semantic ('expected' verdict), and determinism guarantees. With an output schema present, no return-value documentation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented in the schema, including the 'omit to list' behavior and the include_html caveat. The description restates the no-arg vs. scenario-id behavior but adds no format or syntax detail beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get a ready-made selector-drift test case with a known-correct answer') and scopes the use case ('for testing this MCP server or an agent workflow end to end'). It also implicitly distinguishes itself from diff_mappings by naming that tool as the downstream consumer of its output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly covers both call modes ('call with no arguments to list the scenarios; call with a scenario id to get baseline and current mappings') and prescribes the exact follow-up sequence with diff_mappings. It even tells the agent how to interpret a mismatch, removing any ambiguity about what to do with the result.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

score_geo_signalsA
Read-onlyIdempotent
Inspect

Analyze a webpage HTML (or full HTML) for GEO (Generative Engine Optimization) signals. Returns a score /60 with per-check results and improvement tips. GEO = optimizing pages for AI-powered search engines (ChatGPT Search, Perplexity, etc.).

NameRequiredDescriptionDefault
head_htmlYesRaw HTML of the <head> section (or full page HTML) to analyze

Output Schema

NameRequiredDescription
gradeNo
scoreNo
checksNo
passedNo
max_scoreNo
total_checksNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so safety is covered. The description adds a useful output summary ('score /60 with per-check results and improvement tips'), but an output schema exists, so that value is partly redundant. It says nothing about which checks are performed or how the /60 is composed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with the action and output front-loaded and the GEO parenthetical placed where it clarifies rather than clutters. No filler sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations, an output schema, and a fully documented single parameter, the description covers what an agent needs to invoke it. The main remaining gap is that it never enumerates what GEO checks are scored, which would help an agent decide if this tool answers its question.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter, and schema coverage is 100% with the schema itself stating 'or full page HTML'. The description's '(or full HTML)' adds almost nothing beyond the schema. Baseline 3 applies since the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (analyze/score) and resource (webpage <head> or full HTML) plus the exact domain (GEO signals). The purpose is unambiguous and doesn't overlap conceptually with nearby siblings like calculate_readability or security_headers_check, though it never names an alternative explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: you infer you should call it when you have HTML and want a GEO assessment. The definition of GEO ('optimizing pages for AI-powered search engines') is context, not a when-to-use rule. No exclusions or named alternatives are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_jira_issuesA
Read-only
Inspect

Search Jira using JQL (Jira Query Language). Returns matching issues with key fields. Ideal for finding open bugs, sprint tickets, or issues by label/assignee/component. BYOK — credentials transit in-memory only, never stored.

NameRequiredDescriptionDefault
jqlYesJQL query string, e.g. "project = PROJ AND status = Open AND assignee = currentUser() ORDER BY priority DESC"
fieldsNoFields per issue. Default: summary, status, assignee, priority, issuetype, labels, created, updated
jira_emailYesAtlassian account email
jira_tokenYesAtlassian API token
max_resultsNoMax issues to return (default: 10, max: 50)
jira_base_urlYesAtlassian base URL, e.g. "https://mycompany.atlassian.net"

Output Schema

NameRequiredDescription
jqlNo
totalNo
issuesNo
returnedNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, non-destructive, open-world semantics, so the description's real contribution is the BYOK disclosure — credentials transit in-memory only and are never stored — which an agent cannot infer from any structured field. It stops short of mentioning pagination/result caps or JQL error behavior, so 4 not 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences: what it does, what it returns, when to use it, then the credential caveat. Nothing is repeated and the core operation is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and annotations carry the safety profile. The description covers the one thing structured data omits — that four credentials are required and handled in-memory — leaving only minor gaps like result limits and rate limits.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and every parameter (jql, fields, max_results, and the three auth fields) is documented inline, so the baseline of 3 applies. The description adds no parameter-level meaning beyond naming JQL and 'key fields'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names the specific verb+resource ('Search Jira using JQL') and states the return ('matching issues with key fields'). It doesn't differentiate itself from the sibling fetch_jira_issue (single-issue retrieval vs. query-based search), which keeps it at 4 rather than 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Ideal for finding open bugs, sprint tickets, or issues by label/assignee/component' clause implies when to reach for this tool, but there is no explicit when-not guidance and no named alternative (fetch_jira_issue, fetch_confluence_page) to route the agent away.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

secret_scanA
Read-onlyIdempotent
Inspect

Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), Slack / Teams / Discord webhook URLs, private keys, JWTs, database connection strings, Bearer and Basic auth headers, and hard-coded values assigned to password/secret/token names (a value read at run time — Cypress.env(), os.getenv() — is not flagged). Returns findings with type, severity, line and a redacted preview. Regex-based, no network.

NameRequiredDescriptionDefault
inputYesText to scan — or an array of texts / {id, text} for a batch (one result each, worst gate on top).
typesNoComma-separated families (default all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, teams, discord, cookie, twilio, sendgrid, jwt, private_key, connection_string, bearer, basic_auth, generic, or pattern names. An unknown value is rejected, never a silent "clean".

Output Schema

NameRequiredDescription
summaryNo
findingsNo
risk_levelNo
input_linesNo
scanned_typesNo
secrets_foundNo
findings_countNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so the safety profile is covered. The description adds real value beyond that: 'Regex-based, no network' discloses the detection mechanism and its false-positive implications, and it explains that findings are returned with type, severity, line and a redacted preview, plus that runtime-read values are not flagged.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, followed by the detection inventory and two short closing sentences on output and mechanism. The long vendor enumeration is dense but each item earns its place by telling the agent what will be caught.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't document return values, and it still summarizes the finding shape. It covers input, mechanism, and an important exclusion, but omits how this differs from the sibling 'detect_secrets', which is the one thing an agent still needs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both parameters are documented in the schema, including the batch form and the comma-separated 'types' families, so the baseline of 3 applies. The description's enumeration of secret families loosely reinforces the 'types' parameter but adds no syntax or default behavior beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Scan text or code for leaked secrets') and enumerates the exact secret families detected, so an agent knows precisely what the tool produces. It does not, however, differentiate itself from the near-identical sibling 'detect_secrets', leaving the agent to guess which of the two to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the tool is for scanning text/code for secrets, and the parenthetical about run-time reads (Cypress.env(), os.getenv()) clarifies a boundary condition. There is no explicit when-to-use/when-not guidance and no mention of the competing 'detect_secrets' or 'redact_pii' siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

security_headers_checkA
Read-only
Inspect

Analyse the HTTP security headers of a public URL OR of raw response headers you paste in. Grades each header (A–F) for: Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, X-XSS-Protection, Cross-Origin-Opener-Policy, Cross-Origin-Resource-Policy, and Cross-Origin-Embedder-Policy. Returns an overall score (0–100), per-header grades, missing headers, and fix snippets for Express, Nginx, and Apache. For localhost/private targets the remote server cannot reach, pass the headers parameter instead of url.

NameRequiredDescriptionDefault
urlNoOptional. Full public URL to check (e.g. https://example.com). Omit it entirely when using `headers`. The server cannot reach localhost/private IPs.
headersNoOptional, and sufficient on its own (no url needed). The response headers to grade, either as an object {"strict-transport-security": "max-age=...", ...} or as the raw header block pasted as a string (e.g. `curl -sI` output). Use this to audit a local server the remote MCP cannot reach.

Output Schema

NameRequiredDescription
fixNo
keyNo
urlNo
weakNo
gradeNo
scoreNo
valueNo
headerNo
sourceNo
weightNo
detailsNo
missingNo
weak_countNo
missing_countNo
overall_gradeNo
headers_checkedNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare a read-only, non-destructive, open-world operation, so the safety profile is covered. The description adds genuinely behavioral context beyond that: the A–F grading scheme, the fixed set of ten headers, the 0–100 score, and the reachability limitation for private targets; it loses a point only because the return-value sentence partly duplicates the existing output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose and the url/headers split are front-loaded in the first sentence, and the localhost fallback closes usefully. The long enumeration of ten header names and the return-format sentence are somewhat redundant with the schema and output schema, but they are not padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema, full annotation coverage, and 100% schema description coverage, the description only needs to cover purpose, routing, and edge cases, which it largely does. The remaining gap is that both parameters are optional and the description never says what happens if neither `url` nor `headers` is supplied.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both `url` and `headers` are already fully documented, including accepted formats and the localhost caveat. The description restates the same guidance rather than adding new meaning, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: analysing HTTP security headers of a URL or pasted header block, and enumerates exactly which headers are graded. It is clear what the tool does, but it never differentiates itself from overlapping siblings such as `web_security_audit`, `cors_checker`, or `parse_http_headers`, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit routing rule: use `url` for public targets, but pass `headers` instead for localhost/private targets the remote server cannot reach. That is a concrete when-to-use condition, though it offers no guidance on when to prefer this tool over the broader `web_security_audit` sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

shield_analyzeA
Read-only
Inspect

One-call safety gate for an LLM response: grounding (with source), prompt injection on the response AND the prompt (checks.injection / checks.injection_prompt), toxicity, PII, guardrail rules, quality, plus an LLM review on Groq (server key; model "none" skips it and sends nothing out). Returns PASS/FIX/BLOCK, a 0-100 safety score, per-check results and fixes. PII gates the verdict: SSN or card → BLOCK, email → FIX, phone/IP flagged only.

NameRequiredDescriptionDefault
modelNoLLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default openai/gpt-oss-20b), or "none".
rulesNoSame format as guardrail_test.
promptNoThe original prompt — also scanned for injection; pass it whenever you have it.
sourceNoReference text for the grounding check.
responseYesThe LLM response to check.

Output Schema

NameRequiredDescription
flagsNo
gradeNo
scoreNo
checksNo
verdictNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing the verdict taxonomy (PASS/FIX/BLOCK), the 0-100 score, the per-check result payload, and the exact PII severity thresholds (SSN/card → BLOCK, email → FIX, phone/IP flagged only). It also surfaces the important data-egress fact that model 'none' sends nothing externally and that the Groq review uses a server key, which is precisely the context openWorldHint alone cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose and verdicts, then check inventory, then the PII gating rule — every clause carries information. It is a single dense paragraph with nested parentheticals (checks.injection / checks.injection_prompt) that would read better split into a short purpose line plus behavior notes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a multi-check composite tool, the description covers what is evaluated, what comes back (verdict, score, per-check results, fixes), the PII thresholds that drive the verdict, and the external-call escape hatch. With annotations and an output schema present, nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns an extra point by explaining the consequence of the 'model' parameter ('none' skips the review and sends nothing out) rather than just listing model names, and by reinforcing that prompt is itself scanned for injection. It does not clarify the rule object shape beyond deferring to guardrail_test.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('one-call safety gate for an LLM response') and enumerates the exact checks performed: grounding, injection on response and prompt, toxicity, PII, guardrail rules, quality, and LLM review. It is clearly the aggregate sibling of toxicity_scan, prompt_injection_scan, redact_pii and guardrail_test, and an agent can tell it apart without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete conditional guidance: pass the prompt 'whenever you have it', set model to 'none' to skip the external LLM review, and rules use the same format as guardrail_test. What is missing is an explicit statement that this replaces the individual scanners (toxicity_scan, prompt_injection_scan, redact_pii) and when to prefer those lighter tools instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

similarity_scoreA
Read-onlyIdempotent
Inspect

Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.

NameRequiredDescriptionDefault
batchNoBatch mode: array of {reference, hypothesis} pairs.
metricsNoMetrics to compute (default: all). Options: "cosine_bow", "cosine_tfidf", "jaccard", "rouge1", "rouge2", "rougeL", "bleu"
referenceNoReference / expected text (ground truth)
thresholdNoOptional pass/fail threshold (0-1). Applies to ROUGE-L F1 score.
hypothesisNoHypothesis / actual text (LLM output)

Output Schema

NameRequiredDescription
f1No
modeNo
countNo
recallNo
resultsNo
precisionNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds only 'No API key needed' and 'Supports batch mode'; it says nothing about determinism of scoring, cost/latency for many metrics, or what happens when both single and batch inputs are given. Useful but thin beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with the verb and resource, followed by metrics, then use cases. Each sentence carries information; only the metric enumeration partially duplicates the schema, which is the one minor redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 5 documented parameters and an output schema present, the description need not explain return values. The main remaining gap is that zero parameters are marked required, yet the description never states the calling contract — supply either reference+hypothesis or a batch array — which an agent could get wrong.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents reference, hypothesis, batch, metrics, and the threshold-to-ROUGE-L-F1 rule. The description's metric listing and batch mention restate what the schema already says, adding no syntax, mutual-exclusivity, or default guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Compute text similarity between reference and hypothesis') and enumerates the concrete metric families (Cosine BoW/TF-IDF, Jaccard, ROUGE-1/2/L, BLEU), which clearly delimits it as a lexical/overlap scorer. It does not, however, name or distinguish itself from near siblings like embedding_similarity, vector_similarity, bm25_score, or response_quality_score, leaving the agent to infer the boundary from the metric list alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear positive usage contexts ('Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking') and notes batch mode, so an agent knows when the tool fits. There are no exclusions or explicit alternatives ('use embedding_similarity for semantic similarity'), so the guidance is one-sided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

skill_interview
Read-onlyIdempotent
Inspect

Help a person write a SKILL.md they can test. No answers: the questions (why + example), written for the person; invented answers make the result meaningless. With answers: remarks per answer (no model), the SKILL.md, scenarios for test_skill, a starter .ia-eval.yaml.

NameRequiredDescriptionDefault
nameNo
answersNoKeyed by question id.

Output Schema

NameRequiredDescription

No output parameters

sort_linesA
Read-onlyIdempotent
Inspect

Sort, deduplicate, reverse, or filter lines of text. Useful for cleaning import lists, dependencies, log files, and config entries.

NameRequiredDescriptionDefault
trimNoTrim whitespace from each line (default: true)
inputYesMulti-line text to process
filterNoFor "filter": keep lines containing this substring (case-insensitive)
operationNo"sort" (default), "sort_desc", "reverse", "deduplicate", "unique_sort", "filter"
remove_emptyNoRemove empty lines (default: true)

Output Schema

NameRequiredDescription
resultNo
removedNo
line_countNo
original_countNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds no behavioral detail beyond purpose — notably it doesn't disclose that trim and remove_empty default to true, meaning input lines are silently altered, though that is documented in the schema. With annotations carrying the load, a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste, with the operation set front-loaded before the use-case examples. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, full schema description coverage, and rich annotations, the description only needs to establish purpose and context, which it does. A brief note on default-altering behavior (trim/remove_empty) would make it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so every parameter is already documented in the schema. The description restates the same operation list the operation parameter enumerates and adds no format or syntax guidance beyond it. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states specific verbs and the resource ('Sort, deduplicate, reverse, or filter lines of text'), which is clearly distinguishable from text-transform siblings like diff_text or normalize_whitespace. It lacks any explicit naming of an alternative tool, but the operation set is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete usage contexts ('cleaning import lists, dependencies, log files, and config entries'), which tells the agent when to reach for it. It stops short of stating when NOT to use it or naming a sibling alternative, so it is clear context without exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

split_chunksA
Read-onlyIdempotent
Inspect

Split text into chunks of at most N tokens (cl100k_base: ~4 chars/token) with optional overlap. Designed for RAG ingestion pipelines.

NameRequiredDescriptionDefault
inputYesText to split into chunks
overlapNoToken overlap between consecutive chunks (default: 0)
chunk_tokensYesMaximum tokens per chunk (10–8000)

Output Schema

NameRequiredDescription
chunksNo
chunk_countNo
overlap_tokensNo
tokens_per_chunkNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is a safe, read-only, idempotent operation. The description adds useful behavioral context beyond that: tokenization uses cl100k_base with an approximate character-per-token ratio, and overlap is optional. It does not discuss error behavior or output structure, but the output schema covers the latter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and constraint, with no wasted wording. The intended use case is appended efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, read-only chunking utility with full schema coverage and an output schema, the description is sufficiently complete. It covers what the tool does, the tokenization model, overlap, and the primary use case without redundant detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all three parameters, including defaults and bounds. The description still adds meaning by identifying the tokenizer (cl100k_base) and the rough character-per-token approximation, which helps an agent reason about chunk_tokens beyond the raw numeric schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Split text into chunks') plus the key constraint ('at most N tokens') and optional overlap. It is clear what the tool does, though it does not explicitly name sibling tools such as truncate_to_tokens or count_tokens to sharpen differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear intended context: 'Designed for RAG ingestion pipelines.' This tells an agent when the tool is appropriate, but it does not state when not to use it or name alternative tools for related token operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ssl_certificate_checkA
Read-only
Inspect

Analyse the SSL/TLS certificate of any HTTPS host. Returns certificate subject, issuer, validity dates, days until expiry, protocol version, cipher suite, key exchange info, and an overall grade (A+, A, B, C, F). Detects expired, self-signed, and weak certificates. Use this to audit TLS posture before production deployment or during security reviews.

NameRequiredDescriptionDefault
hostYesHostname to check (e.g. example.com). Do not include https:// prefix.
portNoPort number (default: 443)

Output Schema

NameRequiredDescription
hostNo
gradeNo
cipherNo
issuerNo
issuesNo
subjectNo
protocolNo
valid_toNo
is_expiredNo
valid_fromNo
is_self_signedNo
days_until_expiryNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true and destructiveHint=false, so the safety profile is covered; the description adds real value by disclosing what is inspected (subject, issuer, expiry, protocol, cipher, key exchange) and what defects are detected. It omits failure behavior for unreachable hosts, non-HTTPS targets, or timeouts, which matters for an open-world network call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three front-loaded sentences with the purpose first and the usage cue last; nothing is filler. The return-field enumeration partially duplicates the output schema, which is mild redundancy but still aids tool selection.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values are already structured, and annotations cover safety; the description fills in the audit intent and defect detection. The only gap is error/edge-case behavior (invalid host, connection failure), which an agent would want before calling against arbitrary hosts.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both parameters (host, port) are documented in the schema including the no-https-prefix constraint and the 443 default. The description adds no parameter-level detail, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Analyse the SSL/TLS certificate of any HTTPS host') and enumerates the returned fields and grading scale, which cleanly separates it from siblings like security_headers_check or web_security_audit. An agent can identify the tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage contexts ('before production deployment or during security reviews'), which is clear guidance on when to reach for it. It stops short of naming when NOT to use it or pointing at a specific alternative sibling, so it is not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

strip_markdownA
Read-onlyIdempotent
Inspect

Strip all Markdown formatting (headers, bold, italic, code fences, links, lists) from text and return clean plain text. Run this before injecting scraped documentation, README files, or user content into an LLM prompt to eliminate redundant markup tokens and reduce cost.

NameRequiredDescriptionDefault
inputYesMarkdown text to convert to plain text

Output Schema

NameRequiredDescription
textNo
original_lengthNo
stripped_lengthNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the safety profile is covered. The description adds behavior beyond that by enumerating exactly which constructs are removed — including code fences, which is a non-obvious destructive-ish behavior inside an otherwise safe operation — and by stating the cost-reduction intent. It does not say how fence contents or inline code are treated, so it is useful but not exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: capability first, operational rationale second. No filler, no restating the tool name, and the actionable advice is front-loaded enough to be read at a glance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, fully annotated tool with an output schema, the description covers what the tool does and why an agent should reach for it. The only residual gap is edge-case behavior (code fence content, list nesting, whitespace normalization) which an agent might want before trusting output fidelity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

One parameter with 100% schema description coverage and a worked example in the schema, so the schema carries the semantics. The description adds nothing about format handling (e.g., whether code fence bodies are preserved or stripped), so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('strip') and resource ('Markdown formatting from text') and enumerates the constructs removed (headers, bold, italic, code fences, links, lists), plus the output form ('clean plain text'). An agent can immediately tell this performs the inverse of sibling html_to_markdown rather than token counting (count_tokens) or whitespace cleanup (normalize_whitespace).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit usage context: run it before injecting scraped docs, READMEs, or user content into a prompt, with the stated rationale (fewer markup tokens, lower cost). It lacks explicit exclusions or named alternatives (e.g., when to prefer normalize_whitespace or truncate_to_tokens instead), so it stops short of a full when/when-not treatment.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

system_prompt_builderA
Read-onlyIdempotent
Inspect

Build a structured system prompt from components: role, task, constraints, output format, tone, language, and examples. Generates a production-ready system prompt with token estimate.

NameRequiredDescriptionDefault
roleYesRole/persona (e.g. "Senior QA Engineer", "JSON extraction assistant")
taskNoMain task or objective
toneNoCommunication tone
examplesNoBrief examples to include
languageNoResponse language (e.g. "French")
constraintsNoRules and constraints to follow
output_formatNoExpected output format description

Output Schema

NameRequiredDescription
sectionsNo
system_promptNo
token_estimateNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so the safety profile is fully covered. The description adds only that output is 'production-ready' and includes a token estimate; it says nothing about how components are composed, whether unspecified fields are omitted or defaulted, or what the token estimate applies to. Useful but thin extra context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the first front-loads the input components, the second states the output and the token estimate. No filler, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and the annotations cover safety, so the description is largely sufficient for correct invocation. The remaining gap is sibling routing — nothing tells an agent to prefer this over build_rag_prompt or prompt_template_fill.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each of the seven properties is documented, including an enum for tone, so the schema carries the semantics. The description merely restates the same field names without adding format hints, defaults, or guidance such as what 'examples' expects as a string versus a list. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Build a structured system prompt') and enumerates the component inputs (role, task, constraints, output format, tone, language, examples), so an agent knows exactly what it produces. It does not differentiate itself from near-siblings such as build_rag_prompt, few_shot_formatter, or prompt_template_fill, which an agent would need to disambiguate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the component list suggests 'use this when you need to assemble a system prompt from parts', but there is no explicit when-to-use, no prerequisites, and no named alternative among the many prompt-related siblings. An agent can infer the context but must guess at routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_skillB
Read-only
Inspect

Test a SKILL.md. No model, every call: lint (gate on lint.valid) and review (description lacks what/when, words shared with other_skills, leftovers, no done-check, missing files; why + fix each). Unless lint_only, a model picks a skill per request from names+descriptions: trigger_accuracy PASS ≥80, FIX ≥50, else BLOCK; your scenarios = gate, generated = signal; unanswered = INCONCLUSIVE. Server key 10 runs/day.

NameRequiredDescriptionDefault
filesNoPaths in its folder.
modelNoDefault openai/gpt-oss-20b.
api_keyNoUsed once, not stored.
dir_nameNoIts folder.
skill_mdYesFull SKILL.md, frontmatter included.
lint_onlyNo
scenariosNoMax 20, near misses included.
other_skillsNoInstalled next to it.
scenario_countNo

Output Schema

NameRequiredDescription
lintNo
reviewNo
verdictNo

TDQS

B3.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds substantial context beyond them: lint is a hard gate on lint.valid, the model-based phase only runs when lint_only is false, scoring thresholds are PASS ≥80 / FIX ≥50 / else BLOCK, and unanswered scenarios yield INCONCLUSIVE. It also discloses a server-key rate limit of 10 runs/day, which is exactly the kind of constraint annotations cannot express. The one gap is that non-idempotence (idempotentHint=false) is never explained as model nondeterminism.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is compact and roughly front-loaded, with the core operations stated first. But the compression is aggressive to the point of opacity: 'leftovers', 'no done-check', 'why + fix each', and 'your scenarios = gate, generated = signal' are fragments that require re-reading. Length is fine; the telegraphic style costs clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain return values, and it does cover the lint gate, the review checks, the model phase, thresholds, and rate limits — a lot for a 9-parameter tool. What is missing is any usage framing relative to siblings and any clarification of the vague review check names, which leave an agent unsure what 'leftovers' or 'no done-check' actually detect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 78%, so the schema already documents most parameters and the baseline is 3. The description does add real meaning for a few: scenarios act as the gate while generated scenarios are only a signal, other_skills is used for description word-overlap checks, and lint_only short-circuits the model phase. It says nothing about model, api_key, dir_name, files, or scenario_count beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb and resource ('Test a SKILL.md') and then decomposes it into concrete operations: lint, review, and optional model-based trigger scoring. An agent can tell this validates a skill file rather than, say, interviewing its author. However, it never distinguishes itself from the sibling tool 'skill_interview', which plausibly covers adjacent ground.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit 'use this when' statement and no exclusion of alternatives such as skill_interview or run_eval_contract. The only routing signal is implicit: 'Unless lint_only, a model picks a skill...', which describes internal behavior rather than when an agent should choose this tool. The agent must infer usage from the operation list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

text_statsA
Read-onlyIdempotent
Inspect

Compute comprehensive statistics for any text: character count (with and without spaces), word count, line count, sentence count, paragraph count, and estimated reading time in minutes. Sentence counting is abbreviation-aware — titles (Mr., Dr.), acronyms (U.S., i.e., p.m.), initials, decimals, URLs and emails do not end a sentence, and a text with no terminal punctuation still counts as one. Use for validating form field lengths, evaluating LLM output verbosity, or content auditing.

NameRequiredDescriptionDefault
inputYesThe text to analyse

Output Schema

NameRequiredDescription
charsNo
linesNo
wordsNo
sentencesNo
paragraphsNo
chars_no_spaceNo
reading_time_minutesNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/idempotentHint/non-destructive/openWorld=false, so safety is covered. The description still adds genuine behavioral detail beyond them: sentence counting is abbreviation-aware (titles, acronyms, initials, decimals, URLs, emails do not terminate a sentence) and unpunctuated text counts as one sentence, plus reading time is an estimate in minutes. That is exactly the kind of non-obvious counting behavior an agent cannot infer from annotations or schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the metric inventory, then edge cases, then use cases — a clean most-important-first ordering. Every clause carries information; nothing repeats the name or the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be described, and annotations cover the safety profile for a pure read. With one required string parameter, the description supplies the remaining unknowns: what is measured, how the trickiest metric (sentences) behaves at the boundaries, and why an agent would call it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter has 100% schema description coverage ('The text to analyse'), so the schema carries the load and baseline 3 applies. The description adds only the framing 'any text' and implicit edge-case handling (URLs, emails, missing punctuation) — useful but not syntax or format guidance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

A specific verb (compute) plus resource (text statistics), with the exact metric set enumerated: character count with/without spaces, word, line, sentence, paragraph, and reading time. It is unambiguous what is returned. It never names a sibling it might be confused with (count_code_lines is the nearest overlap), so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The final sentence gives three concrete usage contexts — validating form field lengths, evaluating LLM output verbosity, content auditing — which tells an agent when this tool is the right pick. There are no explicit exclusions or named alternatives, so it is clear context without deliberately routing away from siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

timestamp_convertA
Read-only
Inspect

Convert between Unix timestamps (seconds or milliseconds) and ISO-8601 / UTC date strings. Auto-detects epoch vs. millisecond format. Omit input to get the current time. Returns iso, unix_s, unix_ms, utc, date, and time fields.

NameRequiredDescriptionDefault
inputNoUnix timestamp (number, seconds or ms) or ISO date string. Omit to get the current time.

Output Schema

NameRequiredDescription
isoNo
utcNo
dateNo
timeNo
unix_sNo
unix_msNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, and the description complements them by disclosing the return field set (iso, unix_s, unix_ms, utc, date, time) and the auto-detection behavior. The 'omit input returns current time' clause also coherently explains the idempotentHint=false annotation. It does not contradict anything, though it stays silent on edge cases like ambiguous digit counts.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core verb and resource, and the four sentences contain no filler. The closing enumeration of return fields is slightly redundant since an output schema already exists, which keeps it from a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations covering the safety profile, a documented parameter, and an output schema for return values, the description supplies everything needed to call it correctly. Minor gaps remain around timezone assumptions for ISO strings lacking an offset and unit-ambiguity rules, but these do not block correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter already documents 'Unix timestamp (number, seconds or ms) or ISO date string. Omit to get the current time.' The description largely restates this, adding only the millisecond auto-detection nuance, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and conversion pair ('Convert between Unix timestamps ... and ISO-8601 / UTC date strings') and explicitly covers both directions plus unit variants (seconds or milliseconds). Against siblings like number_base_convert or cron_parse, an agent can immediately tell this is the timestamp/date tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete usage context: 'Omit input to get the current time' and 'Auto-detects epoch vs. millisecond format,' which tells the agent it need not pre-normalize units. It stops short of naming an alternative or a when-not-to-use condition, so it lands just below the explicit-routing tier.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

token_budget_calculatorA
Read-onlyIdempotent
Inspect

Plan token allocation across system prompt, user input, context/RAG chunks, and expected output. Warns if budget exceeds the model's context window (any model from list_llm_models).

NameRequiredDescriptionDefault
modelYesModel id from list_llm_models (e.g. gpt-6.1-sol, claude-sonnet-5-5, gemini-3.8-flash)
contextNoActual context text (will estimate tokens)
user_inputNoActual user input text (will estimate tokens)
system_promptNoActual system prompt text (will estimate tokens)
context_tokensNoToken count for RAG context / documents
user_input_tokensNoToken count for user message
system_prompt_tokensNoToken count for system prompt
expected_output_tokensNoExpected max output tokens

Output Schema

NameRequiredDescription
modelNo
warningsNo
breakdownNo
context_windowNo
fits_in_windowNo
remaining_tokensNo
utilization_percentNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds useful behavior by stating that it warns when the budget exceeds the model's context window and that models come from list_llm_models. It does not explain token estimation from raw text or output behavior, though an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose and followed by the key warning behavior. There is no redundant or filler text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has rich structured context: full schema coverage, annotations, and an output schema. The description covers the purpose and warning condition adequately. A minor gap is that it does not clarify the text-vs-token input modes, but the schema handles that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all eight parameters, including the text-vs-token-count alternatives. The description reinforces the allocation categories and model source but adds no syntax or format details beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: planning token allocation across system prompt, user input, context/RAG chunks, and expected output. It also mentions the warning behavior for exceeding the model's context window. It does not explicitly contrast itself with siblings like count_tokens or context_window_check, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the planning and budget-warning framing, and the description points to list_llm_models for valid model IDs. However, it gives no explicit when-to-use guidance versus count_tokens, context_window_check, or estimate_llm_cost, and no exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

toxicity_scanA
Read-onlyIdempotent
Inspect

Scan text for toxic language, hate speech, bias/stereotype framing, violence, sexual and self-harm content. Lexical + structural pattern matching (identity term + predicate), not a semantic classifier — returns per-category risk plus the named rules that fired, so every finding can be checked. Languages: English for every category; French for insults, coarse language, direct threats and self-harm intent; anything else reads clean, so clean is not proof of safety. Text is also matched after undoing leetspeak and lookalike letters. Self-harm is critical only on a first-person intent — a mention ("call 988 if you have suicidal thoughts") is medium. Useful for LLM safety guardrail testing and triage; signal-only, not a calibrated CI gate.

NameRequiredDescriptionDefault
textYesText to scan
categoriesNoCategories to check (default: all)

Output Schema

NameRequiredDescription
methodNo
resultsNo
text_lengthNo
overall_riskNo
categories_checkedNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes far beyond the readOnly/idempotent annotations: states the detection method is lexical+structural rather than semantic, discloses per-language coverage gaps ('anything else reads clean, so clean is not proof of safety'), explains leetspeak/lookalike normalization, and gives the specific severity rule for self-harm. This is exactly the kind of behavioral context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense and front-loaded — coverage and method come first, caveats after — with no filler sentences. It reads as a long run-on block rather than structured clauses, which slightly hurts scannability for an agent parsing it, but every sentence carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a signal-only scanner with two parameters, the description covers method, language limits, normalization, severity edge cases, and intended use; an output schema exists, so it need not explain return shape, though it helpfully notes findings come with the rules that fired.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both `text` and `categories` are already documented, and the description adds little parameter-level detail beyond noting that text is pre-normalized before matching. With the schema doing the heavy lifting, baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource ('scan text for toxic language...'), enumerates exactly which categories are covered, and implicitly differentiates from the sibling bias_detect by scoping bias into a broader multi-category scan. An agent knows precisely what this tool does without inspecting the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly positions the tool ('useful for LLM safety guardrail testing and triage; signal-only, not a calibrated CI gate'), which tells the agent both when to reach for it and when not to rely on it. It stops short of naming sibling alternatives (e.g. bias_detect vs. this tool's bias category), so routing between the two still requires inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

transform_json_arrayA
Read-onlyIdempotent
Inspect

Transform a JSON array using common operations: pluck (extract specific fields), filter (by field value), sort_by (field), group_by (field), count_by (field), uniq_by (field). Useful for processing MCP tool results and LLM structured outputs.

NameRequiredDescriptionDefault
nNoFor first_n / last_n: number of items
pathNoOptional dot-notation path to the array within the JSON object (e.g. "data.items")
fieldNoField to operate on (for sort_by, group_by, count_by, uniq_by, filter)
inputYesThe JSON containing an array (or an object with an array at `path`) — a JSON string, or the value itself.
fieldsNoComma-separated field list for "pluck" (e.g. "id,name,email")
filter_opNoFor "filter": "==" | "!=" | ">" | ">=" | "<" | "<=" | "contains" | "exists" | "!exists"
operationYesOperation: "pluck", "filter", "sort_by", "group_by", "count_by", "uniq_by", "reverse", "first_n", "last_n", "flatten"
sort_orderNoFor sort_by: "asc" (default) or "desc"
filter_valueNoFor "filter": value to compare against

Output Schema

NameRequiredDescription
countNo
fieldNo
orderNo
totalNo
fieldsNo
resultNo
removedNo
operationNo
group_countNo
unique_valuesNo
removed_duplicatesNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and a closed-world scope, so the safety profile is fully covered structurally. The description adds only the post-processing use case and nothing about error behavior, invalid path handling, or how non-array input is treated — modest added value above annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences with no filler; the operation enumeration is the most useful part and comes first. The parenthetical glosses per operation are economical, though the list is incomplete.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and all 9 parameters are schema-documented. Given the tool's moderate complexity, the description plus structured fields are nearly sufficient; the gap is the missing mention of reverse/first_n/last_n/flatten and any guidance on the optional path parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter is documented with enum-like value lists, so the schema carries the semantic load. The description's operation list partially duplicates the schema and is actually incomplete relative to it, adding no syntax or interaction detail (e.g. how field pairs with filter_op).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Transform) and resource (JSON array) and enumerates the six primary operations, distinguishing it from simple converters like json_to_yaml or flatten_json. However, it omits four operations present in the schema (reverse, first_n, last_n, flatten), so the stated scope understates the tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Useful for processing MCP tool results and LLM structured outputs" implies a usage context but names no alternatives and no exclusions. Sibling tools such as flatten_json, extract_json_path, and sort_lines overlap enough that an agent would benefit from explicit routing guidance that is absent here.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

truncate_to_tokensA
Read-onlyIdempotent
Inspect

Truncate text to at most N tokens (cl100k_base: ~4 chars/token) to avoid exceeding an LLM context window. Optionally keeps the end of the text instead of the start (useful for keeping recent conversation history). Reports whether truncation occurred and the estimated token count.

NameRequiredDescriptionDefault
inputYesText to truncate
from_endNoKeep the end of the text instead of the start (default: false)
max_tokensYesMaximum number of tokens to keep

Output Schema

NameRequiredDescription
textNo
truncatedNo
tokens_estimateNo
original_tokens_estimateNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered. The description adds useful behavior the annotations cannot convey: the tokenizer and its ~4 chars/token approximation, the head-vs-tail retention option, and that the result reports whether truncation occurred.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core action, followed by the optional behavior and the return signal. No filler or redundant restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values needn't be spelled out, and the description still flags that truncation status and token count are reported. The approximation caveat on token counting is a valuable completeness touch; only a note on very short or empty input edge cases is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description goes beyond the schema by naming the cl100k_base tokenizer and its rough chars-per-token ratio, which tells the agent how max_tokens is actually interpreted rather than just restating 'Maximum number of tokens to keep'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Truncate text to at most N tokens') plus the mechanism and intent (avoid exceeding an LLM context window). This clearly separates it from siblings like count_tokens or context_window_check, which measure rather than modify text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete use case ('to avoid exceeding an LLM context window') and one scenario for the from_end flag ('keeping recent conversation history'). It doesn't explicitly name alternatives such as count_tokens or split_chunks or say when not to use it, which keeps it just under a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

unescape_htmlA
Read-onlyIdempotent
Inspect

Convert HTML entities (&amp;, &lt;, &gt;, &quot;, &#x27;, and numeric &#NNN;) back to plain characters. Use when processing HTML-encoded text from APIs, email content, or legacy database fields before passing to an LLM or displaying to users.

NameRequiredDescriptionDefault
inputYesHTML-encoded string to unescape

Output Schema

NameRequiredDescription
unescapedNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so the safety profile is covered. The description adds the supported entity set (named, decimal numeric, hex numeric), which is genuine spec detail. However it says nothing about unsupported/invalid entities, double-encoded input, or whether unknown sequences are left untouched — the main behavioral risk for an unescape operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: the first states what it does plus the entity grammar, the second states when to use it. Nothing redundant, and the core action sentence is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Return values are covered by the output schema and safety by annotations, so the description only needs purpose, use case and input expectations — all present. It is slightly short on edge-case behavior (invalid or nested entities), which an agent might want for malformed input.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a single required 'input' string, so the baseline is 3. The description earns above baseline by clarifying which entity syntaxes are recognized (&amp;, &lt;, &quot;, &#x27;, &#NNN;), which tells the agent what input forms are valid beyond the schema's one example.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb ('Convert ... back to plain characters') and resource ('HTML entities'), and enumerates the supported entity forms including numeric ones. The directionality ('back to plain characters', 'unescape') implicitly distinguishes it from the sibling escape_html, but that sibling is never named.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names concrete contexts where the tool applies (HTML-encoded text from APIs, email content, legacy database fields) and the downstream purpose (passing to an LLM or display). It does not name escape_html as the opposite-direction alternative, but the use case framing is clear with no ambiguity about when to reach for it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

url_decodeA
Read-onlyIdempotent
Inspect

Decode a percent-encoded URL string back to plain text. Use when parsing query parameters from raw URLs or when displaying encoded values to users.

NameRequiredDescriptionDefault
inputYesURL-encoded string to decode

Output Schema

NameRequiredDescription
decodedNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds usage context but says nothing extra about behavior — e.g., what happens on malformed input (invalid percent sequences) or whether it decodes '+' as space.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero waste; the core action is front-loaded before the usage guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, fully documented utility with an output schema and complete annotations, the description covers everything an agent needs to select and invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter with 100% schema description coverage plus an example, so the schema already documents it fully. The description adds no syntax or format detail beyond what the schema provides, which is the baseline expectation here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Decode a percent-encoded URL string back to plain text.' Specifying 'percent-encoded URL' clearly differentiates it from siblings like base64_decode, decode_jwt, and url_encode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names when to use it: 'parsing query parameters from raw URLs' or 'displaying encoded values to users.' It gives clear context but does not mention exclusions or point to url_encode for the reverse operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

url_encode
Read-onlyIdempotent
Inspect

Percent-encode a string for safe use in query strings, path segments or form-encoded bodies, so it cannot inject or break a URL.

NameRequiredDescriptionDefault
modeNo"component" (default) or "full" for encodeURI behavior
inputYesString to URL-encode

Output Schema

NameRequiredDescription
encodedNo
validate_agent_trajectoryA
Read-onlyIdempotent
Inspect

Run declarative assertions on an agent trace (OpenAI tool calls, Anthropic tool_use, LangChain run trees, ReAct logs). Deterministic, no LLM. Assertions: order (tool A before B), must_call, must_not_call, max/min calls, no_error, recovery (a successful step after the last error), no_injection, no_pii. A step is errored when the trace marks it (is_error, status error/failed, success:false) or its text starts with Error:/Traceback — prose is never keyword-scanned. Verdict PASS / FAIL / INCONCLUSIVE (0 steps parsed, never PASS).

NameRequiredDescriptionDefault
traceYesThe trace: JSON (messages array, run tree) or a Thought/Action/Observation text log.
formatNoDefault auto.
assertionsYes

Output Schema

NameRequiredDescription
stepsNo
resultsNo
summaryNo
verdictNo
warningsNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description adds substantial context beyond them: deterministic with no LLM, the exact rule for what counts as an errored step (is_error, status error/failed, success:false, or Error:/Traceback prefix), the guarantee that prose is never keyword-scanned, and the safety property that no_injection/no_pii report types and counts but never values. It also defines the INCONCLUSIVE verdict and the 0-steps-never-PASS rule.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and format support, then compressed into a dense assertion catalogue and verdict rules with no filler. The telegraphic fragment style and long parenthetical list are efficient but slightly harder to parse than plain prose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return shape needn't be restated, yet the description still supplies the verdict semantics and the empty-trace edge case an agent needs to interpret results. Combined with the error-detection rules and assertion semantics, nothing material is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

At 67% schema coverage the schema already documents most fields, but the description adds real semantics: what 'recovery' means (a successful step after the last error), what no_injection vs no_pii scan by default and how `where` overrides it, and how error state is detected. It does not cover `format` behavior or the 30-assertion cap, leaving a small gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Run declarative assertions on an agent trace") and enumerates the trace formats it accepts (OpenAI tool calls, Anthropic tool_use, LangChain run trees, ReAct logs). The assertion catalogue and the PASS/FAIL/INCONCLUSIVE verdict make it clearly distinguishable from siblings like function_call_validate or validate_mcp_response.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The assertion list implicitly tells an agent when this tool is applicable (post-hoc trace auditing), but there is no explicit when-to-use/when-not statement and no routing to alternatives such as prompt_injection_scan or function_call_validate, which overlap on no_injection and call-shape checks. Usage must be inferred from the capability list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_emailA
Read-onlyIdempotent
Inspect

Validate an email address against RFC 5322 syntax before storing it, sending a transactional email, or adding it to a mailing list. Returns { valid, email } — use this to avoid bounces and malformed data.

NameRequiredDescriptionDefault
emailYesEmail address to validate

Output Schema

NameRequiredDescription
emailNo
validNo
reasonNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the description only needs to add context beyond that, and it does: the RFC 5322 scope implies a pure syntax check with no network/MX lookup. The 'avoid bounces' claim slightly oversells a format check, keeping it at 4 rather than 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences with the action first and the use cases after. The 'Returns { valid, email }' fragment partially duplicates the existing output schema, which is the only wasted space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, the single parameter is fully documented in schema, annotations cover the safety profile, and an output schema exists, so return values need no explanation. Nothing an agent needs in order to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single 'email' parameter, so the baseline is 3, and the description earns one more by qualifying the validation rule as RFC 5322 syntax, which adds meaning beyond the schema's 'Email address to validate'. No format examples or edge cases (e.g., internationalized addresses) are given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Validate an email address') and narrows the standard to RFC 5322 syntax, which tells an agent exactly what kind of validation this is. No sibling tool in the list overlaps with email validation, so the purpose is unambiguous in the catalog.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives three concrete trigger contexts: before storing, sending a transactional email, or adding to a mailing list. It does not name an alternative tool or any when-not condition (e.g., that this is format-only and not a deliverability check), so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_mcp_responseA
Read-onlyIdempotent
Inspect

Validate that an MCP tool response conforms to expected format, schema, and content rules. Use this to QA-test any MCP server tool. Supply the tool's actual JSON result and a set of checks to perform.

NameRequiredDescriptionDefault
responseYesThe MCP tool result as a JSON string to validate
min_itemsNoIf response is an array, minimum number of items expected
expected_typeNoExpected top-level type: "object", "array", "string", "number"
required_keysNoComma-separated list of keys that MUST exist in the response (dot-notation for nested: "data.id, data.name")
actual_latencyNoActual measured latency in ms (from the call)
forbidden_keysNoComma-separated list of keys that MUST NOT exist (e.g. "password, secret, token")
max_size_bytesNoMaximum acceptable response size in bytes
max_response_msNoMaximum acceptable latency in ms (will be compared if provided)

Output Schema

NameRequiredDescription
totalNo
checksNo
failedNo
passedNo
verdictNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish that this is a safe, read-only, idempotent, non-destructive, closed-world operation. The description adds that it checks format, schema, and content rules, but it does not explain validation failure behavior, output shape, or other runtime details beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three efficient sentences, front-loaded with the core action and followed by usage and input expectations. Every sentence contributes without repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complete schema coverage, annotations, and presence of an output schema, the description is sufficient for an agent to understand and invoke the tool. It could be stronger by naming nearby alternatives or validation-specific edge cases, but no critical invocation information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all eight parameters are already documented in the input schema. The description only broadly mentions supplying the JSON result and a set of checks, and does not add syntax or semantics beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: validate an MCP tool response against format, schema, and content rules. It clearly distinguishes this as a QA-testing validator for MCP server tool outputs, though it does not explicitly contrast itself with close siblings such as mcp_schema_lint or function_call_validate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: use this to QA-test any MCP server tool, supplying the actual JSON result and checks. It does not state exclusions or name alternative tools, so it falls short of the top score, but the intended context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_urlA
Read-onlyIdempotent
Inspect

Parse and validate a URL. Returns decomposed components: protocol, hostname, port, path, query parameters, hash, and origin.

NameRequiredDescriptionDefault
inputYesURL to validate and parse

Output Schema

NameRequiredDescription
fullNo
hashNo
portNo
validNo
originNo
searchNo
hostnameNo
pathnameNo
protocolNo
query_paramsNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint=false, and destructiveHint=false, so the safety profile is clear. The description adds that the tool returns decomposed URL components, but it does not explain validation failure behavior, accepted URL schemes, or normalization rules.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and followed by the return components. Every sentence earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only parser with an output schema and complete annotation coverage, the description is nearly complete. It omits only edge-case behavior such as how malformed URLs are reported, which the output schema may cover.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the sole parameter is already documented as 'URL to validate and parse'. The description adds no extra parameter semantics beyond the schema, which is the expected baseline for a fully covered single-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb+resource: parse and validate a URL. It also states the return shape, making the tool's function unambiguous. It does not explicitly differentiate itself from any sibling, though none appear to be direct URL-parsing alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the tool name and description: call it when you need to parse or validate a URL. There are no explicit when-to-use conditions, exclusions, or named alternatives, so guidance is only minimally viable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vector_quantizeA
Read-onlyIdempotent
Inspect

Simulate int8 or int4 quantization of float32 embedding vectors. Reduces storage by 4x (int8) or 8x (int4). Returns quantized values, scale factor, and precision loss (MSE). Useful for understanding vector DB compression trade-offs.

NameRequiredDescriptionDefault
bitsNoQuantization bits: 8 (int8, default) or 4 (int4)
vectorYesFloat32 vector to quantize

Output Schema

NameRequiredDescription
mseNo
bitsNo
offsetNo
dimensionNo
quantizedNo
scale_factorNo
compression_ratioNo
storage_bytes_float32No
storage_bytes_quantizedNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and non-destructive behavior, so the safety profile is covered. The description adds useful context: the concrete compression ratios (4x/8x) and the fact that it 'simulates' rather than mutates storage. Return values are partly duplicated from the output schema but the ratio/trade-off framing adds value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with what the tool does, then ratios, return values, and motivation. Every sentence contributes distinct information with no wasted prose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema covering return values, 100% parameter coverage, and annotations carrying the safety profile, the description provides everything an agent needs to decide to call it and how. Nothing material is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are fully documented in the schema, so the description is expected to do little extra. It mentions int8/int4 which maps to the 'bits' parameter but adds no new syntax or format guidance beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('simulate quantization'), the exact resource ('float32 embedding vectors'), and the scope (int8 or int4). An agent can distinguish it from siblings like vector_stats or normalize_vector without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The final sentence gives a concrete use case ('understanding vector DB compression trade-offs'), providing clear positive context for invocation. It doesn't name alternatives like vector_stats or explicitly state when NOT to use it, keeping it just below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vector_similarityA
Read-onlyIdempotent
Inspect

Compute similarity/distance between two float vectors: cosine similarity, dot product, Euclidean and Manhattan distance. Essential for vector DB relevance scoring, embedding evaluation, and nearest-neighbor testing.

NameRequiredDescriptionDefault
metricNoDistance metric (default: all)
vector_aYesFirst vector as array of floats
vector_bYesSecond vector as array of floats

Output Schema

NameRequiredDescription
norm_aNo
norm_bNo
dimensionNo
dot_productNo
interpretationNo
cosine_distanceNo
cosine_similarityNo
euclidean_distanceNo
manhattan_distanceNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is fully covered. The description adds nothing beyond that – no mention of behavior on mismatched dimensions or zero vectors – so it contributes little behavioral value on top of structured data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the verb and resource, with the metric list and use cases packed efficiently. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full schema and an output schema in place, the description need not explain return values, and it covers purpose, metrics and validation contexts. Minor gaps like edge-case behavior (dimension mismatch) keep it short of a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the enum for metric is documented, so the schema carries the parameter burden. The description reinforces the metric names and clarifies inputs are floats, but adds no format or default detail beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (compute) and resource (similarity/distance between float vectors) and enumerates the four supported metrics, so the agent knows exactly what is calculated. It does not name or distinguish itself from related siblings such as similarity_score or embedding_similarity, leaving the agent to infer that this one operates on raw float vectors.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'vector DB relevance scoring, embedding evaluation, nearest-neighbor testing' clause implies usage contexts but gives no explicit when-to-use/when-not conditions and never points to an alternative tool. It is adequate but leaves the selection against siblings ambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vector_statsA
Read-onlyIdempotent
Inspect

Compute statistics for a float vector or matrix of vectors: mean, std, L2 norm, min, max, sparsity, top-K indices. Useful for debugging embedding quality and analyzing vector distributions in a vector DB.

NameRequiredDescriptionDefault
top_kNoReturn indices of top K absolute values (default: 5)
matrixNoMatrix of vectors (overrides vector). Returns per-vector + matrix-level stats. Required unless `vector` is given.
vectorNoSingle vector to analyze. Required unless `matrix` is given.

Output Schema

NameRequiredDescription
maxNo
minNo
stdNo
meanNo
l2_normNo
sparsityNo
dimensionNo
per_vectorNo
matrix_shapeNo
matrix_statsNo
top_k_indicesNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so the safety profile is covered by structured data and the bar is lower. The description adds computation semantics but nothing about output shape, defaults, ordering, or edge cases (empty vector, NaN handling) — the gaps in behavior are filled by the output schema rather than the prose.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste, with the capability statement front-loaded and the motivation second. Nothing in it repeats the schema or the annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema (return values need no explanation), complete parameter descriptions, and annotations covering safety, the description is sufficient to invoke the tool correctly. The only minor gap is that the anyOf constraint (exactly one of vector/matrix required) is left entirely to the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: vector, matrix and top_k each carry their own descriptions, including the 'matrix overrides vector' precedence. The description only echoes 'top-K indices' and adds no syntax, format, or default detail beyond what the schema already provides, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Compute statistics for a float vector or matrix of vectors') and enumerates the exact stats produced (mean, std, L2 norm, min, max, sparsity, top-K), which lets an agent distinguish it from vector_similarity or normalize_vector. It stops short of naming those siblings explicitly, so differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives a use context ('debugging embedding quality', 'analyzing vector distributions in a vector DB'), which is more than nothing. However, it names no alternatives and gives no when-not conditions (e.g. use vector_similarity for pairwise comparison, normalize_vector to transform), leaving routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webhook_endpoint_create
Destructive
Inspect

Create a temporary webhook endpoint that captures incoming HTTP requests for one hour. Returns the webhook id, public URL, expiration timestamp, and current request count. Use together with webhook_endpoint_requests to inspect captured payloads.

NameRequiredDescriptionDefault
base_urlNoOptional public base URL. Default: https://www.ia-qa.com/mcp/webhook (the apex ia-qa.com answers 301 and a redirected POST loses its body, so an apex base_url is normalized to www)

Output Schema

NameRequiredDescription
idNo
urlNo
expires_atNo
request_countNo
retention_minutesNo
webhook_endpoint_requestsA
Read-only
Inspect

Fetch the requests captured by a webhook created with webhook_endpoint_create. Returns the newest requests first with method, headers, query params, body payload, and timestamps.

NameRequiredDescriptionDefault
idYesWebhook id returned by webhook_endpoint_create
limitNoMaximum number of requests to return (1-100, default: 20)

Output Schema

NameRequiredDescription
idNo
requestsNo
expires_atNo
request_countNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered; the description adds useful behavior beyond that: newest-first ordering and the returned fields (method, headers, query params, body, timestamps). It does not cover retention/expiry or empty-result behavior, but adds real context over the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with no waste, and the resource definition plus the creation prerequisite are front-loaded before the return-shape detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and 100% param coverage, the description only needs to orient the agent, which it does. Minor gaps like retention window or behavior when no requests are captured keep it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both id and limit fully documented including the 1-100/default 20 bounds, so the description need not carry parameter meaning. The description adds nothing beyond the schema for parameters, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Fetch') plus a precise resource ('requests captured by a webhook'), and it anchors the resource to the sibling that creates it (webhook_endpoint_create). An agent can distinguish this from webhook_endpoint_create immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage by stating the id must come from webhook_endpoint_create, which tells the agent the prerequisite ordering. However, it gives no explicit when-to-use/when-not or alternative tools, so the guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

web_security_auditA
Read-only
Inspect

Run a comprehensive web security audit combining headers, SSL, CORS, and cookies checks — then use an LLM to produce a prioritised remediation plan. Orchestrates security_headers_check + ssl_certificate_check + cors_test + cookie_security_audit in parallel, merges all findings, then asks an AI model to: (1) rank vulnerabilities by real-world exploitability, (2) generate a remediation roadmap, (3) produce fix code snippets for the detected stack. Returns both raw audit data and the AI analysis. Use this as a one-click security posture assessment.

NameRequiredDescriptionDefault
urlYesFull URL to audit (e.g. https://example.com)
modelNoLLM model for AI analysis (default: "openai/gpt-oss-20b"). Set to "none" to skip AI analysis.
api_keyNoYour Groq or HuggingFace API key. Required to enable AI analysis.

Output Schema

NameRequiredDescription
fixNo
keyNo
urlNo
nameNo
weakNo
gradeNo
scoreNo
testsNo
valueNo
headerNo
issuesNo
secureNo
weightNo
cookiesNo
detailsNo
messageNo
missingNo
httpOnlyNo
sameSiteNo
risk_levelNo
weak_countNo
cookies_foundNo
missing_countNo
overall_gradeNo
origins_testedNo
total_findingsNo
headers_checkedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag readOnly/openWorld/destructive=false, but the description adds behavior beyond them: parallel fan-out, result merging, an external LLM step, and the requirement for a Groq/HuggingFace API key. It also discloses that AI analysis can be skipped via model='none'. It doesn't mention cost/latency of the external API call or the non-determinism implied by idempotentHint=false, leaving minor gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, followed by orchestration detail and the AI steps; nearly every sentence carries information. The numbered AI-step list and 'Returns both raw audit data and the AI analysis' are slightly over-detailed for a definition but still earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, annotations cover the safety profile, and all three params are documented. The description additionally explains the orchestration pipeline, the external API dependency, the skip option, and the return composition — everything an agent needs to invoke it correctly is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents url, model (including 'none' to skip), and api_key. The description's mentions of 'asks an AI model' and the fix-snippet generation add a little rationale for the model/api_key params but no syntax or constraints beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Run a comprehensive web security audit') and enumerates the concrete checks it performs (headers, SSL, CORS, cookies). It explicitly differentiates from siblings by naming the four tools it orchestrates, so an agent can distinguish this aggregate tool from the individual checks without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The closing line 'Use this as a one-click security posture assessment' gives clear usage context, and naming the component tools implies the alternative (call them individually). It does not explicitly state when *not* to use this heavier orchestration versus a single cheap check, so not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

word_frequencyA
Read-onlyIdempotent
Inspect

Analyze word frequency in text. Returns top N words with counts and percentages. Supports English stopword filtering. Useful for content analysis, keyword extraction, and LLM output analysis.

NameRequiredDescriptionDefault
inputYesText to analyze
top_nNoReturn top N words (default: 20, max: 200)
min_lengthNoMinimum word length to include (default: 3)
remove_stopwordsNoRemove common English stopwords (default: true)

Output Schema

NameRequiredDescription
top_wordsNo
total_wordsNo
unique_wordsNo
stopwords_removedNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this a safe, idempotent read, so the bar is lower. The description adds genuine behavioral context on top: the return payload ('counts and percentages') and the fact that stopword filtering is English-only and configurable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with what it does and what it returns. The final 'useful for' sentence is the weakest but still contributes selection context rather than padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter read-only analysis tool with an output schema and full annotation coverage, this is close to sufficient: purpose, return shape, and the stopword option are all covered. Only the min_length parameter goes unaddressed, and the schema handles it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with per-parameter descriptions and a worked example, so the schema carries the load. The description echoes 'top N' and 'stopword filtering' without adding syntax or defaults, and says nothing about min_length, so it earns the baseline rather than more.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Analyze word frequency in text') and adds the shape of the result ('top N words with counts and percentages'). It doesn't explicitly differentiate itself from the nearby text_stats or count_tokens siblings, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The closing list ('content analysis, keyword extraction, and LLM output analysis') implies when to reach for it, which is real guidance. But there is no explicit when-not and no routing away from text_stats or count_tokens, which are the obvious alternatives in this sibling set.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

xml_to_jsonA
Read-onlyIdempotent
Inspect

Convert an XML string to a JSON object. Supports attributes, nested elements, arrays, CDATA, and namespaces. Options: parse numbers, parse booleans, ignore attributes.

NameRequiredDescriptionDefault
inputYesXML string to convert
attr_prefixNoPrefix for attribute keys (default: "@_")
ignore_attrsNoIgnore XML attributes (default: false)
parse_valuesNoAuto-parse numbers and booleans (default: true)

Output Schema

NameRequiredDescription
resultNo
key_countNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so safety is covered. The description adds real behavioral value by listing what the converter handles (attributes, nested elements, arrays, CDATA, namespaces) and the default-on value parsing, which materially changes output shape. It does not say what happens with malformed XML, keeping it at 4.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action, then feature scope, then options. The options clause lightly duplicates schema fields but functions as a useful at-a-glance summary rather than padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return shape needn't be described, and annotations carry the safety profile. The description covers the conversion surface and key option effects; only error handling for invalid XML and the attr_prefix option are unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all four parameters and their defaults. The description restates the parse/ignore options but omits attr_prefix entirely and adds no syntax or format detail beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence gives a specific verb+resource ("Convert an XML string to a JSON object") that cleanly separates it from format siblings like yaml_to_json and json_to_yaml. The remaining sentences enumerate supported XML features, further pinning down scope. It stops short of naming a sibling explicitly, so 4 rather than 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the conversion semantics and the option list; there is no statement of when to prefer this over yaml_to_json or json_to_yaml, and no mention of preconditions (well-formed XML). For a self-evident utility that is adequate, but guidance is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

yaml_to_jsonA
Read-onlyIdempotent
Inspect

Parse a YAML string and return the equivalent JSON value. The reverse of json_to_yaml. Supports nested objects, arrays, anchors, aliases, multi-document streams, and all scalar types. Use when processing config files, CI/CD pipeline definitions, or OpenAPI specs authored in YAML.

NameRequiredDescriptionDefault
inputYesYAML string to parse
multiNoIf true, parse all documents in a multi-document stream and return an array (default: false)

Output Schema

NameRequiredDescription
jsonNo
countNo
documentsNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds genuinely useful behavioral context by enumerating supported YAML features (anchors, aliases, multi-document streams, all scalar types), which tells the agent what inputs will succeed. It omits error behavior on malformed YAML, keeping it below a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences with zero filler; the core action is front-loaded, followed by sibling differentiation, capability scope, and use cases. Every sentence contributes distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and the annotations carry the safety profile. The description covers purpose, scope, and use cases adequately, with only a minor gap in not linking the multi-document support to the `multi` parameter's effect on output shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both `input` and `multi` are already documented in the schema, setting the baseline at 3. The description's mention of multi-document stream support loosely corresponds to the `multi` flag but does not explain the flag or how the return shape changes, adding no meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Parse a YAML string and return the equivalent JSON value") and explicitly positions itself against the sibling json_to_yaml as its inverse. An agent can distinguish it from xml_to_json or format_json without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage contexts (config files, CI/CD pipeline definitions, OpenAPI specs authored in YAML) and names the inverse tool, which aids routing. It stops short of explicit when-not guidance or pointing to alternatives such as json_to_yaml for the opposite direction, so it is clear but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updates
    • Addedskill_interview
    • Changedtest_skill15 fields changed
      • changedInput schema / properties / api_key / description
        Previous value: -"Key for the chosen model's provider; not needed for the default."New value: +"Used once, not stored."
      • changedInput schema / properties / dir_name / description
        Previous value: -"Folder holding the SKILL.md; name must match it."New value: +"Its folder."
      • addedInput schema / properties / files
        Added value: +{
        +  "description": "Paths in its folder.",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • removedInput schema / properties / lint_only / description
        Removed value: -"Format check only: no model, no key. The CI gate for a SKILL.md."
      • changedInput schema / properties / model / description
        Previous value: -"e.g. gpt-4o-mini. Default openai/gpt-oss-20b (Groq, server key)."New value: +"Default openai/gpt-oss-20b."
      • changedInput schema / properties / other_skills / description
        Previous value: -"The skills installed next to this one. Default: six generic built-ins."New value: +"Installed next to it."
      • removedInput schema / properties / scenario_count / description
        Removed value: -"Requests to generate when scenarios is omitted (default 6)."
      • changedInput schema / properties / scenarios / description
        Previous value: -"Fixed requests, near misses included (should_trigger false). Max 20."New value: +"Max 20, near misses included."
      • changedInput schema / properties / skill_md / description
        Previous value: -"Full content of the SKILL.md file to test, YAML frontmatter included."New value: +"Full SKILL.md, frontmatter included."
      • addedOutput schema / properties / review
        Added value: +{
        +  "type": "object"
        +}
      • removedOutput schema / properties / scenarios
        Removed value: -{
        -  "items": {
        -    "type": "object"
        -  },
        -  "type": "array"
        -}
      • removedOutput schema / properties / scenarios_source
        Removed value: -{
        -  "type": "string"
        -}
      • removedOutput schema / properties / score
        Removed value: -{
        -  "type": "number"
        -}
      • removedOutput schema / properties / step_adherence
        Removed value: -{
        -  "type": "number"
        -}
      • removedOutput schema / properties / trigger_accuracy
        Removed value: -{
        -  "type": "number"
        -}
  2. 2 tool updates
    • Changedcors_checker13 fields changed
      • addedInput schema / properties / credentials
        Added value: +{
        +  "description": "True for fetch with credentials: \"include\"",
        +  "type": "boolean"
        +}
      • changedInput schema / properties / method / description
        Previous value: -"HTTP method to simulate (default: GET)"New value: +"Method the page uses (default: GET)"
      • changedInput schema / properties / origin / description
        Previous value: -"Origin header to simulate (default: https://yourdomain.com)"New value: +"Origin of the calling page (default: https://yourdomain.com)"
      • addedInput schema / properties / request_headers
        Added value: +{
        +  "description": "Headers the page sets, e.g. [\"Authorization\", \"Content-Type: application/json\"]. Names only are sent.",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • removedOutput schema / properties / allHeaders
        Removed value: -{}
      • addedOutput schema / properties / checks
        Added value: +{}
      • addedOutput schema / properties / fix
        Added value: +{}
      • removedOutput schema / properties / method
        Removed value: -{}
      • addedOutput schema / properties / reason
        Added value: +{}
      • addedOutput schema / properties / request_sent
        Added value: +{}
      • removedOutput schema / properties / testedOrigin
        Removed value: -{}
      • removedOutput schema / properties / url
        Removed value: -{}
      • addedOutput schema / properties / verdict
        Added value: +{
        +  "enum": [
        +    "allowed",
        +    "blocked",
        +    "unknown"
        +  ],
        +  "type": "string"
        +}
    • Changedfind_tool3 fields changed
      • changedInput schema / properties / category / description
        Previous value: -"Optional: filter by category — data | encoding | text | llm | qa | rag | dev | security | web"New value: +"Optional filter, one of the categories above. Any other value is refused."
      • changedInput schema / properties / max_results / description
        Previous value: -"Maximum tools to return (default 10, max 50). Results are ranked by IDF-weighted relevance, so common words like \"test\" do not inflate the list."New value: +"Maximum tools to return (default 10, max 50). Best match first."
      • changedInput schema / properties / query / description
        Previous value: -"Keyword(s) to search in tool name and description (e.g. \"cors\", \"token\", \"vector\", \"json\")"New value: +"Keywords, e.g. \"cors\", \"token\", \"vector\", \"json\""
  3. 1 tool update
    • Changedmcp_server_evaluate7 fields changed
      • addedInput schema / properties / test_tool_args
        Added value: +{
        +  "additionalProperties": true,
        +  "description": "Real arguments, sent to the tested server. Without them, required ones are invented.",
        +  "type": "object"
        +}
      • changedInput schema / properties / test_tool_name / description
        Previous value: -"Tool to call in the tools/call step. Without it, the first tool annotated readOnlyHint: true is called, or none."New value: +"Tool to call. Default: a read-only one, preferring no required arg."
      • addedOutput schema / properties / next_steps
        Added value: +{
        +  "type": "array"
        +}
      • addedOutput schema / properties / scope
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / tool_behaviour
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / what_this_does_not_prove
        Added value: +{
        +  "type": "array"
        +}
      • addedOutput schema / properties / what_this_proves
        Added value: +{
        +  "type": "array"
        +}
  4. 1 tool update
    • Changedrun_eval_contract3 fields changed
      • changedInput schema / examples
        Previous value: -[
        -  {
        -    "inline_contract": {
        -      "expectations": {
        -        "min_score": 70
        -      },
        -      "metadata": {
        -        "name": "smoke",
        -        "version": "1.0"
        -      },
        -      "scenarios": [
        -        {
        -          "ground_truth": "Paris",
        -          "id": "capital",
        -          "input": "Capital of France? One word."
        -        }
        -      ]
        -    }
        -  }
        -]New value: +[
        +  {
        +    "inline_contract": {
        +      "expectations": {
        +        "contains": [
        +          "Paris"
        +        ],
        +        "min_score": 70
        +      },
        +      "metadata": {
        +        "name": "smoke",
        +        "version": "1.0"
        +      },
        +      "scenarios": [
        +        {
        +          "ground_truth": "Paris",
        +          "id": "capital",
        +          "input": "Capital of France? One word."
        +        }
        +      ]
        +    }
        +  }
        +]
      • addedInput schema / properties / api_keys / properties / mistral
        Added value: +{
        +  "type": "string"
        +}
      • changedInput schema / properties / inline_contract / description
        Previous value: -"The contract as an object: \"metadata\" ({name, version, model?, provider?}), \"expectations\" ({min_score?}) and \"scenarios\" ([{id, input, ground_truth?}]). generate_eval_yaml scaffolds one."New value: +"The contract as an object: \"metadata\" ({name, version, model?, provider?}), \"expectations\", \"evaluators\" and \"scenarios\" ([{id, input, expectations?}]). generate_eval_yaml scaffolds one."
  5. 5 tool updates
    • Changedcompare_models1 field changed
      • changedInput schema / examples
        Previous value: -[
        -  {
        -    "models": [
        -      "gpt-4o",
        -      "claude-3-5-sonnet-20241022"
        -    ]
        -  }
        -]New value: +[
        +  {
        +    "models": [
        +      "gpt-6.1-sol",
        +      "claude-sonnet-5-5"
        +    ]
        +  }
        +]
    • Changedcontext_window_check1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Target model name (e.g. gpt-4o, claude-3.5-sonnet)"New value: +"Model id from list_llm_models (e.g. gpt-6.1-sol, claude-sonnet-5-5)"
    • Changedllm_generate1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b. Other ids from list_llm_models, HuggingFace ones included, are BYOK-only and will be rejected."New value: +"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b. Other ids from list_llm_models, HuggingFace ones included, are BYOK-only and will be rejected."
    • Changedshield_analyze1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b (default openai/gpt-oss-20b), or \"none\"."New value: +"LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default openai/gpt-oss-20b), or \"none\"."
    • Changedtoken_budget_calculator1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model name (e.g. gpt-4o, claude-3.5-sonnet, gemini-3.5-flash)"New value: +"Model id from list_llm_models (e.g. gpt-6.1-sol, claude-sonnet-5-5, gemini-3.8-flash)"
  6. 2 tool updates
    • Changedcompare_models9 fields changed
      • changedInput schema / properties / models / description
        Previous value: -"Array of 2-5 model names (e.g. [\"gpt-4o\",\"claude-3.5-sonnet\",\"gemini-3.5-flash\"])"New value: +"Array of 2-5 model names (e.g. [\"gpt-5\",\"claude-sonnet-4-6\",\"gemini-3.5-flash\"])"
      • removedInput schema / properties / use_case
        Removed value: -{
        -  "description": "Optimize recommendation for this criterion",
        -  "enum": [
        -    "cost",
        -    "context",
        -    "reasoning",
        -    "multimodal",
        -    "speed"
        -  ],
        -  "type": "string"
        -}
      • addedOutput schema / properties / cheapest
        Added value: +{}
      • removedOutput schema / properties / cost_per_1k_total
        Removed value: -{
        -  "type": "string"
        -}
      • addedOutput schema / properties / largest_context
        Added value: +{}
      • removedOutput schema / properties / model
        Removed value: -{}
      • addedOutput schema / properties / prices_as_of
        Added value: +{}
      • removedOutput schema / properties / recommendation
        Removed value: -{}
      • removedOutput schema / properties / use_case
        Removed value: -{}
    • Changedllm_fit_finder6 fields changed
      • changedInput schema / properties / top_n / description
        Previous value: -"Number of recommendations to return (default: 5)"New value: +"Number of models to return (default: 5)"
      • changedInput schema / properties / use_case / description
        Previous value: -"Primary use case: chatbot | code | rag | summarization | classification | reasoning | agents | multilingual"New value: +"Only \"reasoning\" filters (requires the feature); chatbot | code | rag | summarization | classification | agents | multilingual rank nothing"
      • addedOutput schema / properties / note
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / order
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / prices_as_of
        Added value: +{}
      • removedOutput schema / properties / score
        Removed value: -{
        -  "type": "number"
        -}
  7. 1 tool update
    • Changedresponse_quality_score1 field changed
      • changedInput schema / properties / expected_keywords / description
        Previous value: -"Keywords a correct answer must contain (case-insensitive). Scored as coverage."New value: +"Keywords a correct answer must contain, as whole words (case-insensitive; a plural in -s/-es counts). Scored as coverage."
  8. 152 tool updates
    • Changedab_test_report1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "variant_a": {
        +      "name": "A",
        +      "scores": [
        +        70,
        +        71,
        +        69
        +      ]
        +    },
        +    "variant_b": {
        +      "name": "B",
        +      "scores": [
        +        70,
        +        71,
        +        69
        +      ]
        +    }
        +  }
        +]
    • Changedanalyze_diff_bugs1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "version1": "if (amount > 0) charge(amount);",
        +    "version2": "if (amount >= 0) charge(amount);"
        +  }
        +]
    • Changedanalyze_responses1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "reference": "Paris is the capital of France.",
        +    "responses": [
        +      {
        +        "label": "A",
        +        "text": "Lyon is the capital."
        +      },
        +      {
        +        "label": "B",
        +        "text": "Paris is the capital of France."
        +      }
        +    ]
        +  }
        +]
    • Changedbase64_decode1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "aMOpbGxvIPCfjI0="
        +  }
        +]
    • Changedbase64_encode1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "héllo 🌍"
        +  }
        +]
    • Changedbias_detect1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "responses": [
        +      "He is great.",
        +      "She is great."
        +    ]
        +  }
        +]
    • Changedbm25_score1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "documents": [
        +      "shipping times",
        +      "refund for damaged items",
        +      "office hours"
        +    ],
        +    "query": "refund damaged"
        +  }
        +]
    • Changedbuild_rag_prompt1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "chunks": [
        +      {
        +        "source": "a.md",
        +        "text": "c1 text"
        +      }
        +    ],
        +    "query": "q?"
        +  }
        +]
    • Changedcalculate_readability1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "The cat sat on the mat. The dog ran."
        +  }
        +]
    • Changedcase_convert1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "helloWorld fooBar",
        +    "to": "snake"
        +  }
        +]
    • Changedcheck_contrast_ratio1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "background": "#fff",
        +    "foreground": "#000"
        +  }
        +]
    • Changedcolor_convert1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "rgb(18, 52, 86)"
        +  }
        +]
    • Changedcompare_models1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "models": [
        +      "gpt-4o",
        +      "claude-3-5-sonnet-20241022"
        +    ]
        +  }
        +]
    • Changedcompare_responses1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "response_a": "Paris is the capital.",
        +    "response_b": "Paris is the capital."
        +  }
        +]
    • Changedconsistency_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "responses": [
        +      "The capital of France is Paris.",
        +      "Paris is the capital of France."
        +    ]
        +  }
        +]
    • Changedcontext_window_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "messages": [
        +      {
        +        "content": "hi",
        +        "role": "user"
        +      }
        +    ],
        +    "model": "gpt-4o"
        +  }
        +]
    • Changedconversation_analyze1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "messages": [
        +      {
        +        "content": "What is your refund policy?",
        +        "role": "user"
        +      },
        +      {
        +        "content": "Refunds are available within 30 days.",
        +        "role": "assistant"
        +      }
        +    ]
        +  }
        +]
    • Changedcookie_security_audit1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "url": "https://www.ia-qa.com"
        +  }
        +]
    • Changedcors_checker1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "url": "https://www.ia-qa.com/mcp"
        +  }
        +]
    • Changedcors_test1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "origin": "https://other.example",
        +    "url": "https://www.ia-qa.com/mcp"
        +  }
        +]
    • Changedcot_analyzer1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "expected_conclusion": "7",
        +    "reasoning": "Step 1: 12 apples minus 5 eaten.\nStep 2: 12 - 5 = 7.\nTherefore, the answer is 7."
        +  }
        +]
    • Changedcount_code_lines1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "// c\nconst a = 1; // x\n\n/* b\n c */\nlet b;",
        +    "language": "javascript"
        +  }
        +]
    • Changedcount_tokens1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "Hello world, this is a test."
        +  }
        +]
    • Changedcreate_confluence_page1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "confluence_base_url": "https://yourcompany.atlassian.net",
        +    "confluence_email": "you@example.com",
        +    "confluence_token": "<your-atlassian-token>",
        +    "space_key": "QA",
        +    "test_suite": {
        +      "acceptance_criteria": [
        +        "A reset link expires 30 minutes after it is sent."
        +      ]
        +    }
        +  }
        +]
    • Changedcron_parse1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "expression": "*/5 * * * *"
        +  }
        +]
    • Changedcron_validator1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "expression": "0 0 29 2 *",
        +    "next_runs_count": 1
        +  }
        +]
    • Changeddecode_jwt1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "token": "eyJhbGciOiJIUzI1NiJ9.eyJleHAiOjQxMDI0NDQ4MDB9.sig"
        +  }
        +]
    • Changeddetect_language1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "Il rilascio è fallito perché la migrazione del database è scaduta dopo dieci minuti."
        +  }
        +]
    • Changeddetect_secrets1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "redis://cache:6379"
        +  }
        +]
    • Changeddiff_mappings1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "after": {
        +      "elements": [
        +        {
        +          "name": "Log in",
        +          "role": "button",
        +          "selector": "#submit"
        +        }
        +      ],
        +      "page": "login"
        +    },
        +    "before": {
        +      "elements": [
        +        {
        +          "name": "Sign in",
        +          "role": "button",
        +          "selector": "#submit"
        +        }
        +      ],
        +      "page": "login"
        +    }
        +  }
        +]
    • Changeddiff_text1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "a": "a\nb\nc",
        +    "b": "a\nB\nc"
        +  }
        +]
    • Changedembedding_similarity1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "text_a": "the cat sat",
        +    "text_b": "quantum field theory"
        +  }
        +]
    • Changedenv_parse1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "PORT=3000"
        +  }
        +]
    • Changedescape_html1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "<a href=\"x\">'&'</a>"
        +  }
        +]
    • Changedestimate_llm_cost1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input_tokens": 1000000,
        +    "model": "gpt-4o",
        +    "output_tokens": 1000000
        +  }
        +]
    • Changedextract_json_from_text1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "Here:\n```json\n{\"a\":[1,2]}\n```\nthanks"
        +  }
        +]
    • Changedextract_json_path1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": {
        +      "a": {
        +        "b": [
        +          {
        +            "c": 1
        +          },
        +          {
        +            "c": 2
        +          }
        +        ]
        +      }
        +    },
        +    "path": "a.b.0.c"
        +  }
        +]
    • Changedextract_links1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "nothing here"
        +  }
        +]
    • Changedextract_todos1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "// todo: refactor\n// todo(jc): rename"
        +  }
        +]
    • Changedfetch_confluence_page1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "confluence_base_url": "https://yourcompany.atlassian.net",
        +    "confluence_email": "you@example.com",
        +    "confluence_token": "<your-atlassian-token>",
        +    "page_id": "123456"
        +  }
        +]
    • Changedfetch_jira_issue1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "issue_key": "PROJ-1",
        +    "jira_base_url": "https://yourcompany.atlassian.net",
        +    "jira_email": "you@example.com",
        +    "jira_token": "<your-atlassian-token>"
        +  }
        +]
    • Changedfetch_veille_feed1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {}
        +]
    • Changedfew_shot_formatter1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "examples": [
        +      {
        +        "input": "a",
        +        "output": "b"
        +      },
        +      {
        +        "input": "c",
        +        "output": "d"
        +      }
        +    ]
        +  }
        +]
    • Changedfind_tool1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "query": "leaked api key"
        +  }
        +]
    • Changedfix_gherkin1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "api_key": "<your-api-key>",
        +    "gherkin": "Feature: Login\n  Scenario: ok\n    given I open the page\n    Then I see the form",
        +    "model": "gpt-4o-mini",
        +    "warnings": [
        +      "Step keyword \"given\" should be capitalised"
        +    ]
        +  }
        +]
    • Changedflatten_json1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": {
        +      "a": {
        +        "b": 1,
        +        "c": [
        +          1,
        +          2
        +        ]
        +      }
        +    }
        +  }
        +]
    • Changedformat_bytes1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "bytes": 1536,
        +    "standard": "iec"
        +  }
        +]
    • Changedformat_json1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "{\"a\":1,\"b\":{\"a\":2}}"
        +  }
        +]
    • Changedformat_table1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": [
        +      {
        +        "a": 1
        +      }
        +    ]
        +  }
        +]
    • Changedfunction_call_validate1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "allowed_functions": [
        +      {
        +        "name": "get_weather",
        +        "required_args": [
        +          "city"
        +        ]
        +      }
        +    ],
        +    "function_call": {
        +      "arguments": {
        +        "city": "Paris"
        +      },
        +      "name": "get_weather"
        +    }
        +  }
        +]
    • Changedgenerate_ci_workflow13 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {}
        +]
      • changedInput schema / properties / app_base_url / description
        Previous value: -"URL the drift gate waits for before capturing (default: http://127.0.0.1:3000). Must match config.baseUrl in .ia-qa/config.json."New value: +"[selector_drift] Default http://127.0.0.1:3000; must match .ia-qa/config.json."
      • changedInput schema / properties / app_start_command / description
        Previous value: -"Command that boots the app for the selector_drift gate, e.g. \"npm run start:ci\". Cannot be guessed — omitted, the step carries an explicit TODO and the note says so, because a plausible default would silently map nothing."New value: +"[selector_drift] Boots the app, e.g. \"npm run start:ci\". Never guessed: omitted, the step is a TODO."
      • changedInput schema / properties / cli_tools / description
        Previous value: -"IA-QA tool names to run as deterministic gates, e.g. [\"secret_scan\",\"prompt_injection_scan\"]. Tools with no known CI recipe get a --stdin step flagged in notes."New value: +"Tools to gate on, e.g. [\"secret_scan\",\"prompt_injection_scan\"]."
      • changedInput schema / properties / contract_path / description
        Previous value: -"Path to the .ia-eval.yaml contract, relative to the repo root (default: evals/smoke.ia-eval.yaml). Only used when the gate includes eval_contract."New value: +"Default evals/smoke.ia-eval.yaml."
      • changedInput schema / properties / cron / description
        Previous value: -"Cron expression when triggers include 'schedule' (default: '0 6 * * 1' — Mondays 06:00 UTC)."New value: +"For schedule (default '0 6 * * 1')."
      • changedInput schema / properties / fail_on_fail / description
        Previous value: -"Fail the build on a FAIL/PARTIAL verdict (default: true). Set false to report without gating."New value: +"Default true; false reports without gating."
      • changedInput schema / properties / gate / description
        Previous value: -"Which gate to emit. eval_contract = LLM eval via the action (default). cli_checks = deterministic CLI assertions. selector_drift = an E2E selector-drift gate via @ia-qa/self-healing (boots the app, captures, diffs against the committed baseline, branches on exit code 0/1/2). both = CLI checks + eval. all = CLI checks, then drift, then eval."New value: +"Default eval_contract. both = CLI checks + eval; all = CLI checks, drift, eval."
      • changedInput schema / properties / min_score / description
        Previous value: -"Override the contract min_score (0-100). Omit to use the value in the contract."New value: +"Override the contract min_score (0-100)."
      • changedInput schema / properties / node_version / description
        Previous value: -"Node version for the CLI steps (default: \"20\")."New value: +"Default \"20\"."
      • changedInput schema / properties / provider / description
        Previous value: -"LLM provider the contract runs against — decides which repository secret the workflow wires (default: groq)."New value: +"Decides which repo secret is wired (default groq)."
      • changedInput schema / properties / triggers / description
        Previous value: -"Workflow triggers (default: push + pull_request)."New value: +"Default push + pull_request."
      • removedInput schema / properties / workflow_name / description
        Removed value: -"Workflow display name (default: \"IA-QA Quality Gate\")."
    • Changedgenerate_curl1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "method": "DELETE",
        +    "url": "https://x.com"
        +  }
        +]
    • Changedgenerate_eval_yaml1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "description": "Summarize a customer support ticket in one sentence."
        +  }
        +]
    • Changedgenerate_hmac1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "algorithm": "sha256",
        +    "message": "msg",
        +    "secret": "k"
        +  }
        +]
    • Changedgenerate_html_report1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "results": {}
        +  }
        +]
    • Changedgenerate_json_ld1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "faq_items": [
        +      {
        +        "answer": "A.",
        +        "question": "Q?"
        +      }
        +    ],
        +    "type": "FAQPage"
        +  }
        +]
    • Changedgenerate_password1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "length": 24,
        +    "numbers": true,
        +    "symbols": true,
        +    "uppercase": true
        +  }
        +]
    • Changedgenerate_slug1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "Héllo Wörld 2026"
        +  }
        +]
    • Changedgenerate_test_cases1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "feature": "Login form with email and password"
        +  }
        +]
    • Changedgenerate_uuid1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "count": 5
        +  }
        +]
    • Changedget_testing_guidelines1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "topic": "rag-pipeline"
        +  }
        +]
    • Changedguardrail_test1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "response": "Your refund is approved.",
        +    "rules": [
        +      {
        +        "type": "must_include",
        +        "value": "refund"
        +      },
        +      "no_pii"
        +    ]
        +  }
        +]
    • Changedhallucination_check6 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "answer": "Le remboursement prend plus de 7 jours ouvrés.",
        +    "context": "Le remboursement prend plus de 7 jours ouvrés."
        +  }
        +]
      • changedInput schema / properties / answer / description
        Previous value: -"The LLM-generated answer to verify"New value: +"The LLM answer to verify (with context)."
      • addedInput schema / properties / cases
        Added value: +{
        +  "description": "Batch instead of answer + context: one result per case, and the worst gate overall.",
        +  "items": {
        +    "properties": {
        +      "answer": {
        +        "type": "string"
        +      },
        +      "context": {
        +        "type": "string"
        +      },
        +      "id": {
        +        "type": "string"
        +      }
        +    },
        +    "required": [
        +      "answer",
        +      "context"
        +    ],
        +    "type": "object"
        +  },
        +  "maxItems": 50,
        +  "type": "array"
        +}
      • changedInput schema / properties / context / description
        Previous value: -"The source/reference text that should ground the answer"New value: +"The source text that should ground it."
      • changedInput schema / properties / strict / description
        Previous value: -"If true, every sentence in the answer must be supported (default: false)"New value: +"Every sentence must be supported (default false)."
      • removedInput schema / required
        Removed value: -[
        -  "answer",
        -  "context"
        -]
    • Changedhash_text1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "algorithm": "md5",
        +    "input": "héllo wörld 🌍"
        +  }
        +]
    • Changedhtml_to_markdown1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "<h1>T</h1><p>a <strong>b</strong> <a href=\"https://x.com\">c</a></p><ul><li>i</li></ul>"
        +  }
        +]
    • Changedhttp_status_lookup1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "code": 418
        +  }
        +]
    • Changedidentify_caller1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {}
        +]
    • Changedjira_to_test_suite10 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "api_key": "<your-api-key>",
        +    "issue": {
        +      "description": "A registered user can request a reset link; the link expires after 30 minutes.",
        +      "key": "PROJ-1",
        +      "summary": "Password reset by email"
        +    },
        +    "model": "gpt-4o-mini"
        +  }
        +]
      • changedInput schema / properties / api_key / description
        Previous value: -"Your LLM provider API key (OpenAI sk-, Anthropic sk-ant-, Google AIzaSy-, etc.)."New value: +"Your LLM provider key."
      • changedInput schema / properties / confluence_pages / description
        Previous value: -"Optional array of pre-fetched Confluence page objects from fetch_confluence_page, used as documentation context."New value: +"Pages from fetch_confluence_page, as documentation."
      • changedInput schema / properties / issue / description
        Previous value: -"Pre-fetched issue object from fetch_jira_issue, OR a mock object with fields: key, summary, description (plain text or Markdown), status, issue_type, priority, labels, comments. Use this for offline/CI testing without Jira credentials."New value: +"The ticket: key, summary, description, and optionally status, issue_type, priority, labels, comments (fetch_jira_issue returns this shape)."
      • changedInput schema / properties / issue_key / description
        Previous value: -"Jira issue key to fetch automatically, e.g. \"PROJ-123\". Required if issue is not provided."New value: +"e.g. \"PROJ-123\", to fetch with the jira_* fields."
      • removedInput schema / properties / jira_base_url / description
        Removed value: -"Atlassian base URL. Required for auto-fetch mode."
      • removedInput schema / properties / jira_email / description
        Removed value: -"Atlassian account email. Required for auto-fetch mode."
      • removedInput schema / properties / jira_token / description
        Removed value: -"Atlassian API token. Required for auto-fetch mode."
      • changedInput schema / properties / max_tokens / description
        Previous value: -"Maximum tokens for the LLM response. Default: 8192. Increase for large tickets with many ACs; decrease to reduce cost on simple tickets."New value: +"Default 8192."
      • changedInput schema / properties / model / description
        Previous value: -"LLM model to use, e.g. \"gpt-4o-mini\", \"claude-3-5-haiku-20241022\", \"gemini-3.5-flash\"."New value: +"e.g. \"gpt-4o-mini\", \"claude-haiku-4-5-20251001\", \"gemini-3.5-flash\"."
    • Changedjson_diff1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "after": {
        +      "a": 1,
        +      "b": 2
        +    },
        +    "before": {
        +      "a": 1,
        +      "b": 2
        +    }
        +  }
        +]
    • Changedjson_schema_generate7 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": {
        +      "a": [
        +        1
        +      ],
        +      "b": true,
        +      "f": 1.5,
        +      "n": 1,
        +      "o": {
        +        "k": null
        +      },
        +      "s": "x"
        +    }
        +  }
        +]
      • changedInput schema / properties / required_all / description
        Previous value: -"Mark all detected object properties as required (default: true)"New value: +"Mark fields present in every observed object as required (default: true)"
      • addedInput schema / properties / samples
        Added value: +{
        +  "description": "More samples of the same shape, merged with input: a field seen as null and as a date becomes [\"string\",\"null\"]; one missing somewhere is not required.",
        +  "type": [
        +    "array",
        +    "string"
        +  ]
        +}
      • removedOutput schema / properties / format
        Removed value: -{}
      • removedOutput schema / properties / items
        Removed value: -{
        -  "type": "object"
        -}
      • removedOutput schema / properties / type
        Removed value: -{
        -  "type": "string"
        -}
      • addedOutput schema / properties / warnings
        Added value: +{
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
    • Changedjson_schema_validate1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "schema": {
        +      "$defs": {
        +        "P": {
        +          "properties": {
        +            "n": {
        +              "type": "string"
        +            }
        +          },
        +          "type": "object"
        +        }
        +      },
        +      "properties": {
        +        "p": {
        +          "$ref": "#/$defs/P"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    "value": {
        +      "p": {
        +        "n": "five"
        +      }
        +    }
        +  }
        +]
    • Changedjson_to_csv1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": [
        +      {
        +        "a": 1
        +      },
        +      {
        +        "b": 2
        +      }
        +    ]
        +  }
        +]
    • Changedjson_to_yaml1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": {
        +      "a": "x: y",
        +      "c": null,
        +      "d": "yes",
        +      "e": "0123"
        +    }
        +  }
        +]
    • Changedlatency_benchmark1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "endpoints": "https://www.ia-qa.com/mcp/ping",
        +    "iterations": 3
        +  }
        +]
    • Changedlevenshtein_distance1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "a": "kitten",
        +    "b": "sitting"
        +  }
        +]
    • Changedlint_commit_message1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "message": "feat!: drop node 18"
        +  }
        +]
    • Changedlist_llm_models1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "provider": "groq"
        +  }
        +]
    • Changedlist_local_tests1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {}
        +]
    • Changedllm_fit_finder1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "features": [
        +      "vision"
        +    ],
        +    "top_n": 5,
        +    "use_case": "chatbot"
        +  }
        +]
    • Changedllm_format_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "expected_format": "yaml",
        +    "output": "a: 1\nb: [1, 2]"
        +  }
        +]
    • Changedllm_generate1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "prompt": "Reply with the single word OK."
        +  }
        +]
    • Changedllm_json_schema_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "output": "{\"name\":\"a\",\"age\":3}",
        +    "schema": {
        +      "properties": {
        +        "age": {
        +          "minimum": 0,
        +          "type": "integer"
        +        },
        +        "name": {
        +          "type": "string"
        +        },
        +        "role": {
        +          "enum": [
        +            "admin",
        +            "user"
        +          ]
        +        }
        +      },
        +      "required": [
        +        "name",
        +        "age"
        +      ],
        +      "type": "object"
        +    }
        +  }
        +]
    • Changedllm_output_validator1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "expected_format": "json",
        +    "must_include": "a",
        +    "must_not_include": "TODO",
        +    "output": "{\"a\":1}"
        +  }
        +]
    • Changedlorem_ipsum1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "paragraphs": 3,
        +    "sentences_per_paragraph": 2
        +  }
        +]
    • Changedmcp_schema_lint1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "tool_definition": {
        +      "description": "Return the current weather for a city. Use it when the user asks about the weather now.",
        +      "inputSchema": {
        +        "properties": {
        +          "city": {
        +            "description": "City name, e.g. Paris",
        +            "type": "string"
        +          }
        +        },
        +        "required": [
        +          "city"
        +        ],
        +        "type": "object"
        +      },
        +      "name": "get_weather"
        +    }
        +  }
        +]
    • Changedmcp_server_evaluate1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "url": "https://www.ia-qa.com/mcp"
        +  }
        +]
    • Changedmcp_server_health_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "manifest": "{\"tools\":[{\"name\":\"get_weather\",\"description\":\"Return the current weather for a city. Use it when the user asks about the weather now.\",\"inputSchema\":{\"type\":\"object\",\"required\":[\"city\"],\"properties\":{\"city\":{\"type\":\"string\",\"description\":\"City name, e.g. Paris\"}}}}]}"
        +  }
        +]
    • Changedmerge_json1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "base": {
        +      "a": {
        +        "x": 1,
        +        "y": 2
        +      }
        +    },
        +    "override": {
        +      "a": {
        +        "y": 3
        +      }
        +    }
        +  }
        +]
    • Changedmetamorphic_check16 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "base": {
        +      "output": "Paris is the capital and largest city of France."
        +    },
        +    "variants": [
        +      {
        +        "output": "Paris is the capital and largest city of France.",
        +        "relation": "case"
        +      }
        +    ]
        +  }
        +]
      • changedInput schema / properties / api_key / description
        Previous value: -"OpenAI API key — required only when mode is embeddings."New value: +"OpenAI key, embeddings mode only."
      • changedInput schema / properties / base / description
        Previous value: -"The reference run: the original question and the answer your system produced for it."New value: +"The original run."
      • changedInput schema / properties / base / properties / output / description
        Previous value: -"Required — the answer your system produced for the original question."New value: +"The answer to the original question."
      • changedInput schema / properties / base / properties / question / description
        Previous value: -"Optional — the original question, echoed back in the output for readability."New value: +"Optional, echoed back."
      • changedInput schema / properties / baseline_guard / description
        Previous value: -"Correctness floor applied to the BASE answer before anything is scored. Failing it returns INVALID, not FAIL."New value: +"Floor on the BASE answer; failing it returns INVALID."
      • changedInput schema / properties / baseline_guard / properties / min_length / description
        Previous value: -"Minimum base answer length in chars (default 40)."New value: +"Default 40 chars."
      • changedInput schema / properties / baseline_guard / properties / must_not_match / description
        Previous value: -"Refusal patterns, matched case-insensitively in the FIRST 200 CHARS of the base answer (refusals lead; matching anywhere would flag a long correct answer that merely mentions one). Replaces the FR+EN default list, never merges — pass [] to disable."New value: +"Refusal patterns, checked in the first 200 chars. Replaces the default FR+EN list; [] disables."
      • changedInput schema / properties / mode / description
        Previous value: -"tfidf (default): free, lexical, deterministic — but a genuine paraphrase rarely reaches 0.80, so gate on case/typo and treat paraphrase as a trend. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key. translation requires this mode."New value: +"tfidf (default, lexical: gate on case/typo, read paraphrase as a trend) or embeddings (OpenAI, needs api_key; required for translation)."
      • changedInput schema / properties / require_all / description
        Previous value: -"If true (default), every gated variant must pass. KEEP THE DEFAULT for any run you gate on. Setting it false is not a tolerance dial but an off switch: relations have asymmetric pass rates (a typo variant usually scores ~1.0 because the answer really is identical), so one trivial row is enough to hold the whole run at PASS while a paraphrase fails. When that happens the result carries an explicit warning naming the failed rows."New value: +"Default true: every gated variant must pass. Keep it for a gate — false lets one easy typo row hold the run at PASS while a paraphrase fails."
      • changedInput schema / properties / thresholds / description
        Previous value: -"Per-relation threshold overrides. Calibrate on your own corpus before gating — the defaults are starting points, not measurements."New value: +"Per-relation overrides, 0-1. Calibrate on your own data before gating."
      • changedInput schema / properties / variants / description
        Previous value: -"Answers produced for transformed versions of the same question, each tagged with the relation that was applied."New value: +"Answers to transformed versions of the question."
      • removedInput schema / properties / variants / items / properties / id / description
        Removed value: -"Optional identifier (default: variant_<n>)."
      • changedInput schema / properties / variants / items / properties / output / description
        Previous value: -"Required — the answer your system produced for the transformed question."New value: +"The answer to the transformed question."
      • changedInput schema / properties / variants / items / properties / question / description
        Previous value: -"Optional — the transformed question, echoed back for readability."New value: +"Optional, echoed back."
      • changedInput schema / properties / variants / items / properties / relation / description
        Previous value: -"Transformation applied to the QUESTION. Decides the metric and the default threshold."New value: +"Transformation applied to the question; sets the metric and threshold."
    • Changedminify_js1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "code": "function add(a, b) {\n  return a + b;\n}"
        +  }
        +]
    • Changedmock_from_schema1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "count": 20,
        +    "schema": {
        +      "properties": {
        +        "email": {
        +          "format": "email",
        +          "type": "string"
        +        },
        +        "id": {
        +          "maximum": 9,
        +          "minimum": 5,
        +          "type": "integer"
        +        },
        +        "tag": {
        +          "enum": [
        +            "a",
        +            "b"
        +          ]
        +        }
        +      },
        +      "required": [
        +        "id",
        +        "email"
        +      ],
        +      "type": "object"
        +    }
        +  }
        +]
    • Changedmodel_info1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "model": "gpt-4o"
        +  }
        +]
    • Changedmultimodal_eval_guide17 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "metric": "clip_score",
        +    "score": 72
        +  }
        +]
      • removedInput schema / properties / action / description
        Removed value: -"guide (default) = reference thresholds/interpretation. clip_score/fid_score/vqa_accuracy = compute that metric. pipeline = run all three."
      • changedInput schema / properties / api_key / description
        Previous value: -"[vqa_accuracy] Your API key for the provider (BYOK)."New value: +"[vqa_accuracy] Key for the model's provider."
      • changedInput schema / properties / clip / description
        Previous value: -"[pipeline] {image_url, text} for CLIP."New value: +"[pipeline] {image_url, text}"
      • changedInput schema / properties / fid / description
        Previous value: -"[pipeline] {real_images, generated_images} for FID."New value: +"[pipeline] {real_images, generated_images}"
      • changedInput schema / properties / generated_images / description
        Previous value: -"[fid_score] Array of generated image URLs."New value: +"[fid_score] URLs."
      • changedInput schema / properties / image_base64 / description
        Previous value: -"[clip_score/vqa_accuracy] Base64-encoded image data."New value: +"[clip_score, vqa_accuracy]"
      • changedInput schema / properties / image_mime_type / description
        Previous value: -"[clip_score/vqa_accuracy] MIME type for base64 image."New value: +"[clip_score, vqa_accuracy]"
      • changedInput schema / properties / image_url / description
        Previous value: -"[clip_score/vqa_accuracy] Public URL of the image."New value: +"[clip_score, vqa_accuracy] Public image URL."
      • changedInput schema / properties / metric / description
        Previous value: -"[guide only] Metric to explain."New value: +"[guide]"
      • changedInput schema / properties / model / description
        Previous value: -"[vqa_accuracy] VLM model ID (default: gpt-4o)."New value: +"[vqa_accuracy] Default gpt-4o."
      • changedInput schema / properties / real_images / description
        Previous value: -"[fid_score] Array of real image URLs."New value: +"[fid_score] URLs."
      • changedInput schema / properties / score / description
        Previous value: -"[guide only] Optional score value to interpret."New value: +"[guide] A score to interpret."
      • changedInput schema / properties / system_prompt / description
        Previous value: -"[vqa_accuracy] Optional system prompt."New value: +"[vqa_accuracy]"
      • changedInput schema / properties / test_cases / description
        Previous value: -"[vqa_accuracy] Array of {question, accepted_answers} objects."New value: +"[vqa_accuracy]"
      • changedInput schema / properties / text / description
        Previous value: -"[clip_score only] Text description to compare against the image."New value: +"[clip_score] Text to compare with the image."
      • changedInput schema / properties / vqa / description
        Previous value: -"[pipeline] VQA config object (same inputs as vqa_accuracy)."New value: +"[pipeline] Same inputs as vqa_accuracy."
    • Changedneedle_haystack_generate1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "needle": "The code is 4242.",
        +    "position": "middle",
        +    "question": "What is the code?",
        +    "tokens": 2000
        +  }
        +]
    • Changednormalize_vector1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "vector": [
        +      3,
        +      4
        +    ]
        +  }
        +]
    • Changednormalize_whitespace1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "collapse_blanks": true,
        +    "input": "a  \r\nb\t\r\n\r\n\r\n\r\nc",
        +    "line_ending": "lf",
        +    "max_blank_lines": 1,
        +    "trim_lines": true
        +  }
        +]
    • Changednumber_base_convert1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "from_base": 16,
        +    "input": "ff",
        +    "to_base": 10
        +  }
        +]
    • Changedopenapi_validate1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": {
        +      "info": {
        +        "title": "t",
        +        "version": "1"
        +      },
        +      "openapi": "3.0.3",
        +      "paths": {
        +        "/a": {
        +          "get": {
        +            "operationId": "getA",
        +            "responses": {
        +              "200": {
        +                "description": "ok"
        +              }
        +            }
        +          }
        +        }
        +      }
        +    }
        +  }
        +]
    • Changedoptimize_prompt_tokens1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "text": "Please   kindly    make sure that you   basically   summarize   the following text."
        +  }
        +]
    • Changedparse_csv1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "delimiter": ";",
        +    "input": "a;b\n1;2"
        +  }
        +]
    • Changedparse_http_headers1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "headers": "Content-Type: text/html\nSet-Cookie: a=1\nSet-Cookie: b=2"
        +  }
        +]
    • Changedpost_jira_comment1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "issue_key": "PROJ-1",
        +    "jira_base_url": "https://yourcompany.atlassian.net",
        +    "jira_email": "you@example.com",
        +    "jira_token": "<your-atlassian-token>",
        +    "test_suite": {
        +      "acceptance_criteria": [
        +        "A reset link expires 30 minutes after it is sent."
        +      ]
        +    }
        +  }
        +]
    • Changedpr_gatekeeper1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "commit_message": "fix(pay): accept a zero amount",
        +    "diff": "diff --git a/pay.js b/pay.js\n--- a/pay.js\n+++ b/pay.js\n@@ -1 +1 @@\n-if (amount > 0) charge(amount);\n+if (amount >= 0) charge(amount);"
        +  }
        +]
    • Changedprompt_injection_scan1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "以前の注文をキャンセルしてください。"
        +  }
        +]
    • Changedprompt_template_fill1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "template": "Hi {{name}}",
        +    "variables": {
        +      "name": "A"
        +    }
        +  }
        +]
    • Changedprompt_test_suite1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "system_prompt": "You are a helpful assistant.",
        +    "user_prompt": "Say hi"
        +  }
        +]
    • Changedrag_relevance_rank1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "chunks": [
        +      "a b c",
        +      "d e f"
        +    ],
        +    "query": "quantum"
        +  }
        +]
    • Changedrate_tool1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "comment": "Formatted a 2 MB payload correctly.",
        +    "score": 4,
        +    "tool_name": "format_json"
        +  }
        +]
    • Changedredact_pii1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "order #12345678"
        +  }
        +]
    • Changedregex_test1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "flags": "g",
        +    "input": "a 12-34 b 5-6",
        +    "pattern": "\\d+-\\d+"
        +  }
        +]
    • Changedrerank_evaluate8 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "api_key": "<your-api-key>",
        +    "passages": [
        +      {
        +        "relevant": true,
        +        "text": "Paris is the capital of France."
        +      },
        +      {
        +        "relevant": false,
        +        "text": "Bananas are rich in potassium."
        +      }
        +    ],
        +    "query": "capital of France"
        +  }
        +]
      • changedInput schema / properties / api_key / description
        Previous value: -"Your NVIDIA API key (BYOK), used only when no passage carries a score. Transits RAM for the single call, never stored."New value: +"NVIDIA key, used only when no passage has a score; never stored."
      • changedInput schema / properties / passages / description
        Previous value: -"Array of passage objects to rank (min 2, max 20)"New value: +"2 to 20 passages."
      • changedInput schema / properties / passages / items / properties / relevant / description
        Previous value: -"Ground truth: is this passage relevant?"New value: +"Ground truth."
      • changedInput schema / properties / passages / items / properties / score / description
        Previous value: -"Relevance score from YOUR reranker. Present on every passage → ranking is done from these (offline, no key). Higher = more relevant. Score all passages or none."New value: +"Your reranker's score (higher = more relevant). On all passages or none."
      • removedInput schema / properties / query / description
        Removed value: -"The search query or question to rank against"
      • changedInput schema / properties / threshold / description
        Previous value: -"Minimum Precision@k to PASS (0-1, default 0.5)"New value: +"Minimum Precision@k to PASS (default 0.5)."
      • changedInput schema / properties / top_k / description
        Previous value: -"k for Precision@k evaluation (default 3)"New value: +"Default 3."
    • Changedresponse_quality_score1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "question": "q",
        +    "response": "anything"
        +  }
        +]
    • Changedrun_eval_contract4 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "inline_contract": {
        +      "expectations": {
        +        "min_score": 70
        +      },
        +      "metadata": {
        +        "name": "smoke",
        +        "version": "1.0"
        +      },
        +      "scenarios": [
        +        {
        +          "ground_truth": "Paris",
        +          "id": "capital",
        +          "input": "Capital of France? One word."
        +        }
        +      ]
        +    }
        +  }
        +]
      • changedInput schema / properties / api_keys / description
        Previous value: -"API keys to use for LLM generation (all optional — falls back to server env vars)"New value: +"Your keys per provider. Only Groq falls back to the server key; any other provider needs yours."
      • changedInput schema / properties / contract_path / description
        Previous value: -"Absolute or relative path to a .ia-eval.yaml file (required unless inline_contract is provided)"New value: +"A .ia-eval.yaml under the server's working directory (local use)."
      • changedInput schema / properties / inline_contract / description
        Previous value: -"Raw contract object (alternative to contract_path). Must contain top-level \"metadata\" ({name, version, model?, provider?}), \"expectations\" ({min_score?}), and \"scenarios\" ([{id, input, ground_truth?}]) — scenarios alone are rejected. Use generate_eval_yaml to scaffold one."New value: +"The contract as an object: \"metadata\" ({name, version, model?, provider?}), \"expectations\" ({min_score?}) and \"scenarios\" ([{id, input, ground_truth?}]). generate_eval_yaml scaffolds one."
    • Changedrun_pr_gate_pipeline1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "git_diff": "diff --git a/pay.js b/pay.js\n--- a/pay.js\n+++ b/pay.js\n@@ -1 +1 @@\n-if (amount > 0) charge(amount);\n+if (amount >= 0) charge(amount);"
        +  }
        +]
    • Changedrun_semantic_tests1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "cases": [
        +      {
        +        "actual": "Paris is the capital of France",
        +        "expected": "The capital of France is Paris"
        +      }
        +    ]
        +  }
        +]
    • Changedrun_vlm_test_suite14 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "api_key": "<your-api-key>",
        +    "image_url": "https://www.ia-qa.com/media/discover-report-poster.png",
        +    "model": "gemini-2.5-flash",
        +    "test_cases": [
        +      {
        +        "assertion_type": "contains",
        +        "assertion_value": "yes",
        +        "question": "Does the image contain text? Answer yes or no."
        +      }
        +    ]
        +  }
        +]
      • changedInput schema / properties / api_key / description
        Previous value: -"API key for the model provider (OpenAI sk-, Anthropic sk-ant-, or Google AIzaSy...)."New value: +"Key for the model's provider."
      • removedInput schema / properties / image_base64 / description
        Removed value: -"Base64-encoded image data (required unless image_url is provided)."
      • changedInput schema / properties / image_mime_type / description
        Previous value: -"MIME type of the image if using image_base64 (default: image/jpeg)."New value: +"For image_base64 (default image/jpeg)."
      • changedInput schema / properties / image_url / description
        Previous value: -"Public URL of the image to evaluate (required unless image_base64 is provided)."New value: +"Public image URL (or image_base64)."
      • removedInput schema / properties / model / description
        Removed value: -"VLM model to use."
      • changedInput schema / properties / model / enum
        Previous value: -[
        -  "gpt-4o",
        -  "gpt-4o-mini",
        -  "claude-3-5-sonnet-20241022",
        -  "claude-3-5-haiku-20241022",
        -  "gemini-3.5-flash",
        -  "gemini-2.5-flash"
        -]New value: +[
        +  "gpt-4o",
        +  "gpt-4o-mini",
        +  "claude-sonnet-4-6",
        +  "claude-haiku-4-5-20251001",
        +  "gemini-3.5-flash",
        +  "gemini-2.5-flash"
        +]
      • removedInput schema / properties / system_prompt / description
        Removed value: -"Optional system prompt sent to the VLM."
      • removedInput schema / properties / test_cases / description
        Removed value: -"Array of test cases to run."
      • removedInput schema / properties / test_cases / items / properties / assertion_type / description
        Removed value: -"Assertion to run on the VLM response. semantic_contains uses TF-IDF cosine similarity ≥ 0.4."
      • changedInput schema / properties / test_cases / items / properties / assertion_value / description
        Previous value: -"Expected value for the assertion (not needed for json_format)."New value: +"Not needed for json_format."
      • removedInput schema / properties / test_cases / items / properties / id / description
        Removed value: -"Optional identifier for this case."
      • removedInput schema / properties / test_cases / items / properties / question / description
        Removed value: -"Question to ask the VLM about the image."
      • changedInput schema / properties / threshold / description
        Previous value: -"Pass rate threshold for overall verdict (default: 80, 0–100)."New value: +"Pass rate for an overall PASS (default 80)."
    • Changedrun_vlm_test_suite_batch14 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "api_keys": {
        +      "gemini-2.5-flash": "<your-google-key>",
        +      "gpt-4o-mini": "<your-openai-key>"
        +    },
        +    "image_url": "https://www.ia-qa.com/media/discover-report-poster.png",
        +    "models": [
        +      "gpt-4o-mini",
        +      "gemini-2.5-flash"
        +    ],
        +    "test_cases": [
        +      {
        +        "question": "Does the image contain text? Answer yes or no."
        +      }
        +    ]
        +  }
        +]
      • changedInput schema / properties / api_keys / description
        Previous value: -"Map of model ID → API key. Example: { \"gpt-4o\": \"sk-...\", \"claude-3-5-sonnet-20241022\": \"sk-ant-...\" }"New value: +"Model id → API key."
      • removedInput schema / properties / image_base64 / description
        Removed value: -"Base64-encoded image data (required unless image_url is provided)."
      • changedInput schema / properties / image_mime_type / description
        Previous value: -"MIME type of the image if using image_base64 (default: image/jpeg)."New value: +"For image_base64 (default image/jpeg)."
      • changedInput schema / properties / image_url / description
        Previous value: -"Public URL of the image to evaluate (required unless image_base64 is provided)."New value: +"Public image URL (or image_base64)."
      • removedInput schema / properties / models / description
        Removed value: -"Array of model IDs to compare (runs in parallel)."
      • changedInput schema / properties / models / items / enum
        Previous value: -[
        -  "gpt-4o",
        -  "gpt-4o-mini",
        -  "claude-3-5-sonnet-20241022",
        -  "claude-3-5-haiku-20241022",
        -  "gemini-3.5-flash",
        -  "gemini-2.5-flash"
        -]New value: +[
        +  "gpt-4o",
        +  "gpt-4o-mini",
        +  "claude-sonnet-4-6",
        +  "claude-haiku-4-5-20251001",
        +  "gemini-3.5-flash",
        +  "gemini-2.5-flash"
        +]
      • removedInput schema / properties / system_prompt / description
        Removed value: -"Optional system prompt sent to every VLM."
      • removedInput schema / properties / test_cases / description
        Removed value: -"Array of test cases to run against every model."
      • removedInput schema / properties / test_cases / items / properties / assertion_type / description
        Removed value: -"Assertion to run on the VLM response."
      • changedInput schema / properties / test_cases / items / properties / assertion_value / description
        Previous value: -"Expected value for the assertion (not needed for json_format)."New value: +"Not needed for json_format."
      • removedInput schema / properties / test_cases / items / properties / id / description
        Removed value: -"Optional identifier for this case."
      • removedInput schema / properties / test_cases / items / properties / question / description
        Removed value: -"Question to ask the VLM about the image."
      • changedInput schema / properties / threshold / description
        Previous value: -"Pass rate threshold for overall verdict (default: 80, 0–100)."New value: +"Pass rate for an overall PASS (default 80)."
    • Changedsandbox_scenario1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "scenario": "swap-label"
        +  }
        +]
    • Changedscore_geo_signals1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "head_html": "<head><title>Refund policy | Shop</title><meta name=\"description\" content=\"How refunds work at Shop.\"></head>"
        +  }
        +]
    • Changedsearch_jira_issues1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "jira_base_url": "https://yourcompany.atlassian.net",
        +    "jira_email": "you@example.com",
        +    "jira_token": "<your-atlassian-token>",
        +    "jql": "project = PROJ AND status = \"To Do\""
        +  }
        +]
    • Changedsecret_scan4 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "redis://cache:6379"
        +  }
        +]
      • changedInput schema / properties / input / description
        Previous value: -"Text or code to scan for secrets"New value: +"Text to scan — or an array of texts / {id, text} for a batch (one result each, worst gate on top)."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "array"
        +]
      • changedInput schema / properties / types / description
        Previous value: -"Comma-separated families to scan (default: all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, twilio, sendgrid, jwt, private_key, connection_string, bearer, basic_auth, generic. Individual pattern names (e.g. \"aws_access_key\", \"github_fine\") are also accepted. An unknown value is rejected with an error — a scoped scan never silently returns \"clean\"."New value: +"Comma-separated families (default all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, teams, discord, cookie, twilio, sendgrid, jwt, private_key, connection_string, bearer, basic_auth, generic, or pattern names. An unknown value is rejected, never a silent \"clean\"."
    • Changedsecurity_headers_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "url": "https://www.ia-qa.com"
        +  }
        +]
    • Changedshield_analyze6 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "model": "none",
        +    "prompt": "When will my order arrive?",
        +    "response": "Your order #1234 will arrive tomorrow."
        +  }
        +]
      • changedInput schema / properties / model / description
        Previous value: -"LLM model for AI-powered deep analysis (default: \"openai/gpt-oss-20b\"). Set to \"none\" to skip LLM check. Supports any model from list_llm_models."New value: +"LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b (default openai/gpt-oss-20b), or \"none\"."
      • changedInput schema / properties / prompt / description
        Previous value: -"Optional original prompt. Used for quality scoring AND scanned for prompt injection in its own right (checks.injection_prompt) — pass it whenever you have it, it is where the attack actually lands."New value: +"The original prompt — also scanned for injection; pass it whenever you have it."
      • changedInput schema / properties / response / description
        Previous value: -"The LLM-generated response to analyze"New value: +"The LLM response to check."
      • changedInput schema / properties / rules / description
        Previous value: -"Optional guardrail rules array (same format as guardrail_test tool)"New value: +"Same format as guardrail_test."
      • changedInput schema / properties / source / description
        Previous value: -"Optional reference/source text for hallucination grounding check"New value: +"Reference text for the grounding check."
    • Changedsimilarity_score1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "hypothesis": "the cat sat on the mat",
        +    "reference": "the cat sat on the mat",
        +    "threshold": 0.5
        +  }
        +]
    • Changedsort_lines1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "b\na\nb\nc",
        +    "operation": "deduplicate"
        +  }
        +]
    • Changedsplit_chunks1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "chunk_tokens": 12,
        +    "input": "Refunds are accepted within 30 days. Damaged items are replaced for free. Contact support to start a return.",
        +    "overlap": 2
        +  }
        +]
    • Changedssl_certificate_check1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "host": "www.ia-qa.com"
        +  }
        +]
    • Changedstrip_markdown1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "# Title\n**bold** and [link](http://x.com) and `code`\n- item"
        +  }
        +]
    • Changedsystem_prompt_builder1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "constraints": [
        +      "no PII"
        +    ],
        +    "output_format": "json",
        +    "role": "support agent"
        +  }
        +]
    • Changedtest_skill8 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "skill_md": "---\nname: json-formatter\ndescription: Pretty-print a JSON string the user pastes.\n---\nFormat the JSON with 2-space indentation."
        +  }
        +]
      • changedInput schema / properties / api_key / description
        Previous value: -"API key for the chosen model provider. Not required when using the default Groq model."New value: +"Key for the chosen model's provider; not needed for the default."
      • changedInput schema / properties / dir_name / description
        Previous value: -"Name of the folder holding the SKILL.md. The spec requires name to match it; checked only when given."New value: +"Folder holding the SKILL.md; name must match it."
      • changedInput schema / properties / lint_only / description
        Previous value: -"Run the format check only: no model call, no key, deterministic. The CI gate for a SKILL.md — gate on lint.valid."New value: +"Format check only: no model, no key. The CI gate for a SKILL.md."
      • changedInput schema / properties / model / description
        Previous value: -"LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to openai/gpt-oss-20b (Groq, server key)."New value: +"e.g. gpt-4o-mini. Default openai/gpt-oss-20b (Groq, server key)."
      • changedInput schema / properties / other_skills / description
        Previous value: -"The other skills installed next to this one, [{\"name\", \"description\"}], so triggering is tested against the real neighbours. Default: six built-in generic skills."New value: +"The skills installed next to this one. Default: six generic built-ins."
      • changedInput schema / properties / scenario_count / description
        Previous value: -"Number of requests to generate when scenarios is omitted: half should-load, half near misses. Default: 6."New value: +"Requests to generate when scenarios is omitted (default 6)."
      • changedInput schema / properties / scenarios / description
        Previous value: -"Fixed user requests for a reproducible gate: [{\"prompt\": \"...\", \"should_trigger\": true}]. Include near misses with should_trigger false. Max 20. When omitted, scenario_count requests are generated."New value: +"Fixed requests, near misses included (should_trigger false). Max 20."
    • Changedtext_stats1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "One two three.\nFour five!\n\nSix?"
        +  }
        +]
    • Changedtimestamp_convert1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": 0
        +  }
        +]
    • Changedtoken_budget_calculator1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "context_tokens": 1000,
        +    "expected_output_tokens": 500,
        +    "model": "gpt-4o"
        +  }
        +]
    • Changedtoxicity_scan1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "text": "Ce contrat est conclu avec un partenaire."
        +  }
        +]
    • Changedtransform_json_array1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "field": "v",
        +    "input": [
        +      {
        +        "v": 2
        +      },
        +      {
        +        "v": 10
        +      },
        +      {
        +        "v": 1
        +      }
        +    ],
        +    "operation": "sort_by",
        +    "sort_order": "desc"
        +  }
        +]
    • Changedtruncate_to_tokens1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "short",
        +    "max_tokens": 50
        +  }
        +]
    • Changedunescape_html1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "&#128512; &#x1F600;"
        +  }
        +]
    • Changedurl_decode1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "a%20b%26c"
        +  }
        +]
    • Changedurl_encode1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "a b&c=d/é"
        +  }
        +]
    • Changedvalidate_agent_trajectory14 fields changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "assertions": [
        +      {
        +        "tool": "search",
        +        "type": "must_call"
        +      }
        +    ],
        +    "trace": [
        +      {
        +        "role": "assistant",
        +        "tool_calls": [
        +          {
        +            "function": {
        +              "arguments": "{}",
        +              "name": "search"
        +            }
        +          }
        +        ]
        +      }
        +    ]
        +  }
        +]
      • removedInput schema / properties / assertions / description
        Removed value: -"List of assertions to validate against the trace."
      • changedInput schema / properties / assertions / items / properties / after / description
        Previous value: -"[order] Tool that must be called after."New value: +"[order] Called after."
      • changedInput schema / properties / assertions / items / properties / before / description
        Previous value: -"[order] Tool that must be called first."New value: +"[order] Called first."
      • removedInput schema / properties / assertions / items / properties / id / description
        Removed value: -"Optional assertion identifier."
      • changedInput schema / properties / assertions / items / properties / max / description
        Previous value: -"[max_calls] Maximum number of allowed calls."New value: +"[max_calls]"
      • changedInput schema / properties / assertions / items / properties / min / description
        Previous value: -"[min_calls] Minimum number of required calls."New value: +"[min_calls]"
      • changedInput schema / properties / assertions / items / properties / sensitivity / description
        Previous value: -"[no_injection] Default low = critical patterns only (override, jailbreak, fake role delimiters). Higher catches more and fires on ordinary text (\"act as\", {{templates}}) — pattern matching, not a classifier."New value: +"[no_injection] Default low (critical patterns only); higher also fires on ordinary text."
      • changedInput schema / properties / assertions / items / properties / tool / description
        Previous value: -"Tool name to check (for must_call, must_not_call, max_calls, min_calls)."New value: +"For must_call, must_not_call, max_calls, min_calls."
      • changedInput schema / properties / assertions / items / properties / type / description
        Previous value: -"Assertion type. no_injection runs prompt_injection_scan on every tool result (indirect injection: text a third party wrote that the agent then reads); no_pii runs redact_pii on the final answer (a leak to the user). Both report types and counts, never the matched values."New value: +"no_injection scans tool results (indirect injection); no_pii scans the final answer. Both report types and counts, never values."
      • changedInput schema / properties / assertions / items / properties / types / description
        Previous value: -"[no_pii] Comma-separated PII types (email, phone, ssn, credit_card, ip_address, jwt). Default: all."New value: +"[no_pii] Comma-separated: email, phone, ssn, credit_card, ip_address, jwt. Default all."
      • changedInput schema / properties / assertions / items / properties / where / description
        Previous value: -"[no_injection, no_pii] What to scan. Defaults: no_injection → tool_results, no_pii → answer (PII in a tool result is often legitimate — an order carries the customer email)."New value: +"[no_injection, no_pii] Override what is scanned."
      • changedInput schema / properties / format / description
        Previous value: -"Trace format. auto (default) detects automatically."New value: +"Default auto."
      • changedInput schema / properties / trace / description
        Previous value: -"Agent execution trace as JSON (OpenAI messages array, LangChain run tree) or plain text log (Thought/Action/Observation format)."New value: +"The trace: JSON (messages array, run tree) or a Thought/Action/Observation text log."
    • Changedvalidate_email1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "email": "john.doe@example.com"
        +  }
        +]
    • Changedvalidate_mcp_response1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "forbidden_keys": "password",
        +    "response": "{\"user\":{\"name\":\"x\"}}"
        +  }
        +]
    • Changedvalidate_url1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "https://www.ia-qa.com/mcp?x=1#y"
        +  }
        +]
    • Changedvector_quantize1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "vector": [
        +      0.1,
        +      -0.5,
        +      0.9,
        +      0
        +    ]
        +  }
        +]
    • Changedvector_similarity1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "vector_a": [
        +      1,
        +      0
        +    ],
        +    "vector_b": [
        +      0,
        +      1
        +    ]
        +  }
        +]
    • Changedvector_stats1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "vector": [
        +      1,
        +      2,
        +      3,
        +      4
        +    ]
        +  }
        +]
    • Changedweb_security_audit1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "model": "none",
        +    "url": "https://www.ia-qa.com"
        +  }
        +]
    • Changedwebhook_endpoint_create1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {}
        +]
    • Changedwebhook_endpoint_requests1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "id": "<id returned by webhook_endpoint_create>"
        +  }
        +]
    • Changedword_frequency1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "the cat and the hat and the bat",
        +    "remove_stopwords": false,
        +    "top_n": 1
        +  }
        +]
    • Changedxml_to_json1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "<r a=\"1\"><i>x</i><i>y</i></r>"
        +  }
        +]
    • Changedyaml_to_json1 field changed
      • addedInput schema / examples
        Added value: +[
        +  {
        +    "input": "a: 1\nb: [x, y]"
        +  }
        +]
  9. 1 tool update
    • Changedrun_eval_contract2 fields changed
      • addedInput schema / properties / api_keys / properties / deepseek
        Added value: +{
        +  "type": "string"
        +}
      • addedInput schema / properties / api_keys / properties / xai
        Added value: +{
        +  "type": "string"
        +}
  10. 1 tool update
    • Changedllm_generate1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected."New value: +"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b. Other ids from list_llm_models, HuggingFace ones included, are BYOK-only and will be rejected."
  11. 2 tool updates
    • Changedfunction_call_validate4 fields changed
      • changedInput schema / properties / allowed_functions / description
        Previous value: -"List of allowed function definitions"New value: +"List of allowed function definitions, in OpenAI, MCP or Anthropic form, or as {name, required_args, optional_args}"
      • addedInput schema / properties / allowed_functions / items / properties / inputSchema
        Added value: +{
        +  "additionalProperties": true,
        +  "description": "MCP form",
        +  "type": "object"
        +}
      • addedInput schema / properties / allowed_functions / items / properties / input_schema
        Added value: +{
        +  "additionalProperties": true,
        +  "description": "Anthropic form",
        +  "type": "object"
        +}
      • addedInput schema / properties / allowed_functions / items / properties / parameters
        Added value: +{
        +  "additionalProperties": true,
        +  "description": "OpenAI form: a JSON Schema of the arguments",
        +  "type": "object"
        +}
    • Changedredact_pii1 field changed
      • changedInput schema / properties / types / description
        Previous value: -"Comma-separated types to redact (default: all). Options: email, phone, ssn, credit_card, ip_address, jwt"New value: +"Comma-separated types to redact (default: all). Options: email, phone, ssn, iban, credit_card, ip_address, jwt"
  12. 7 tool updates
    • Changedcompare_models1 field changed
      • changedInput schema / properties / models / description
        Previous value: -"Array of 2-5 model names (e.g. [\"gpt-4o\",\"claude-3.5-sonnet\",\"gemini-2.0-flash\"])"New value: +"Array of 2-5 model names (e.g. [\"gpt-4o\",\"claude-3.5-sonnet\",\"gemini-3.5-flash\"])"
    • Changedjira_to_test_suite1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"LLM model to use, e.g. \"gpt-4o-mini\", \"claude-3-5-haiku-20241022\", \"gemini-2.0-flash\"."New value: +"LLM model to use, e.g. \"gpt-4o-mini\", \"claude-3-5-haiku-20241022\", \"gemini-3.5-flash\"."
    • Changedllm_generate1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.6-27b; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected."New value: +"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected."
    • Changedmodel_info1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model name (e.g. \"gpt-4o\", \"claude-3.5-sonnet\", \"gemini-2.5-pro\")"New value: +"Model name (e.g. \"gpt-4o\", \"claude-3.5-sonnet\", \"gemini-3.1-pro-preview\")"
    • Changedrun_vlm_test_suite1 field changed
      • changedInput schema / properties / model / enum
        Previous value: -[
        -  "gpt-4o",
        -  "gpt-4o-mini",
        -  "claude-3-5-sonnet-20241022",
        -  "claude-3-5-haiku-20241022",
        -  "gemini-1.5-flash",
        -  "gemini-2.0-flash"
        -]New value: +[
        +  "gpt-4o",
        +  "gpt-4o-mini",
        +  "claude-3-5-sonnet-20241022",
        +  "claude-3-5-haiku-20241022",
        +  "gemini-3.5-flash",
        +  "gemini-2.5-flash"
        +]
    • Changedrun_vlm_test_suite_batch1 field changed
      • changedInput schema / properties / models / items / enum
        Previous value: -[
        -  "gpt-4o",
        -  "gpt-4o-mini",
        -  "claude-3-5-sonnet-20241022",
        -  "claude-3-5-haiku-20241022",
        -  "gemini-1.5-flash",
        -  "gemini-2.0-flash"
        -]New value: +[
        +  "gpt-4o",
        +  "gpt-4o-mini",
        +  "claude-3-5-sonnet-20241022",
        +  "claude-3-5-haiku-20241022",
        +  "gemini-3.5-flash",
        +  "gemini-2.5-flash"
        +]
    • Changedtoken_budget_calculator1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model name (e.g. gpt-4o, claude-3.5-sonnet, gemini-2.0-flash)"New value: +"Model name (e.g. gpt-4o, claude-3.5-sonnet, gemini-3.5-flash)"

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    D
    maintenance
    Provides MCP tools that give LLM agents a full QA engineer workflow: scanning projects, generating deterministic test suites, executing them across browser/API/mobile, diagnosing failures, and proposing fixes that require human approval.
    -
  • A
    license
    Not graded
    quality
    D
    maintenance
    Multi-tool MCP server for AI agents with 29 tools across web scraping, SEO analysis, screenshot and PDF generation, domain intelligence, content extraction, multi-chain EVM blockchain queries, and security toolkit. Free tier available with no auth required.
    33 npm
    1
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.