Skip to main content
Glama

IA-QA — 152 QA & Dev Tools for AI Agents

shield_analyze

Read-only

One-call safety gate for an LLM response: grounding (with source), prompt injection on the response AND the prompt (checks.injection / checks.injection_prompt), toxicity, PII, guardrail rules, quality, plus an LLM review on Groq (server key; model "none" skips it and sends nothing out). Returns PASS/FIX/BLOCK, a 0-100 safety score, per-check results and fixes. PII gates the verdict: SSN or card → BLOCK, email → FIX, phone/IP flagged only.

Input Schema

NameRequiredDescriptionDefault
modelNoLLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default openai/gpt-oss-20b), or "none".
rulesNoSame format as guardrail_test.
promptNoThe original prompt — also scanned for injection; pass it whenever you have it.
sourceNoReference text for the grounding check.
responseYesThe LLM response to check.

Output Schema

NameRequiredDescriptionDefault
flagsNo
gradeNo
scoreNo
checksNo
verdictNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / model / description
      Previous value: -"LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b (default openai/gpt-oss-20b), or \"none\"."New value: +"LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default openai/gpt-oss-20b), or \"none\"."
  2. Changed6 schema fields changed
    • addedInput schema / examples
      Added value: +[
      +  {
      +    "model": "none",
      +    "prompt": "When will my order arrive?",
      +    "response": "Your order #1234 will arrive tomorrow."
      +  }
      +]
    • changedInput schema / properties / model / description
      Previous value: -"LLM model for AI-powered deep analysis (default: \"openai/gpt-oss-20b\"). Set to \"none\" to skip LLM check. Supports any model from list_llm_models."New value: +"LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b (default openai/gpt-oss-20b), or \"none\"."
    • changedInput schema / properties / prompt / description
      Previous value: -"Optional original prompt. Used for quality scoring AND scanned for prompt injection in its own right (checks.injection_prompt) — pass it whenever you have it, it is where the attack actually lands."New value: +"The original prompt — also scanned for injection; pass it whenever you have it."
    • changedInput schema / properties / response / description
      Previous value: -"The LLM-generated response to analyze"New value: +"The LLM response to check."
    • changedInput schema / properties / rules / description
      Previous value: -"Optional guardrail rules array (same format as guardrail_test tool)"New value: +"Same format as guardrail_test."
    • changedInput schema / properties / source / description
      Previous value: -"Optional reference/source text for hallucination grounding check"New value: +"Reference text for the grounding check."
  3. Changed1 schema field changed
    • changedInput schema / properties / prompt / description
      Previous value: -"Optional original prompt (used for quality scoring and injection detection)"New value: +"Optional original prompt. Used for quality scoring AND scanned for prompt injection in its own right (checks.injection_prompt) — pass it whenever you have it, it is where the attack actually lands."
  4. Changed1 schema field changed
    • changedInput schema / properties / model / description
      Previous value: -"LLM model for AI-powered deep analysis (default: \"qwen/qwen3-32b\"). Set to \"none\" to skip LLM check. Supports any model from list_llm_models."New value: +"LLM model for AI-powered deep analysis (default: \"openai/gpt-oss-20b\"). Set to \"none\" to skip LLM check. Supports any model from list_llm_models."
  5. Changed1 schema field changed
    • addedOutput schema / properties
      Added value: +{
      +  "checks": {
      +    "type": "object"
      +  },
      +  "flags": {},
      +  "grade": {},
      +  "score": {},
      +  "verdict": {}
      +}
  6. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "additionalProperties": true,
      +  "type": "object"
      +}
  7. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing the verdict taxonomy (PASS/FIX/BLOCK), the 0-100 score, the per-check result payload, and the exact PII severity thresholds (SSN/card → BLOCK, email → FIX, phone/IP flagged only). It also surfaces the important data-egress fact that model 'none' sends nothing externally and that the Groq review uses a server key, which is precisely the context openWorldHint alone cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose and verdicts, then check inventory, then the PII gating rule — every clause carries information. It is a single dense paragraph with nested parentheticals (checks.injection / checks.injection_prompt) that would read better split into a short purpose line plus behavior notes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a multi-check composite tool, the description covers what is evaluated, what comes back (verdict, score, per-check results, fixes), the PII thresholds that drive the verdict, and the external-call escape hatch. With annotations and an output schema present, nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns an extra point by explaining the consequence of the 'model' parameter ('none' skips the review and sends nothing out) rather than just listing model names, and by reinforcing that prompt is itself scanned for injection. It does not clarify the rule object shape beyond deferring to guardrail_test.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('one-call safety gate for an LLM response') and enumerates the exact checks performed: grounding, injection on response and prompt, toxicity, PII, guardrail rules, quality, and LLM review. It is clearly the aggregate sibling of toxicity_scan, prompt_injection_scan, redact_pii and guardrail_test, and an agent can tell it apart without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete conditional guidance: pass the prompt 'whenever you have it', set model to 'none' to skip the external LLM review, and rules use the same format as guardrail_test. What is missing is an explicit statement that this replaces the individual scanners (toxicity_scan, prompt_injection_scan, redact_pii) and when to prefer those lighter tools instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.