Skip to main content
Glama

IA-QA — 152 QA & Dev Tools for AI Agents

bias_detect

Read-onlyIdempotent

Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone). Returns a bias score (0-100), sentiment analysis per variant, pairwise Jaccard similarity, and a human-readable verdict. No API key needed — runs entirely locally.

Input Schema

NameRequiredDescriptionDefault
responsesYesArray of variant responses to compare for bias

Output Schema

NameRequiredDescriptionDefault
verdictNo
lengthCVNo
biasScoreNo
sentimentsNo
avgSimilarityNo
minSimilarityNo
sentimentVarianceNo
pairwiseSimilaritiesNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedInput schema / examples
      Added value: +[
      +  {
      +    "responses": [
      +      "He is great.",
      +      "She is great."
      +    ]
      +  }
      +]
  2. Changed13 schema fields changed
    • changedOutput schema / properties / avgSimilarity / type
      Previous value: -"string"New value: +"number"
    • addedOutput schema / properties / biasScore / type
      Added value: +"number"
    • changedOutput schema / properties / lengthCV / type
      Previous value: -"string"New value: +"number"
    • changedOutput schema / properties / minSimilarity / type
      Previous value: -"string"New value: +"number"
    • removedOutput schema / properties / negative
      Removed value: -{}
    • addedOutput schema / properties / pairwiseSimilarities / items
      Added value: +{
      +  "properties": {
      +    "a": {
      +      "type": "string"
      +    },
      +    "b": {
      +      "type": "string"
      +    },
      +    "score": {
      +      "type": "number"
      +    }
      +  },
      +  "type": "object"
      +}
    • addedOutput schema / properties / pairwiseSimilarities / type
      Added value: +"array"
    • removedOutput schema / properties / positive
      Removed value: -{}
    • removedOutput schema / properties / ratio
      Removed value: -{}
    • changedOutput schema / properties / sentimentVariance / type
      Previous value: -"string"New value: +"number"
    • addedOutput schema / properties / sentiments / items
      Added value: +{
      +  "properties": {
      +    "negative": {
      +      "type": "number"
      +    },
      +    "positive": {
      +      "type": "number"
      +    },
      +    "ratio": {
      +      "type": "number"
      +    },
      +    "variantId": {
      +      "type": "string"
      +    }
      +  },
      +  "type": "object"
      +}
    • addedOutput schema / properties / sentiments / type
      Added value: +"array"
    • addedOutput schema / properties / verdict / type
      Added value: +"string"
  3. Changed1 schema field changed
    • addedOutput schema / properties
      Added value: +{
      +  "avgSimilarity": {
      +    "type": "string"
      +  },
      +  "biasScore": {},
      +  "lengthCV": {
      +    "type": "string"
      +  },
      +  "minSimilarity": {
      +    "type": "string"
      +  },
      +  "negative": {},
      +  "pairwiseSimilarities": {},
      +  "positive": {},
      +  "ratio": {},
      +  "sentimentVariance": {
      +    "type": "string"
      +  },
      +  "sentiments": {},
      +  "verdict": {}
      +}
  4. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "additionalProperties": true,
      +  "type": "object"
      +}
  5. First observed

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, so the safety profile is covered. The description still adds genuine context beyond them: 'No API key needed — runs entirely locally' tells the agent there is no network/auth dependency, and the enumerated outputs set expectations for result shape. It stops short of noting cost, runtime, or minimum-variant requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core operation, then the return payload, then the operational note. No filler and nothing buried at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be described, yet the description summarizes them anyway, and the local-execution note closes the auth question. The only gap is that it doesn't state how many variants are needed or how variantId values affect scoring.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is well documented, so the baseline would be 3. The description goes further by explaining how the array must be constructed — one entry per demographic variant drawn from the same prompt template — which is a semantic constraint the schema only partially encodes via minItems.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Analyse a set of LLM responses generated from the same prompt template but with different demographic variants') and enumerates exactly what it returns (bias score, sentiment, Jaccard similarity, verdict). It never distinguishes itself from nearby siblings like toxicity_scan or guardrail_test, which is the only thing keeping it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The precondition that responses must come from the same prompt template with differing demographic variants is implied strongly enough to guide correct input construction. However, there is no explicit statement of when to reach for this tool versus toxicity_scan or guardrail_test, and no exclusions or alternatives are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.