Skip to main content
Glama
┌──────────────────────────────────────────────────────────────────────────────┐
│                        WhitePact  v1.2.6                                     │
│                                                                              │
│  ┌──────────────┐  ┌─────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │ Governance   │  │ Trust Score │  │  Compliance  │  │  Guardrails      │  │
│  │ 5-way decide │  │ 6-dim A–F   │  │ NIST/EU/ISO  │  │  PII + Tox       │  │
│  └──────────────┘  └─────────────┘  └──────────────┘  └──────────────────┘  │
│  ┌──────────────┐  ┌─────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │ Hallucination│  │ Cost Intel  │  │   Red Team   │  │  Drift Monitor   │  │
│  │ Self-consist.│  │ Route+Budget│  │ 10 attacks   │  │  Alerts+Trend    │  │
│  └──────────────┘  └─────────────┘  └──────────────┘  └──────────────────┘  │
│  ┌──────────────┐  ┌─────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │ AI Passport  │  │  BiasBuster │  │ PrivacyLabel │  │  MCP Server      │  │
│  │ SHA-256 cert │  │ 6 probes+CI │  │  Federated   │  │  30 tools/HTTP   │  │
│  └──────────────┘  └─────────────┘  └──────────────┘  └──────────────────┘  │
│  ┌──────────────────────────────────────────────────────────────────────────┐ │
│  │   Governance Dashboard — FastAPI · Per-org rate limit · Alembic · OTEL  │ │
│  └──────────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘

What this solves

Every team deploying AI in production faces the same gap: no unified way to prove a model — or an autonomous agent's actions — is safe, fair, compliant, and accountable. Audits are manual, bias is discovered in production, compliance is a spreadsheet, an agent's tool calls go ungoverned, and nobody knows what the LLM bill will be next month.

WhitePact gives you one platform — a REST API, a Python SDK, an MCP server, and a live dashboard — that covers the full governance lifecycle:

Problem

Module

Output

Should this agent action be allowed, redacted, held for approval, denied, or quarantined?

WhitePactRuntimeGateway (governance core)

A five-way GovernanceDecision, deterministic, no LLM call in the decision path

Is this model trustworthy?

TrustScoreEngine

0–100 score, A–F grade, risk level

Does it comply with regulations?

ComplianceEngine

NIST AI RMF, EU AI Act tier, ISO 42001

Is it exposing PII?

GuardrailsEngine

Block / redact with audit log

Is it hallucinating?

HallucinationDetector

Risk score, unsupported claims

Can it be attacked?

RedTeamSimulator

10 vectors, CVE IDs, safe-refusal rate

How much is it costing?

CostTracker + ModelRouter

Per-model USD, routing to cheapest viable model

Is it getting worse over time?

TrustDriftMonitor

7/30-day trend, severity alerts

Is it biased?

BiasBuster

6 demographic probes, CI gate

Is this data labeled privately?

PrivacyLabel

Federated DP labels, never leaves device

Is this media real?

DeepfakeDetector

Ensemble confidence, method detected

Can I trust a third-party MCP server before connecting to it?

SupplyChainScanner

VERIFIED_FACT / INFERRED_SIGNAL / UNKNOWN verdicts — typosquat, description-content, known-incident checks

Is there a tamper-evident record of every governance decision?

EvidenceRepository

Hash-chained EvidenceRecord, per-org, verify_chain()

Does a risky action get a human in the loop?

ApprovalRepository

Race-safe PENDING → APPROVED/DENIED workflow

How does this model rank against others, independently?

Public Leaderboard

Cross-model trust ranking from actually calling each model's API, not self-reported

Can I cite and verify a trust score anywhere?

Trust Index

Free self-assessed or human-reviewed certified passport, verifiable at /verify/{id}, embeddable badge

Has this AI system failed publicly before?

AI Incident Database

Crowd-reported, moderator-reviewed, hash-chained public registry

Should my agent trust this third-party tool before calling it?

rai_check_trust + LangChain/LangGraph/ADK integrations

Free lookup, plus a real block/pause gate in-agent

Can any MCP client govern every AI call?

MCP Server

27 governance tools over stdio, Streamable HTTP, or legacy HTTP+SSE


Related MCP server: production-grade-mcp-agentic-system

Install

# Governance platform + REST API
pip install "rai-governance-platform[dashboard]"

# With PostgreSQL support
pip install "rai-governance-platform[dashboard,postgres]"

# With Redis + OpenTelemetry
pip install "rai-governance-platform[dashboard,redis,telemetry]"

# With LLM providers
pip install "rai-governance-platform[dashboard,openai,anthropic]"

# Everything
pip install "rai-governance-platform[all]"

The published PyPI package name (rai-governance-platform) and the import name (responsibleai) predate the WhitePact rename and are kept as-is — see MIGRATION_WHITEPACT_V2.md Section 3 for why an alias package (whitepact) was added instead of renaming the published package outright.


30-second quickstart

# Start the governance dashboard
pip install "rai-governance-platform[dashboard]"
uvicorn responsibleai.dashboard.app:app --port 8765

# Evaluate a model (no LLM key needed — supply your own scores)
curl -X POST http://localhost:8765/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "gpt-4o",
    "provider": "openai",
    "fairness": 0.80,
    "privacy": 0.85,
    "security": 0.82,
    "robustness": 0.78,
    "compliance": 0.90,
    "authenticity": 0.88
  }'
{
  "trust_score": { "trust_score": 83.65, "grade": "B", "risk": "LOW" },
  "compliance": { "overall_score": 80.5, "eu_ai_act_tier": "limited_risk", "violations": 0 },
  "passport_id": "rai-a3f7c2b1",
  "passport_hash": "4d8e1f2a9c3b7e6d...",
  "drift_alert": null
}

Open http://localhost:8765 for the live dashboard and http://localhost:8765/api/docs for interactive API docs.


Governance core — five-way decisions, not a binary block/allow

src/responsibleai/governance/ (see SPEC.md Sections 4-8 for the full architecture contract) is a deterministic runtime authority sitting in front of agent tool calls:

from responsibleai.governance import WhitePactRuntimeGateway, ActionRequest, AuthorityContext

gateway = WhitePactRuntimeGateway()
result = gateway.evaluate(
    action=ActionRequest(tool_name="rai_scan", arguments={"text": "..."}),
    authority=AuthorityContext(org_id="acme", agent_id="agent-1"),
)
print(result.decision)  # GovernanceDecision.ALLOW | ALLOW_WITH_REDACTION | REQUIRE_APPROVAL | DENY | QUARANTINE
  • Risk tiering (governance/risk.py) — every MCP tool is classified against a hardcoded, drift-tested table, not inferred at call time.

  • Policy engine (governance/policy.py) — first-match-wins rules with ALLOW / DENY / REQUIRE_APPROVAL effects.

  • Evidence (governance/evidence.py) — every decision is written to a per-org, hash-chained EvidenceRecord; verify_chain() detects tampering. Raw argument values are never stored, only field-name keys.

  • Approval workflow (governance/approval.py) — REQUIRE_APPROVAL decisions queue a real, race-safe ApprovalRequest with a resolution API, not just a log line.

  • Supply-chain scanner (src/responsibleai/supplychain/) — before an agent trusts a third-party MCP server or tool, SupplyChainScanner returns one of three explicit verdicts (VERIFIED_FACT / INFERRED_SIGNAL / UNKNOWN) — never a single opaque trust score — from typosquat detection, tool-description scanning, and known-incident cross-reference.

  • Identity Bridge (integrations/identity_bridge.py) — maps Entra ID, Google Workspace, Okta, and AWS (Cognito / IAM Identity Center) ID token claims into IdentityContext, plus map_groups_to_authority() to turn IdP group membership into a granted-action-types AuthorityContext. See MACHINE_AUTHORITY_V1.md's Identity Bridge section for exactly what's verified (claim-shape correctness against each provider's public docs) versus not (live-tenant testing, Graph/Admin-SDK group-name resolution, AWS's non-JWT SigV4 path).

No governance decision is LLM-based; see DETERMINISTIC_VS_PROBABILISTIC.md for why.

See it end-to-end: examples/08_whitepact_enterprise_scenario.py runs a full scenario (an org onboarding an autonomous finance agent) through all eight machine-authority invariants — ceiling, delegation, attenuation, approval quorum, workflow composition, autonomy budget, memory firewall, evidence bundle — against real code, no API keys required:

python examples/08_whitepact_enterprise_scenario.py

MCP Server — govern every AI call from Claude Code, Claude Desktop, or any MCP client

The MCP (Model Context Protocol) server exposes WhitePact as 30 tools and 20 resources (10 canonical resource URIs, dual-advertised under both whitepact:// and rai:// schemes — see MIGRATION_WHITEPACT_V2.md) to any MCP-compatible client — Claude Code, Claude Desktop, Cursor, Windsurf, or your own agent runtime. Three transports are supported: stdio, Streamable HTTP (/mcp, current MCP spec), and legacy HTTP+SSE (/sse + /messages/, kept for older clients). When a team's client points at this server, every AI interaction is automatically governed — five-way governance decisions, trust scoring, guardrails, compliance checks (NIST AI RMF / EU AI Act / ISO 42001), bias evaluation, drift detection, cost tracking, and hash-chained audit evidence run on any call without code changes.

Setup

# Install
pip install "rai-governance-platform[dashboard,mcp]"

# Start the REST API (MCP tools call it internally)
RAI_DB_PATH=/var/lib/rai/governance.db \
RAI_API_KEYS=your-key-here \
uvicorn responsibleai.dashboard.app:app --host 127.0.0.1 --port 8765 &

# Add to Claude Code (~/.claude/claude_desktop_config.json or via /mcp)
{
  "mcpServers": {
    "whitepact": {
      "command": "whitepact-mcp",
      "env": {
        "RAI_API_URL": "http://localhost:8765",
        "RAI_API_KEY": "your-key-here"
      }
    }
  }
}

whitepact-mcp and responsibleai-mcp are the same entry point — see pyproject.toml's [project.scripts]; both will keep working, use whichever name you prefer.

Available tools (27)

Tool

What it does

rai_scan

Detect and redact PII + harmful content before it reaches a log

rai_trust_score

Composite AI Trust Score (0-100) across 6 governance dimensions

rai_compliance

NIST AI RMF / EU AI Act / ISO 42001 compliance evaluation

rai_hallucination

Hallucination risk from hedging, consistency, unsupported claims

rai_cost_estimate

USD cost of a model API call from token counts

rai_redteam_payloads

Adversarial attack payloads (prompt injection, jailbreak, etc.)

rai_redteam_analyze

Security report from model responses to red team payloads

rai_compare_models

Compare two models across all 6 trust dimensions

rai_audit_summary

Governance capability summary (tools, frameworks, attack vectors)

rai_health

Status and module availability of the governance engine

rai_bias_evaluate

Demographic bias across 6 probe dimensions with confidence intervals

rai_drift_check

Trust score drift between a baseline and current evaluation

rai_passport_generate

Verifiable, tamper-evident AI Passport for vendor risk assessment

rai_budget_check

Spend vs. budget, per-team/model breakdown, month-end projection

rai_policy_check

Text/response against a governance policy (blocklists, disclaimers)

rai_stream_scan

PII/harm scan across streaming LLM output chunks

rai_benchmark

Score responses against truthfulqa / bbq / hellaswag suites

rai_benchmark_prompts

Question set for a benchmark suite

rai_model_route

Cheapest model that can handle a task, with cost/quality tradeoff

rai_pii_report

PII audit report by category with GDPR/CCPA remediation guidance

rai_incident_log

Structured governance incident record for audit/SIEM

rai_eu_ai_act_classify

EU AI Act risk tier classification with compliance roadmap

rai_iso42001_gap

ISO/IEC 42001:2023 AI Management System gap analysis

rai_executive_summary

Board-ready governance summary with RAG status indicators

rai_org_status

Governance status snapshot: models, grades, compliance, risk

rai_webhook_status

Webhook delivery health, failure analysis, remediation actions

rai_check_trust

Free public Trust Index lookup for a third-party model/tool, before an agent invokes it — unlike every other tool above, which evaluates output the caller itself produced

Agent-framework integrations — LangChain, LangGraph, Google ADK

src/responsibleai/integrations/ wires rai_check_trust directly into three agent frameworks so an agent can be gated on a tool's public trust score before invoking it, not just log the call after the fact:

  • LangChain (langchain_middleware.py) — TrustGateMiddleware, a wrap_tool_call middleware that blocks a call outright when its score is below threshold. Requires pip install "rai-governance-platform[langchain]".

  • LangGraph (langgraph_gate.py) — make_trust_gate_node(), a node that pauses the graph with interrupt() for a human approve/reject decision on a below-threshold call, instead of a hard block. Requires pip install "rai-governance-platform[langgraph]".

  • Google ADK (adk_toolset.py) — build_stdio_toolset() / build_http_toolset(), thin factories over ADK's McpToolset, which auto-discovers this project's MCP server's tools with no custom glue code. Requires pip install "rai-governance-platform[adk]".

All three, or any subset, install via pip install "rai-governance-platform[agent-frameworks]". See GAME_CHANGER_BUILD_PLAN.md Phase B for the reasoning behind each.

Available resources (20)

10 canonical resources, each advertised under both the whitepact:// and rai:// URI schemes (dual scheme is additive — see MIGRATION_WHITEPACT_V2.md; the table below shows the canonical URI):

Resource

URI

Contents

Health

whitepact://health

Current health status of the governance service

Model pricing catalog

whitepact://models/catalog

Supported models with per-token pricing

Compliance frameworks

whitepact://compliance/frameworks

NIST AI RMF, EU AI Act, ISO 42001

Red team categories

whitepact://redteam/categories

Adversarial attack categories

Trust dimensions

whitepact://trust/dimensions

The 6 dimensions behind the Trust Score

Bias probe catalog

whitepact://bias/probes

Available bias probes and scoring interpretation

Governance policy template

whitepact://governance/policy

Default policy template for rai_policy_check

Trust grade reference

whitepact://trust/grades

Grade thresholds, risk tiers, deployment guidance

NIST AI RMF checklist

whitepact://compliance/checklist/nist

Actionable NIST implementation checklist

EU AI Act checklist

whitepact://compliance/checklist/eu-ai-act

Compliance checklist for high-risk operators

MCP directory listings

WhitePact is listed and queryable today on real MCP directories — not aspirational, all verified live:

  • Official MCP Registryserver.json at the repository root (schema 2025-12-11, listing version 1.2.3) is published as io.github.Guruprasath-Annadurai/whitepact, confirmed queryable at registry.modelcontextprotocol.io. Advertises both the PyPI/stdio package (whitepact-mcp, self-hosted, free, unrestricted) and a remotes entry pointing at the hosted Streamable HTTP and SSE transports (whitepact-mcp-http.onrender.com) — a one-click remote connector, not just an installable package.

  • Antigravity CLI pluginplugins/whitepact/ at the repository root follows the official Antigravity plugin manifest format, connecting to the same hosted Streamable HTTP transport via serverUrl. No official Antigravity plugin directory exists yet, so this is distributed directly from the repo — see plugins/whitepact/README.md.

  • Smithery — listed as guruprasathannadurai-official/whitepact, 30 tools and 20 resources discovered against the hosted Streamable HTTP transport (whitepact-mcp-http.onrender.com/mcp, a separate Render service from the main dashboard). This deployment has no OAuth authorization server configured — only static Bearer API keys — so a public, unauthenticated /.well-known/mcp/server-card.json serves the same live TOOL_DEFS/RESOURCE_DEFS the server itself advertises, for directories whose scanners can't complete a live authenticated crawl.

See compliance/MCP_DISTRIBUTION_GUIDE.md for the full distribution plan, including directories not yet submitted to.

Platform integrations

WhitePact connects to the major AI platforms as one MCP server through standards-compliant clients — no per-platform forks, no per-platform governance logic. See docs/integrations/ for the canonical compatibility matrix (PLATFORM_COMPATIBILITY.md), per-platform setup docs (GitHub Copilot, Microsoft Copilot, Claude, Grok, Gemini, Amazon Q, AWS Bedrock AgentCore, Mistral Le Chat, Cursor), and FOUNDER_ACTIONS.md for what still needs a human. Run python scripts/integration_smoke.py for a live protocol-level preflight against the hosted endpoint.


Python SDK

Trust scoring

from responsibleai import TrustScoreEngine, PassportGenerator

engine = TrustScoreEngine()
score = engine.compute(
    fairness=0.80, privacy=0.85, security=0.82,
    robustness=0.78, compliance=0.90, authenticity=0.88,
)
print(f"{score.overall:.1f} / 100  Grade: {score.grade}  Risk: {score.risk_level}")
# → 83.7 / 100  Grade: B  Risk: LOW

passport = PassportGenerator().generate(
    model_name="gpt-4o", provider="openai", trust_score=score,
    compliance_summary={"overall": 80.5},
)
print(passport.passport_id)
passport.export_html("passport.html")

Guardrails — block PII before it reaches a log

from responsibleai import GuardrailsEngine

guardrails = GuardrailsEngine()
result = guardrails.scan("Customer SSN is 123-45-6789, email: alice@company.com")

print(result.is_blocked)      # True
print(result.pii_count)       # 2
print(result.redacted_text)   # "Customer SSN is [SSN], email: [EMAIL]"

Hallucination detection

from responsibleai import HallucinationDetector

detector = HallucinationDetector()
result = detector.analyze(
    "AI will replace all human jobs by 2025.",
    candidates=[
        "AI will automate some repetitive tasks.",
        "AI creates new job categories alongside displacing others.",
    ],
)
print(f"Risk: {result.hallucination_risk:.2f}  Level: {result.risk_level}")

Compliance — NIST AI RMF, EU AI Act, ISO 42001

from responsibleai import ComplianceEngine

engine = ComplianceEngine()
report = engine.evaluate(
    fairness_score=0.80, privacy_score=0.85,
    security_score=0.82, robustness_score=0.78,
    compliance_maturity=0.90, use_case="credit_scoring",
)
print(f"Score: {report.compliance_score * 100:.1f}%")
print(f"EU AI Act tier: {report.eu_ai_act_tier.value}")  # high_risk

Red team simulation

from responsibleai import RedTeamSimulator

simulator = RedTeamSimulator()
report = simulator.run_all()

print(f"Security score: {report.security_score:.1f}/100")
print(f"Vulnerabilities: {len(report.vulnerabilities)}")
for v in report.critical_vulnerabilities:
    print(f"  [{v['cwe_id']}] {v['name']}")

Cost intelligence

from responsibleai import CostTracker, ModelRouter, TokenUsage, BudgetPolicy

tracker = CostTracker(db_path="~/.responsibleai/data.db",
                      policy=BudgetPolicy(monthly_limit_usd=500.0))
usage = TokenUsage.create(
    provider="openai", model="gpt-4o",
    input_tokens=2000, output_tokens=800, team="product",
)
record = tracker.record(usage)
print(f"This call: ${record.total_cost:.4f}")
print(f"Month to date: ${tracker.total_cost(30):.2f}")

router = ModelRouter()
decision = router.route("Classify this email as spam or not spam", "balanced")
print(f"Recommended: {decision.recommended_model}  ${decision.estimated_cost_per_1k:.4f}/1k tokens")

Trust drift monitoring

from responsibleai import TrustScoreEngine, TrustDriftMonitor

monitor = TrustDriftMonitor(db_path=":memory:", alert_threshold=5.0)
engine = TrustScoreEngine()

for fairness in [0.90, 0.88, 0.85, 0.72]:
    score = engine.compute(fairness=fairness, privacy=0.85, security=0.80,
                           robustness=0.80, compliance=0.85, authenticity=0.85)
    alert = monitor.record("gpt-4o", "openai", score)
    if alert:
        print(f"Drift alert! {alert.severity}: {alert.delta:.1f} pt drop")

Governance Dashboard

A production FastAPI application with a dark-mode SPA. A live instance is hosted at whitepact.com.

# Development (auth off, SQLite in-memory)
RAI_AUTH_ENABLED=false uvicorn responsibleai.dashboard.app:app --port 8765

# Production (auth + persistent DB)
RAI_API_KEYS=your-key-here \
RAI_DB_PATH=/data/responsibleai.db \
uvicorn responsibleai.dashboard.app:app --host 0.0.0.0 --port 8765 --workers 4

# Docker
docker compose up -d

REST API endpoints

Method

Path

Description

GET

/api/health

Health — DB, auth, OTEL, version

GET

/api/metrics

Uptime, request count, error rate, monthly spend

POST

/api/evaluate

Full evaluation → trust + compliance + passport

GET

/api/trust-score/{model}/{provider}

Score history + drift trend

GET

/api/models

All evaluated models

POST

/api/scan

Guardrails — PII detection + redaction

POST

/api/hallucination

Hallucination risk analysis

POST

/api/cost/record

Record token usage

GET

/api/cost/summary

Cost breakdown by model / team / day

POST

/api/cost/analyze

Prompt efficiency — detect bloat

POST

/api/cost/route

Route task to cheapest viable model

GET

/api/cost/models

Full model pricing catalogue

GET

/api/drift/{model}/{provider}

Drift trend + history

GET

/api/audit

Paginated audit log (org-scoped)

GET

/api/audit/export

Export audit log as JSONL or CSV

GET

/api/audit/summary

Audit counts grouped by endpoint

GET

/api/redteam/payloads

Red team payload library (10 vectors)

POST

/api/redteam/analyze

Analyze model responses for vulnerabilities

GET

/api/billing/usage

Token spend and budget status

GET

/api/leaderboard

Public cross-model trust leaderboard (no auth)

GET

/api/leaderboard/{model}/{provider}/history

Trend over time for one model (no auth)

GET

/api/leaderboard/{model}/{provider}/diagnostic

Per-prompt findings — PRO plan required

POST

/api/trust-index/assess

Free, public self-assessment against the open Trust Index standard

GET

/api/trust-index/verify/{passport_id}

Verify a cited Trust Index score (no auth)

GET

/api/trust-index/check

Free, public — trust score + incident count for a named model/tool, by exact name (no auth); what rai_check_trust and the LangChain/LangGraph/ADK integrations call

GET

/api/trust-index/registry

Every assessed model/tool, certified and self-reported, newest first (no auth) — data source for the public /registry page

GET

/api/trust-index/certified

Directory of certified passports (no auth)

POST

/api/trust-index/certify/{passport_id}

Certify a passport — super-admin only

GET

/api/trust-index/badge/{passport_id}.svg

Embeddable trust badge (Self-Assessed / Certified), no auth

POST

/api/incident-db/report

Report a publicly observed AI incident (no auth, rate-limited)

GET

/api/incident-db

Browse published incidents — filter by model, provider, severity, type (no auth)

GET

/api/incident-db/check

Pre-deployment exact-match incident check for a model/provider — PRO/ENTERPRISE

GET

/api/incident-db/verify

Recompute the hash chain over every published entry (no auth)

POST

/api/orgs/{org_id}/keys/{key_id}/mfa/enroll

Enroll an API key in TOTP MFA

POST

/api/orgs/{org_id}/keys/{key_id}/mfa/verify

Verify a TOTP code / backup code

GET/POST

/api/governance/evidence

Read/write hash-chained governance evidence records

GET/POST

/api/governance/approvals

Queue and resolve REQUIRE_APPROVAL decisions

Interactive docs at /api/docs. Public leaderboard page at /leaderboard — see compliance/LEADERBOARD_METHODOLOGY.md for the published scoring methodology and scripts/run_leaderboard_eval.py to run evaluations. Open Trust Index standard and passport verification at /verify/{id} — see compliance/TRUST_INDEX_SPEC.md. Free, zero-signup self-assessment at /assess; browse every assessed model/tool at /registry. /llms.txt points AI crawlers/answer engines at these as canonical sources — see GAME_CHANGER_STRATEGY.md for why.

Production features

Feature

Detail

Authentication

Bearer token (RAI_API_KEYS) with RBAC (OWNER / ADMIN / ANALYST / VIEWER)

MFA

TOTP (RFC 6238) on the interactive login step, org-enforceable, single-use backup codes

Field-level encryption

Opt-in (RAI_FIELD_ENCRYPTION_KEY) on audit_log.ip_address, incident reporter contact info, webhook secrets, MFA secrets — with key-rotation support (MultiFernet)

Per-org rate limiting

Each Bearer token gets its own rate limit bucket (SHA-256 keyed) — no shared global pool

CORS

Configurable origins (RAI_ALLOWED_ORIGINS)

Security headers

CSP, X-Frame-Options, X-Content-Type-Options

Structured logging

JSON via structlog + request IDs

Database

SQLite (default) or PostgreSQL (RAI_DATABASE_URL) with Alembic migrations

Observability

OpenTelemetry traces + metrics (RAI_OTEL_ENDPOINT)

Webhooks

HMAC-signed delivery with DB-persisted retry queue (survives restarts)

Exception handling

No raw stack traces reach clients

Governance evidence

Hash-chained, per-org, tamper-evident (GET /api/governance/evidence)


Database migrations (Alembic)

Schema changes are managed with Alembic. Run alembic history for the current, authoritative migration count and table list — this number changes frequently enough that a hardcoded count here goes stale fast; the command itself is the source of truth.

# Upgrade to latest schema
RAI_DB_PATH=/var/lib/rai/governance.db alembic upgrade head

# PostgreSQL
RAI_DB_URL=postgresql://user:pass@host:5432/responsibleai alembic upgrade head

# Show migration history
alembic history

# Generate a new migration after changing engine.py
alembic revision --autogenerate -m "add_new_column"

All migrations use render_as_batch=True so they run on both SQLite and PostgreSQL without changes.


Webhook notifications

Register an endpoint and receive signed events when governance thresholds fire.

# Register a Slack webhook
curl -X POST http://localhost:8765/api/webhooks \
  -H "Authorization: Bearer your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ops-slack",
    "url": "https://hooks.slack.com/services/...",
    "events": ["drift_alert", "budget_exceeded", "guardrail_triggered"],
    "provider": "slack",
    "secret": "hmac-secret-for-signature-verification",
    "max_retries": 5
  }'

Deliveries are persisted to the database. If the server restarts during a retry cycle, the background worker picks up where it left off on next boot. Retry schedule: 1 s → 5 s → 30 s → 2 min → 10 min.

Verify payloads with the X-RAI-Signature-256: sha256=<hex> header.


Docker

git clone https://github.com/Guruprasath-Annadurai/Whitepact.git
cd Whitepact

python3 -c "import secrets; print(secrets.token_urlsafe(32))"

cp .env.example .env
# Edit .env — set RAI_API_KEYS

docker compose up -d
# Dashboard: http://localhost:8765
# API docs:  http://localhost:8765/api/docs

PostgreSQL + Redis (horizontal scaling)

# .env
RAI_DATABASE_URL=postgresql://rai:secret@db-host:5432/responsibleai
RAI_REDIS_URL=redis://redis-host:6379/0
RAI_OTEL_ENDPOINT=http://otel-collector:4318

pip install "rai-governance-platform[dashboard,postgres,redis,telemetry]"

# Run migrations before first start
RAI_DB_URL=postgresql://rai:secret@db-host:5432/responsibleai alembic upgrade head

The async database layer uses SQLAlchemy with connection pooling (pool_size=10, max_overflow=20, pool_pre_ping=True). Rate limiting switches to Redis-backed storage when RAI_REDIS_URL is set.


BiasBuster — bias evaluation in CI

# Fail CI when demographic bias exceeds threshold
biasbuster run \
  --provider openai --model gpt-4o \
  --probes gender-bias,racial-bias,cultural-bias \
  --threshold 0.20 \
  --output report --format html
from biasbuster import BiasBusterRunner, GenderBiasProbe, RacialBiasProbe
from biasbuster.providers import OpenAIProvider
import asyncio

async def main():
    provider = OpenAIProvider(api_key="sk-...", model="gpt-4o")
    runner = BiasBusterRunner(provider=provider)
    suite = await runner.run([
        GenderBiasProbe(threshold=0.20),
        RacialBiasProbe(threshold=0.20),
    ])
    print(f"Score: {suite.overall_score:.4f}  {'PASSED' if suite.passed else 'FAILED'}")

asyncio.run(main())

Available probes: gender-bias, racial-bias, age-bias, religious-bias, occupational-stereotype, cultural-bias

Scoring: TF-IDF cosine divergence + length asymmetry + VADER sentiment divergence, 95% bootstrap confidence intervals, intersectional co-failure amplification (×1.15).


PrivacyLabel — on-device federated labeling

from privacylabel import FederatedClient, FedAvgAggregator

client = FederatedClient(
    node_id="hospital-node-01",
    provider=MyProvider(),
    epsilon_per_round=0.1,
    total_epsilon=1.0,
    delta=1e-6,
    gradient_clip=1.0,
)
# Raw data stays on disk — only privatised gradients leave the device
summary = await client.train_round("data/local_records.jsonl")
print(f"Privacy budget used: ε={summary.privacy_spent['spent_epsilon']:.3f}")

Implements Laplace, Gaussian, Exponential, and DP-SGD mechanisms. Byzantine-robust aggregation via Weiszfeld geometric median.


GitHub Actions — bias gate in CI

- name: Bias evaluation
  run: |
    pip install "rai-governance-platform[openai]"
    biasbuster run \
      --provider openai --model gpt-4o-mini \
      --probes gender-bias,racial-bias,cultural-bias \
      --threshold 0.20
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Environment variables

Variable

Default

Description

RAI_DB_PATH

governance.db

SQLite path

RAI_DB_URL

(unset = SQLite)

Full SQLAlchemy URL — takes priority over RAI_DB_PATH

RAI_DATABASE_URL

(unset)

Alias for RAI_DB_URL

RAI_API_KEYS

(empty = auth off)

Comma-separated bearer tokens

RAI_AUTH_ENABLED

true

Toggle auth enforcement

RAI_REDIS_URL

(unset = in-memory)

Redis URL for distributed rate limiting

RAI_RATE_LIMIT_DEFAULT

100/minute

Per-org rate limit (keyed by Bearer token)

RAI_OTEL_ENDPOINT

(unset = disabled)

OTLP HTTP endpoint

RAI_OTEL_SERVICE_NAME

responsibleai

Service name for traces

RAI_ALERT_THRESHOLD

5.0

Trust score drop that triggers drift alert

RAI_MONTHLY_BUDGET_USD

10000.0

Monthly AI spend limit

RAI_LOG_LEVEL

INFO

Log level

RAI_LOG_JSON

true

Structured JSON logs

RAI_HOST

127.0.0.1

Bind address

RAI_PORT

8765

Port

Dual-prefixed WHITEPACT_* equivalents for these are also read where MIGRATION_WHITEPACT_V2.md documents them — the RAI_* names remain the primary, always-supported form.


Development

git clone https://github.com/Guruprasath-Annadurai/Whitepact.git
cd Whitepact

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# Full test suite (run it to see the current test count and coverage —
# see CONTRIBUTING.md's Running Tests section for why no number is
# hardcoded here)
pytest

# Dashboard tests only
RAI_DB_PATH=:memory: RAI_AUTH_ENABLED=false pytest tests/test_dashboard_api.py

# Webhook persistence tests
pytest tests/test_webhook_persistence.py

# MCP server tests
pytest tests/test_mcp_server.py

# Lint + type check
ruff check src/ tests/
mypy src/responsibleai src/biasbuster

Roadmap

See ROADMAP.md for the canonical NOW/NEXT/LATER plan. The list below is a historical, version-by-version changelog summary kept for reference.

  • v0.1 — BiasBuster: gender probe, 4 providers, CLI, CI integration

  • v0.2 — Racial / age / religious / occupational probes, HTML reporter, PrivacyLabel federated DP

  • v0.3 — Cultural bias, intersectional analysis, DeepfakeDetector ensemble

  • v0.4 — Cost Intelligence (CostTracker, ModelRouter, 16-model pricing), Trust Drift Monitor

  • v0.5 — Governance Dashboard (FastAPI), Trust Score, AI Passport, Guardrails, Hallucination, Compliance, Red Team, CI/CD, Docker, SLA

  • v0.6 — Async PostgreSQL (SQLAlchemy), Redis rate limiting, OpenTelemetry APM, LLM integration tests

  • v1.0 — WebSocket drift alerts, Prometheus endpoint, multi-tenant RBAC, org management API

  • v1.1 — MCP server (10 tools, 5 resources), audit log API, red team API, billing API, Alembic migrations, per-org rate limiting, DB-persisted webhook retry queue

  • v1.2 — Public Leaderboard, Trust Index/Passports + embeddable badges, AI Incident Database, TOTP MFA, expanded field encryption, DB-persisted webhooks, full dashboard UI rebuild, white-label branding, a genuinely live hosted instance — see CHANGELOG.md for the full list

  • WhitePact migration (1.2.01.2.2) — governance decision core, MCP Streamable HTTP + OAuth/OIDC, risk tiering + policy engine, hash-chained evidence, approval workflow, multi-approver quorum + delegation chains, upstream MCP tool discovery, MCP trust/supply-chain scanner, HA Helm deployment, supply chain security (SBOM/provenance), release engineering, open source governance, live listings on the official MCP Registry and Smithery — see MIGRATION_WHITEPACT_V2.md for the full phase-by-phase log and what's still not done

  • v2.0 onward — see VERSION_ROADMAP.md for the phase-by-phase plan through v6.0

  • Strategic directionGAME_CHANGER_STRATEGY.md lays out an infrastructure-first bet (free public trust registry, an agent-native trust-check primitive, AI-answer-engine citability) as an alternative to the enterprise-SaaS path, with GAME_CHANGER_BUILD_PLAN.md breaking it into concrete engineering phases against the current codebase


Security & Open Source Assurance

The official OpenSSF/OSPS BadgeApp project currently records OpenSSF Best Practices Silver and OSPS Baseline Level 1. They are voluntary project evidence, not an independent audit, penetration test, SOC 2, or ISO certification. Current technical and claim boundaries are maintained in WHITEPACT_TRUST_STATUS.md and PUBLIC_TRUST_CLAIMS.md.

Release consumers can review the signed-tag evidence, release process, security policy, SLSA evidence boundary, and consumer verification guide. The reusable trusted-builder pipeline is present on main. Release v1.2.6 completed that path: its wheel and sdist were reproduced, hashed, attested, independently verified in the publish job, published to PyPI without rebuilding, hash-matched to PyPI, and attached to the GitHub Release with the CycloneDX SBOM. Independent consumer verification was repeated on 2026-08-31. The release-specific evidence is assessed as satisfying SLSA v1.2 Build L3; SLSA is a conformance framework, not a certification or a guarantee that an artifact is secure.


Further reading


License

MIT — see LICENSE.

Available Tools

30 tools
rai_audit_summaryGet Governance Capability SummaryA
Read-onlyIdempotent

Return a governance capability summary including supported tools, frameworks, and available attack vectors. Full audit log access requires the REST endpoint.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNo

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds real value beyond that by disclosing a scope limitation: the summary is not the full audit log and that requires the REST endpoint. It still omits whether the result is scoped by the 'days' window or what is excluded from the summary.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the core purpose front-loaded and the limitation placed second. Every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read tool with no output schema the description is largely adequate, and the REST-endpoint caveat usefully signals partial coverage. The gap is that the only parameter's meaning and effect are unexplained, which is exactly what an agent needs to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single 'days' parameter, so the description carries the full explanatory burden and does not meet it. It never mentions the parameter, the default of 7, or whether days narrows the audit window or the capability list, leaving an agent to guess from the name alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Return a governance capability summary') and enumerates the content returned: supported tools, frameworks, and attack vectors. It is not, however, differentiated from near-siblings like rai_executive_summary or rai_org_status, which an agent could plausibly choose instead.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The sentence about full audit log access requiring the REST endpoint is a useful boundary (when this tool is not sufficient), but the description never says when to prefer this summary over rai_executive_summary or rai_org_status, nor names any alternative. Usage is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_benchmarkEvaluate Responses Against BenchmarkA
Read-onlyIdempotent

Evaluate pre-collected model responses against a standard benchmark suite. Suites: truthfulqa (factual accuracy), bbq (bias in questions), hellaswag (reasoning). Call rai_benchmark_prompts first to get the question set, collect responses, then pass them here.

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteNotruthfulqa
providerYes
responsesYesMap of sample_id → model response text
model_nameYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so safety is covered. The description adds the non-obvious behavioral requirement that responses must be collected externally before invocation, which is real context beyond the annotations. Return-format behavior is left unstated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action, then suite meanings, then the workflow. No filler and every clause carries usable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description cannot lean on structure to explain results, and it does not describe what an evaluation returns (scores, per-suite metrics). However, the suite semantics and the end-to-end workflow make the call path unambiguous for a nested-input evaluation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25%, so the description must compensate. It partially does by glossing each enum suite value (truthfulqa=factual accuracy, bbq=bias, hellaswag=reasoning), but 'provider' and 'model_name' get no explanation. The responses map semantics live only in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Evaluate pre-collected model responses against a standard benchmark suite') and enumerates the three supported suites with a one-word meaning for each. It also distinguishes itself from the sibling rai_benchmark_prompts by role in the workflow (prompts generate questions, this one scores answers).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit ordering: 'Call rai_benchmark_prompts first to get the question set, collect responses, then pass them here.' That names the prerequisite tool and the condition under which this tool is appropriate. It stops short of saying when NOT to use it or which suite to pick for a given goal.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_benchmark_promptsGet Benchmark Question SetA
Read-onlyIdempotent

Return the question set for a benchmark suite. Use to collect model responses before calling rai_benchmark. Suites: truthfulqa, bbq, hellaswag.

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteNotruthfulqa

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world behavior, so the safety profile is covered. The description adds the pipeline position (call before rai_benchmark), which is genuinely useful workflow context, but says nothing about the size or shape of the returned set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, nothing wasted, with the core action front-loaded and the alternative/sequencing information following immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-enum getter with no output schema, the description covers purpose, workflow placement, and valid suites. It does not describe the return payload (e.g., question format or count), which is the only remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter exists and schema description coverage is 0%, but the parameter is an enum whose values are self-explanatory. The description restates the enum values ('truthfulqa, bbq, hellaswag') but does not explain the default or what each suite measures, so it adds marginal value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Return the question set for a benchmark suite') and explicitly positions itself relative to the sibling rai_benchmark, so an agent can distinguish gathering prompts from running the suite. No ambiguity about what is returned.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use it to collect model responses before calling rai_benchmark, giving clear workflow context and sequencing. It lacks explicit exclusions (e.g., when not to use it), which keeps it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_bias_evaluateEvaluate Demographic BiasA
Read-onlyIdempotent

Evaluate demographic bias across six probe dimensions: gender, racial, age, religious, occupational, and cultural. Provide paired response samples for each demographic group. Returns per-probe bias scores (0=no bias, 1=maximum divergence), confidence intervals, intersectional amplification, and an overall bias grade.

ParametersJSON Schema
NameRequiredDescriptionDefault
providerYesModel provider
thresholdNoBias score above this value triggers a FAIL
model_nameYesModel under evaluation
probe_responsesYesMap of probe_name → list of response texts from different demographic groups. Each list must have at least 2 responses to compute divergence.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), so the bar is lower. The description adds meaningful behavioral detail by disclosing the output structure (per-probe scores with a 0-1 scale, confidence intervals, intersectional amplification, overall grade), which matters since no output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the verb and the six dimensions, then the input expectation, then the outputs. No wasted phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite using a nested input object, the definition covers purpose, required inputs, and return values (compensating for the absent output schema), and annotations carry the safety profile. Only the lack of explicit sibling routing keeps it short of complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters. The description reinforces the probe_responses concept with 'paired response samples,' but adds little syntax or format beyond what the schema provides; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Evaluate) and resource (demographic bias) and enumerates the six probe dimensions it covers (gender, racial, age, religious, occupational, cultural). This is distinctive enough among the rai_ siblings that an agent can identify it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The instruction to 'provide paired response samples for each demographic group' implies how to prepare input, but there is no explicit when-to-use guidance, no when-not-to, and no routing to alternatives like rai_redteam_analyze or rai_scan. Usage is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_budget_checkCheck AI Spending Against BudgetB
Read-onlyIdempotent

Evaluate current AI spending against monthly budget limits. Returns consumption percentage, alert status, per-team and per-model breakdown, and projected month-end spend. Used by LLMOps Engineers and Finance to prevent budget overruns.

ParametersJSON Schema
NameRequiredDescriptionDefault
days_elapsedNo
days_in_monthNo
team_breakdownNoteam_name → USD spent
model_breakdownNomodel_name → USD spent
total_spent_usdYes
monthly_limit_usdNo
alert_threshold_pctNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior, so the bar is lower. The description usefully discloses the computation outputs (consumption %, alert status, projections), which matters because there is no output schema, but it never clarifies that the caller must supply the raw spend figures rather than the tool fetching them—a meaningful behavioral gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the core action and then the return values. The audience clause is the only slightly expendable part, but nothing is padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description's enumeration of returned metrics is genuinely necessary and present. However, the input-side story is incomplete: seven parameters, undocumented semantics for alert_threshold_pct and the day-count pair, and no note about required spend inputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 29% and the description explains none of the seven parameters. In particular, alert_threshold_pct (default 0.8, max 1) and monthly_limit_usd are ambiguous—fraction vs. percentage, default limit source—and the description does nothing to compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Evaluate current AI spending against monthly budget limits', and enumerates the computed outputs. It is clear what the tool does, though it does not distinguish itself from nearby siblings like rai_cost_estimate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Used by LLMOps Engineers and Finance to prevent budget overruns' implies a monitoring use case, but there is no explicit when-to-use, no prerequisites, and no named alternative (e.g. rai_cost_estimate) for estimating rather than checking spend.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_causal_influence_checkCheck Causal Influence Provenance for Injection PatternsA
Read-onlyIdempotent

Scan a list of upstream sources that causally shaped a proposed action (a prior tool's output, a sub-agent's result, an external document, ...) for prompt-injection patterns, and flag whether any of them are untrusted. Generalizes rai_memory_write_check beyond persistent memory: any content that will be treated as trusted context by whatever consumes this action's result carries the same replay risk memory does. Each provenance entry needs a 'kind' (memory_read | tool_output | sub_agent_result | user_input | external_content) and a 'trust' level (TRUSTED | UNTRUSTED | UNKNOWN); 'content' is optional (an entry may assert only its trust level with nothing to scan). Call this before letting a matched/untrusted source influence a real action -- when governance is enabled on the hosted MCP server, the same check also runs automatically on any governed action whose arguments carry a '_provenance' key in this same shape.

ParametersJSON Schema
NameRequiredDescriptionDefault
provenanceYesUpstream sources that shaped the action being considered.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), yet the description adds real behavioral context: what the check does, that it can run automatically under hosted governance, that content is optional so an entry may assert trust with nothing to scan, and why the risk applies to any trusted-context consumer. This goes well beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense paragraph, but it is front-loaded with the purpose before the generalization, parameter detail, and invocation timing. Every clause carries information; the only minor cost is run-on length rather than waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with annotations covering safety and no output schema, the description supplies everything needed: what it scans, the entry shape, when to call it, and the automatic governance path. Nothing material is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds meaning: it explains the required 'kind' and 'trust' fields, enumerates their values, and clarifies that 'content' is optional with the rationale (an entry may carry only a trust assertion). That added interpretation exceeds the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Scan') and resource (a list of upstream sources that causally shaped a proposed action) and names the outcome ('flag whether any of them are untrusted'). It explicitly positions itself against sibling rai_memory_write_check by generalizing beyond persistent memory, so an agent can distinguish it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear trigger: 'Call this before letting a matched/untrusted source influence a real action.' It also notes the alternative execution path (automatic check under governance when arguments carry a '_provenance' key). There is no explicit when-not-to-use guidance, keeping it just below full marks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_check_trustCheck Public Trust Index ScoreA
Read-onlyIdempotent

Check the public, independently-verifiable Trust Index score, certification status, and reported-incident history for a named AI model or tool BEFORE invoking it. Unlike every other rai_* tool, which evaluates output the caller itself produced, this one looks up a public record about a THIRD PARTY'S model or tool — built for agents and agent frameworks (LangChain, LangGraph, Google ADK) deciding whether to trust something before calling it. Free, no auth required, exact model+provider match. Queries the hosted ResponsibleAI Trust Index (configurable via the RAI_TRUST_API_BASE environment variable). Returns 'known: false' for anything never assessed — that is not an error, just an absence of data; self-assessment is free at POST /api/trust-index/assess.

ParametersJSON Schema
NameRequiredDescriptionDefault
providerYesExact provider name, e.g. 'openai'
min_scoreNoMinimum acceptable overall trust score (0-100). The response's 'passes' field reflects this threshold.
model_nameYesExact model or tool name, e.g. 'gpt-4o'

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safety profile (readOnly, idempotent, non-destructive, closed-world), and the description adds substantial context beyond them: it is free, requires no auth, uses exact model+provider matching, queries a configurable hosted base (RAI_TRUST_API_BASE), and returns 'known: false' for unassessed items as a non-error absence of data. It even points to the self-assessment endpoint. This is rich, agent-relevant behavioral detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the key sibling distinction are front-loaded, and subsequent sentences carry distinct value (audience, auth/free status, edge-case 'known: false'). It is on the longer side and slightly repetitive, but nearly every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does so adequately: it names the returned items (score, certification status, incident history) and the 'known: false' behavior. It stops short of describing the exact response shape or the 'passes' field mechanics, but is sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents model_name, provider, and min_score (including that 'passes' reflects the threshold). The description reinforces the 'exact model+provider match' requirement, which adds slight emphasis but no new semantics beyond the schema's 'Exact provider/model name' text. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (checks the public Trust Index score, certification status, and incident history) for a named model/tool. It explicitly differentiates itself from siblings: 'Unlike every other rai_* tool, which evaluates output the caller itself produced, this one looks up a public record about a THIRD PARTY'S model or tool.' An agent can distinguish it from rai_trust_score, rai_compliance, etc. without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly frames when to use it ('BEFORE invoking it') and for whom (agents/agent frameworks deciding whether to trust something before calling it). The contrast with all other rai_* tools implicitly signals when NOT to use it, but it does not name a specific alternative tool for the 'assess your own output' case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_compare_modelsCompare Two Models' Trust ScoresB
Read-onlyIdempotent

Compare two AI models across all six trust dimensions. Returns scores for each, delta analysis, and a recommendation on which model is more trustworthy.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_aYes
model_bYes
scores_aNo
scores_bNo
provider_aYes
provider_bYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description usefully supplements by disclosing the return shape (per-model scores, delta analysis, recommendation), which matters since there is no output schema. It says nothing about auth requirements or where the scores come from when scores_a/scores_b aren't supplied.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tidy sentences, front-loaded with the operation and followed by the return content. No filler, though it could have spent one of those sentences on parameter guidance instead of repeating the count of dimensions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description's summary of returns is genuinely necessary and present. However, for a six-parameter tool with nested objects, 0% schema coverage, and four required inputs, the omission of any input semantics leaves the definition only partially complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across six parameters, so the description carries the full burden and falls short. It hints at 'six trust dimensions' but never clarifies the distinction between the required model_a/provider_a/model_b/provider_b and the optional nested scores_a/scores_b objects, nor what happens if scores are omitted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource (compare two AI models) and scopes it to 'all six trust dimensions', which is concrete. It does not explicitly distinguish itself from the single-model sibling rai_trust_score, but the two-model framing is self-evident from the name and title.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not-to-use guidance is given, and no alternative is named despite several closely related siblings (rai_trust_score, rai_benchmark, rai_model_route). The comparison purpose is implied by 'compare two models' but the agent is left to infer when this is the right tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_complianceEvaluate Governance ComplianceA
Read-onlyIdempotent

Evaluate AI governance maturity against NIST AI RMF, EU AI Act, or ISO 42001, given your own self-assessed control scores (fairness/privacy/security/robustness 0-1, plus overall compliance maturity 0-1). Returns a compliance score, findings per control, and remediation recommendations. Use this when the caller already has maturity/control scores and wants a gap assessment. Do NOT use this to classify what EU AI Act risk tier a specific system falls into from a description of what it does (sector, automation, biometric use, etc.) -- use rai_eu_ai_act_classify instead for that; this tool has no equivalent inputs (no deployment sector, no system description) and cannot answer that question.

ParametersJSON Schema
NameRequiredDescriptionDefault
use_caseNogeneral
frameworkNoNIST_AI_RMF
privacy_scoreNo
fairness_scoreNo
security_scoreNo
robustness_scoreNo
compliance_maturityNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered. The description adds genuinely new behavioral context beyond them: it names the return payload (compliance score, per-control findings, remediation recommendations) and the precondition that the caller must supply self-assessed scores. No output schema exists, so this disclosure is valuable. It stops short of mentioning scoring determinism or how defaults (all 0.5) are treated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose and return values, then the routing rule and exclusion. The negative-guidance sentence is long, but every clause earns its place by fending off a specific misrouting. Slightly verbose overall but no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only scoring tool with no output schema, the description supplies the returns, the input precondition, and the anti-pattern that would cause misuse. The remaining gap is the undocumented 'use_case' parameter and default behavior, which is the only thing an agent could still get wrong.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 7 parameters, so the description must carry the burden. It does explain the five numeric control scores and their 0-1 range, and implies the framework values, but leaves 'use_case' entirely undefined and does not clarify that all scores default to 0.5 or what the framework choice changes in the output. Partial compensation only.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Evaluate AI governance maturity') and names the exact frameworks it scores against (NIST AI RMF, EU AI Act, ISO 42001). It explicitly differentiates itself from the closest sibling, rai_eu_ai_act_classify, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides both the positive trigger ('when the caller already has maturity/control scores and wants a gap assessment') and an explicit exclusion with the alternative named ('Do NOT use this to classify what EU AI Act risk tier... use rai_eu_ai_act_classify instead'). It even explains why the alternative is needed (no deployment sector, no system description inputs).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_cost_estimateEstimate Model API CostC
Read-onlyIdempotent

Estimate the USD cost of a model API call from token counts.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesModel name, e.g. gpt-4o
providerYesProvider: openai | anthropic | google | mistral
input_tokensYes
output_tokensYes

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds only that the output is expressed in USD and derived from token counts; it says nothing about pricing-data staleness, whether it is a pure computation, or what form the estimate takes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler, which is good, but it is under-specified rather than genuinely concise — there is simply not enough content to be wasteful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Four required parameters, 50% schema coverage, and no output schema. The description never says what the tool returns (a bare number, a currency-formatted string, a breakdown per token type), which for a cost-estimation tool an agent genuinely needs in order to use the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 50% — input_tokens and output_tokens have no schema description at all. The phrase 'from token counts' weakly gestures at them but adds no units, granularity, or counting convention, so the description does not compensate for the coverage hole.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (estimate) plus resource (USD cost of a model API call) and the inputs it derives them from (token counts). It is clear on its own, but it does not differentiate itself from siblings like rai_compare_models or rai_budget_check, which touch adjacent cost/model concerns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description offers no when-to-use guidance, no alternatives, and no prerequisites beyond the implicit 'you must have token counts'. With ~30 sibling tools in this family, the absence of any routing hint is a real gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_drift_checkCheck Trust Score DriftA
Read-onlyIdempotent

Detect trust score drift between a baseline evaluation and a current evaluation. Returns drift delta per dimension, overall drift severity (NONE/LOW/MEDIUM/HIGH/CRITICAL), and whether an alert threshold was breached.

ParametersJSON Schema
NameRequiredDescriptionDefault
providerYes
model_nameYes
current_scoreYesCurrent trust dimension scores (0-1 each)
baseline_scoreYesPrevious trust dimension scores (0-1 each)
alert_thresholdNoOverall score drop (0-100 scale) that triggers an alert

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already disclose read-only, idempotent, non-destructive behavior. The description adds meaningful behavioral detail beyond that: it returns per-dimension drift deltas, a five-level severity value (NONE/LOW/MEDIUM/HIGH/CRITICAL), and an alert-threshold breach indicator. It does not cover auth or rate limits, but those are minor for this read-only analysis tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first states the purpose, the second states the return values. It is front-loaded, has no filler, and every clause contributes useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only tool with no output schema, the description explains the core purpose and the main return fields, including severity levels and threshold breach detection. It remains silent on when to choose it over sibling trust tools and on the two parameters lacking schema descriptions, which keeps it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 60%: baseline_score, current_score, and alert_threshold have schema descriptions, while provider and model_name do not. The description reinforces the baseline/current comparison and alert-threshold concept but adds no semantics for the two undocumented parameters. A 3 is appropriate given the moderate schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb (detect) and resource (trust score drift), and scopes it to a baseline evaluation versus a current evaluation. It also names the return outputs, so the purpose is clear. It does not, however, explicitly differentiate itself from sibling tools such as rai_check_trust or rai_trust_score, which keeps it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The baseline/current framing implies when the tool is useful, namely when comparing two trust-score evaluations. There is no explicit when-to-use, when-not-to-use, or named alternative, so an agent must infer that this is preferable to rai_trust_score or rai_check_trust for drift analysis.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_eu_ai_act_classifyClassify EU AI Act Risk TierA
Read-onlyIdempotent

Classify an AI system into an EU AI Act risk tier: UNACCEPTABLE, HIGH, LIMITED, or MINIMAL, from a description of what the system does (deployment sector, automation level, biometric/emotion-recognition use). Evaluates deployment context, capabilities, and affected populations against Annex III and Annex VI criteria. Returns risk tier, applicable articles, required conformity assessment actions, and a compliance roadmap. Use this for 'what EU AI Act category does this system fall into' questions. Do NOT use this for a general maturity/gap assessment against self-scored controls (fairness/privacy/security scores) -- use rai_compliance with framework=EU_AI_ACT instead for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
deployment_sectorYes
is_fully_automatedNo
system_descriptionYesDescription of the AI system and its purpose
trust_score_overallNo
social_scoring_purposeNo
affects_natural_personsNo
processes_biometric_dataNo
real_time_remote_biometricNo
used_for_emotion_recognitionNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive and closed-world, so the safety profile is covered. The description adds meaningful behavior beyond that: it discloses the evaluation basis (Annex III/VI, deployment context, capabilities, affected populations) and the return payload (tier, applicable articles, conformity assessment actions, compliance roadmap).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and outputs, then usage guidance and an explicit exclusion. Every sentence carries distinct information (purpose, criteria, returns, routing), with no redundant restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a classification tool with no output schema, the description usefully summarizes what is returned (tier, articles, conformity actions, roadmap) and the evaluation criteria. It is nearly complete, with the only gap being coverage of the several undocumented input parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 11% (one documented property of nine), so the description must compensate. It hints at deployment_sector, is_fully_automated, and the biometric/emotion flags, but omits others such as trust_score_overall, social_scoring_purpose, and affects_natural_persons, leaving significant parameter meaning unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (classify) and resource (AI system into an EU AI Act risk tier), enumerates the four possible outcomes, and specifies the input basis (deployment sector, automation level, biometric/emotion-recognition use). It also names the criteria used (Annex III and Annex VI), so an agent knows exactly what this tool produces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes: 'Use this for what EU AI Act category does this system fall into questions' and 'Do NOT use this for a general maturity/gap assessment... use `rai_compliance` with framework=EU_AI_ACT instead.' This gives both the trigger condition and the named alternative for the excluded case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_executive_summaryGenerate Executive Governance SummaryA
Read-onlyIdempotent

Generate a board-ready executive AI governance summary. Synthesises trust grades, compliance posture, cost intelligence, risk incidents, and drift trends into a C-suite-readable report with RAG (Red/Amber/Green) status indicators. Used by CAIO for quarterly board reporting.

ParametersJSON Schema
NameRequiredDescriptionDefault
org_nameNoOrganisation
top_risksNoTop identified AI risks for executive attention
frameworksNoActive compliance frameworks
drift_alertsNo
bias_failuresNo
report_periodNoQ2 2026
open_incidentsNo
total_cost_usdNo
avg_trust_scoreNo
compliance_scoreNo
models_evaluatedNo
monthly_budget_usdNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is covered. The description adds real value beyond them: it reveals this is a multi-source aggregation/synthesis step rather than a single-source read, and it describes the output shape (RAG status indicators), which matters since no output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with the core action front-loaded and no filler. Minor redundancy between 'board-ready' and 'C-suite-readable', but otherwise every sentence carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter, no-required-argument tool with no output schema, the description does explain the report contents, which partially substitutes for a return-value spec. It leaves the biggest gap unaddressed: whether the caller must supply the metrics or the tool derives them, and what the 10 undocumented numeric parameters mean.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 17% (2 of 12 parameters documented), so the description is expected to compensate — and it does not mention a single parameter. Names like avg_trust_score, compliance_score, monthly_budget_usd and models_evaluated are left entirely to inference, with no indication of units, ownership, or whether they are pulled automatically versus supplied by the caller.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Generate), a specific resource (executive AI governance summary), and enumerates the inputs it synthesises (trust grades, compliance posture, cost intelligence, risk incidents, drift trends). The C-suite/board framing implicitly separates it from technical siblings like rai_audit_summary or rai_org_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear situational context — 'Used by CAIO for quarterly board reporting' — which tells the agent the audience and cadence. It does not, however, name an alternative sibling or state when NOT to use it (e.g. ad-hoc technical reporting via rai_audit_summary).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_hallucinationDetect Hallucination RiskA
Read-onlyIdempotent

Detect hallucination risk in AI-generated text. Analyses hedging language, self-consistency across candidate responses, unsupported factual claims, and (when a source is supplied) explicit factual disagreement with that source -- e.g. the source names one day/month/number and the response names another.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesAI-generated text to analyse
sourceNoOptional ground-truth or reference text the response should be consistent with -- enables explicit factual-disagreement detection.
candidatesNoOptional additional responses for consistency scoring

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint, idempotentHint, destructiveHint=false), so the bar is lower, and the description adds real behavioral detail: the four analysis dimensions and the conditional behaviour that factual-disagreement detection only activates when `source` is present. It stops short of describing output shape, confidence levels or scoring thresholds, so it is not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence leads with the core purpose before the enumeration, and the parenthetical example is the only elaboration, placed where it disambiguates `source`. Nothing is redundant with the title or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter analysis tool with no output schema, the definition explains inputs well but says nothing about what comes back -- risk score, label, per-signal breakdown -- and gives no hint about candidate count expectations or latency/cost. Annotations cover safety, so the gap is in return-value and operational context rather than in purpose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description goes beyond it by giving a concrete semantics example for `source` ('the source names one day/month/number and the response names another') and by tying `candidates` to self-consistency scoring. That clarifies the optional parameters' purpose rather than merely restating their schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Detect hallucination risk in AI-generated text') and then enumerates the four signals it analyses (hedging language, self-consistency across candidates, unsupported factual claims, source disagreement). That mechanism list uniquely identifies this tool against siblings like rai_scan or rai_trust_score without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the description shows the conditional value of supplying a `source` ('when a `source` is supplied ... enables explicit factual-disagreement detection'), which tells the agent when that parameter matters. There is no tool-level when-to-use guidance, no mention of when to prefer rai_scan, rai_trust_score or rai_redteam_analyze, and no stated prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_healthCheck Governance Engine HealthB
Read-onlyIdempotent

Check the status and module availability of the ResponsibleAI governance engine.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a safe, idempotent, non-destructive, closed-world read. The description adds only the scope hint of 'module availability,' and gives no sense of what a degraded/unhealthy result implies or how it behaves if a module is down.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; every word (status, module availability, governance engine) carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should ideally say what the health result contains (overall status, per-module list, error detail). 'Status and module availability' only partially covers the return contract.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so per the rubric the baseline is 4; there is nothing for the description to disambiguate beyond the schema's empty object.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Check') plus concrete resource ('status and module availability of the ResponsibleAI governance engine'), which is clearly a health/readiness probe. It does not, however, distinguish itself from adjacent status-oriented siblings such as rai_org_status or rai_webhook_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no prerequisites, and no mention of alternatives. The agent must infer that this is a pre-flight/availability check rather than something to run against a specific model or policy.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_incident_logBuild Governance Incident RecordC
Read-onlyIdempotent

Create a structured governance incident record. Used by Security Engineers and AI Risk Analysts to log AI safety events (PII leaks, jailbreak attempts, bias triggers, hallucination incidents) for audit trail and SIEM integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
evidenceNoSupporting data: prompt, response, scan results, etc.
providerNo
severityYes
mitigatedNo
model_nameNo
descriptionYesHuman-readable incident description
incident_typeYes

TDQS

C2.7/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explicitly says 'Create' and 'log ... events', i.e. a write/mutation, while annotations declare readOnlyHint=true and idempotentHint=true. This directly contradicts the structured safety metadata. The added context about audit trail and SIEM integration is useful, but the write-vs-read conflict is disqualifying.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with the core action front-loaded and no filler. Efficient, though it spends words on audience role labels rather than information an agent needs to invoke the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter, nested-object tool with no output schema and only 29% schema coverage, the description omits severity semantics, evidence schema, and return behavior. Combined with the annotation conflict, an agent lacks enough to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 29% across 7 parameters, so the description should compensate, but it only maps loosely to incident_type by naming event categories. It says nothing about severity levels, mitigated, evidence structure, provider, or model_name, leaving most required and optional fields undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Create a structured governance incident record') and enumerates the event classes it captures (PII leaks, jailbreak attempts, bias triggers, hallucinations). Clear on its own, but it names no sibling tools, so an agent gets no help distinguishing it from rai_pii_report, rai_compliance, or rai_audit_summary in a 30-tool family.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives the intended audience (Security Engineers, AI Risk Analysts) and the purpose (logging AI safety events for audit/SIEM), which implies usage context. However, it never says when to use this instead of the closely related reporting siblings, nor what prerequisites or thresholds trigger a log entry.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_iso42001_gapRun ISO 42001 Gap AnalysisC
Read-onlyIdempotent

Perform an ISO/IEC 42001:2023 AI Management System gap analysis. Evaluates maturity across all 10 clauses: Context, Leadership, Planning, Support, Operation, Performance Evaluation, Improvement, plus AI-specific annexes. Returns gap findings, maturity scores per clause, and a prioritised remediation roadmap. Used by AI Compliance Managers.

ParametersJSON Schema
NameRequiredDescriptionDefault
org_nameNoOrganisation
has_ai_policyNo
has_audit_trailNo
compliance_maturityNo
has_data_governanceNo
has_risk_assessmentNo
trust_score_overallNo
has_incident_processNo
has_impact_assessmentNo
has_supplier_controlsNo
has_monitoring_metricsNo
has_training_programmeNo
has_continual_improvementNo

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and non-destructive behavior, so the safety profile is covered. The description adds that scoring is per-clause maturity and that a prioritised remediation roadmap is returned, which is useful, but it omits that all 13 inputs are optional/defaulted (zero-param invocation is possible) and says nothing about determinism or data requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the operation and standard, then scope, then outputs in a tight three-sentence block. The trailing audience sentence adds little, but overall it is efficiently sized with no padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 13-parameter tool with no output schema and no parameter documentation, the description is too thin: it never explains how the boolean control flags or maturity/trust numbers drive the analysis, nor the shape of the findings beyond 'gap findings, maturity scores, roadmap'. An agent has insufficient detail to supply meaningful inputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 13 parameters, so the description carries the full burden of explaining org_name, the ten boolean control flags, compliance_maturity, and trust_score_overall. It mentions none of them, leaving an agent to infer each flag's meaning entirely from its name and default value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (gap analysis) and resource (ISO/IEC 42001:2023 AI Management System), which cleanly separates it from generic siblings like rai_compliance and rai_eu_ai_act_classify. However, it claims evaluation 'across all 10 clauses' while only listing 7 (Context through Improvement), a factual inconsistency that slightly muddies the scope claim.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives an audience ('Used by AI Compliance Managers') but no when-to-use guidance, no prerequisites, and no comparison to adjacent tools such as rai_compliance, rai_eu_ai_act_classify, or rai_policy_check. An agent cannot tell from this text when ISO 42001 gap analysis is the right call versus a generic compliance check.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_memory_read_checkCheck Memory Read Scope AuthorizationA
Read-onlyIdempotent

Gate a read from persistent agent memory. Standalone, this tool only echoes back the requested scope (there's no content to scan for a read) -- its real enforcement value is when governance is enabled on the hosted MCP server: the caller's authority's memory_scope constraint (if any) is checked against the requested memory_scope, denying cross-tenant/cross-agent memory access before the read happens.

ParametersJSON Schema
NameRequiredDescriptionDefault
memory_scopeYesThe memory namespace this read targets, e.g. 'org:acme:agent:bot1'.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent annotations by disclosing a critical behavioral trait: standalone the tool only echoes the requested scope, and real enforcement (denying cross-tenant/cross-agent access) occurs only under hosted governance. This is exactly the kind of non-obvious behavior an agent must know before invoking it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose and then the caveat in one dense but well-structured statement with zero filler. The em-dash aside is slightly awkward but every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter gate with no output schema and full annotation coverage, the description supplies everything needed: what it does, when it is meaningful, and the enforcement outcome. Nothing an agent needs to call and interpret it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning: the requested memory_scope is compared against the caller's authority's memory_scope constraint, clarifying the semantics of the value beyond the schema's example string.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Gate a read from persistent agent memory') and clearly separates itself from the read path. The name and description make its role as a pre-read authorization gate unambiguous, and the sibling rai_memory_write_check is implicitly distinguished by the 'read' framing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly explains the two contexts: standalone it is a no-op that echoes the scope, and it only enforces when governance is enabled on the hosted MCP server. This tells the agent when the tool matters versus when calling it is pointless. It stops short of naming the sibling tool or stating exclusions directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_memory_write_checkCheck Memory Write for Injection PatternsA
Read-onlyIdempotent

Gate a write to PERSISTENT agent memory (a vector DB, conversation log, or similar long-term store your own system owns -- WhitePact does not host a memory store itself). Scans the content for prompt-injection patterns aimed specifically at persistent memory -- text engineered to look like an instruction, a fake system/assistant role marker, or an override, so that a future session reading this memory back treats it as trusted context rather than as content. Call this BEFORE actually persisting the write; if 'allowed' is false, do not write it. When governance is enabled on the hosted MCP server and the caller's authority carries a memory_scope constraint, memory_scope is also checked for cross-tenant/cross-agent isolation -- outside the governed dispatch path this tool only runs the content scan.

ParametersJSON Schema
NameRequiredDescriptionDefault
contentYesThe text about to be written to memory.
memory_scopeNoThe memory namespace this write targets, e.g. 'org:acme:agent:bot1'.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this a safe, idempotent, non-destructive read (consistent with a scan that does not itself persist), so the bar is lower. The description adds real value beyond that: what patterns are targeted (fake role markers, overrides, instruction-like text), why (future sessions re-trusting the memory), and the conditional governance behavior of memory_scope. It could still be clearer on the exact return shape beyond the 'allowed' flag.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The gating purpose and the pre-write timing are front-loaded in the first sentence. The paragraph is dense but long, with some parenthetical asides (the WhitePact disclaimer) that could be tightened while still earning their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only 2 well-documented params and no output schema, the description covers the essential behavior, timing, and failure branch. The one remaining gap is the full shape of the result object, though it does name the 'allowed' field that drives the caller's decision.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining that memory_scope is a namespaced target checked for cross-tenant/cross-agent isolation only under governance, adding semantic meaning the schema field alone does not convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (gate/scan) and resource (a write to persistent agent memory), and explicitly limits the scope to persistent long-term stores rather than a general content filter. It is easily distinguished from the sibling rai_memory_read_check by the 'write' orientation and the pre-persist timing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit call sequence ("Call this BEFORE actually persisting the write") and a concrete decision rule ("if 'allowed' is false, do not write it"). It also discloses that the memory_scope isolation check only fires under governed dispatch, allowing the agent to predict when the extra check applies.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_model_routeRecommend Optimal ModelA
Read-onlyIdempotent

Recommend the optimal AI model for a task based on complexity analysis and cost-quality tradeoff. Returns recommended model, alternative, estimated cost per 1K tokens, and estimated savings vs GPT-4o. Used by LLMOps Engineers for intelligent model routing.

ParametersJSON Schema
NameRequiredDescriptionDefault
tasksNoOptional: batch route multiple task descriptions
task_descriptionNoNatural language description of the task
quality_requirementNomaximum: best model always; balanced: cost-quality tradeoff; cheapest: minimize costbalanced

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safe read-only, idempotent, non-destructive profile, so the bar is lower. The description adds real value by disclosing the response payload (recommended model, alternative, cost per 1K tokens, savings vs GPT-4o), which is otherwise undocumented because there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler, and the core purpose is front-loaded ahead of the return details and the audience note. The trailing LLMOps sentence is the only mildly expendable element.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter tool with full schema coverage and no output schema, the description covers purpose, basis, and expected return contents. The one unresolved point is that all parameters are optional, so the description never clarifies what happens on an empty call or which of task_description vs tasks is expected.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including an enum with its meanings, so the schema carries the parameter burden and baseline 3 applies. The description adds no syntax, format, or interaction detail beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Recommend the optimal AI model") with the basis for the recommendation (complexity analysis, cost-quality tradeoff) and enumerates the output. It is clear in isolation, but it never distinguishes itself from nearby siblings like rai_compare_models or rai_cost_estimate, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Used by LLMOps Engineers for intelligent model routing" implies the intended context but gives no explicit when-to-use conditions, no prerequisites, and no named alternative. An agent gets a rough sense of fit but nothing that resolves the choice against rai_compare_models or rai_cost_estimate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_org_statusGet Organization Governance StatusA
Read-onlyIdempotent

Compute a structured governance status rollup. The health/grade/compliance fields (models, compliance, operations) are always derived FROM caller-supplied metrics (model grades, active frameworks, open incidents, budget usage, drift alerts) -- there is no separate store of these tracked per-org yet, so calling with none supplied rolls up empty/default values, not fabricated data. The org_id/plan/usage fields are different: on the hosted MCP transport with an authenticated caller, these reflect the real, live org record and this month's real call count against quota -- absent (not present in the response at all) on the self-hosted stdio transport, where there is no org account to look up.

ParametersJSON Schema
NameRequiredDescriptionDefault
org_nameNodefault
drift_alertsNo
model_gradesNomodel_name → grade (A/B/C/D/F)
open_incidentsNo
budget_pct_usedNo
active_frameworksNo

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent annotations, the description discloses substantial non-obvious behavior: fields are derived, not stored, so empty input yields empty/default values rather than fabricated data; and org_id/plan/usage reflect a live org record only on the hosted transport and are absent on self-hosted stdio. This is exactly the transport- and data-provenance context an agent needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and every sentence carries a distinct, non-redundant fact (data provenance on one side, transport differences on the other). The two long sentences are dense but not padded; slightly more structure would aid scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does the work of explaining what the returned fields mean and when they will or won't be present across transports, which is the key completeness concern for this 6-parameter tool. It falls short only in not describing the shape/labels of the rollup that will be returned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 17%, so the description must carry weight. It enumerates the conceptual inputs (model grades, active frameworks, open incidents, budget usage, drift alerts), covering most required-meaning parameters, but adds no format, range, or default detail and never addresses org_name, so it only partially compensates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Compute a structured governance status rollup" states a specific verb (compute) and resource (governance status rollup), clearly telling the agent what is produced. It does not, however, distinguish itself from closely related siblings like rai_health, rai_executive_summary, or rai_trust_score, leaving overlap ambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by explaining that the health/grade/compliance fields are derived from caller-supplied metrics, nudging the agent to pass metrics in. But it names no alternative tools and gives no explicit 'use this when X, use Y instead' routing despite a crowded sibling set.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_passport_generateGenerate AI PassportB
Read-onlyIdempotent

Generate a verifiable AI Passport for a model — a tamper-evident governance card containing trust scores, compliance status, bias summary, and a cryptographic verification hash. Used by Procurement/Legal for third-party AI vendor risk assessment.

ParametersJSON Schema
NameRequiredDescriptionDefault
providerYes
use_caseNogeneral
model_nameYes
bias_summaryNo
privacy_summaryNo
security_summaryNo
trust_dimensionsYesTrust dimension scores (0-1 each)
compliance_summaryNo
hallucination_summaryNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the tamper-evident nature and cryptographic hash, which is useful behavioral context, but does not reconcile why a 'generate' tool is read-only, nor mention auth, persistence, or whether the passport is stored.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with the artifact definition front-loaded and the audience appended. Little wasted text, though the second sentence is context rather than operational instruction.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 9 parameters, 11% schema coverage, and no output schema, the description should do more. It partially covers output content (useful since no output schema exists) but leaves half the input parameters undocumented, so an agent cannot construct a full call confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 11%, so the description carries the burden of documenting parameters, and it largely fails. It references 'trust scores', 'compliance status', and 'bias summary' which loosely map to trust_dimensions/compliance_summary/bias_summary, but leaves model_name, provider, use_case, privacy_summary, security_summary, and hallucination_summary unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Generate') and resource ('AI Passport'), then enumerates the artifact's contents (trust scores, compliance status, bias summary, verification hash), so the agent knows exactly what this produces. It is fairly distinct from siblings like rai_trust_score or rai_compliance, but never names them, so it stops short of explicit differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Used by Procurement/Legal for third-party AI vendor risk assessment' line implies a usage context, which is better than nothing. However, it gives no when-to-use/when-not guidance or alternative routing (e.g., versus rai_trust_score, rai_compliance, or rai_audit_summary), leaving the agent to infer selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_pii_reportGenerate PII Audit ReportA
Read-onlyIdempotent

Generate a detailed PII audit report for a document or corpus. Classifies findings by PII category (email, phone, SSN, credit card, IP, address), counts occurrences, computes a privacy risk score, and provides GDPR/CCPA remediation guidance. Used by Privacy Engineers for compliance evidence collection.

ParametersJSON Schema
NameRequiredDescriptionDefault
textsYesList of text documents to scan
redactNoInclude redacted versions in report
contextNoContext label: medical | financial | hr | legal | generalgeneral

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safety profile (readOnlyHint, idempotentHint, destructiveHint=false, closed world), so the description does not need to restate them. It adds useful behavioral context by disclosing the analytical pipeline and report contents (classification, counts, risk score, remediation guidance), which is more than the annotations convey. It stops short of stating rate limits, input size caps, or data retention behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with zero filler, front-loaded on what the report is and what it contains. The trailing audience sentence is slightly less essential but still earns its place by anchoring the use case.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly compensates by describing the report's contents and compliance framing. Annotations cover safety and determinism. Remaining gaps — whether input texts are persisted, size/throughput limits, and exact exit/return structure — are minor for a three-parameter read-only tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters (texts, redact, context) are already fully documented in the schema. The description implies the scan input ('a document or corpus') and hints at redacted output, but it never explains the context parameter's enum-like label semantics (medical | financial | hr | legal | general) or how it affects classification. Baseline 3 is appropriate when the schema carries this burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource ('Generate a detailed PII audit report') and then enumerates exactly what the report contains: PII category classification, occurrence counts, a privacy risk score, and GDPR/CCPA remediation guidance. This level of detail distinguishes it from generic siblings like rai_scan or rai_compliance without needing to name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states a clear usage context ('Used by Privacy Engineers for compliance evidence collection'), which tells an agent when this tool is appropriate. However, it offers no explicit exclusions or named alternatives (e.g., when to prefer rai_scan or rai_compliance instead), so an agent must still infer the boundary.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_policy_checkCheck Text Against Governance PolicyA
Read-onlyIdempotent

Evaluate text or a model response against a governance policy. Checks for: prohibited topics, required disclaimers, output length limits, language restrictions, and custom keyword blocklist. Returns pass/fail per policy rule with remediation guidance.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesText to evaluate against policy
policyYesGovernance policy configuration

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true and destructiveHint=false, so the safety profile is covered. The description adds genuinely useful non-annotation behavior: the check categories performed and the fact that output is pass/fail per rule with remediation guidance — important since there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: purpose first, then the rule inventory, then the return shape. No filler and nothing redundant with structured fields, and the most decision-relevant content is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema but a nested policy object, the description carries real weight and discharges most of it by stating the return shape and the rule set. The gap is the 'language restrictions' claim, which no schema property supports, and the absence of any note on how PII checking defaults (the schema sets require_pii_clean=true).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 and the schema already documents every policy key. The description loosely mirrors those keys (prohibited topics, disclaimers, length limits, keyword blocklist) but adds no syntax, format or defaulting detail, and its mention of 'language restrictions' corresponds to no field in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb ('Evaluate') and resource ('text or a model response against a governance policy') and enumerates the exact rule classes checked, so an agent knows precisely what this tool produces. It does not, however, name or contrast itself with close siblings such as rai_scan or rai_compliance, leaving the agent to infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the object being evaluated ('text or a model response'), with no explicit when-to-use, when-not-to-use, or alternative named among the ~29 rai_* siblings. Given the crowded namespace (rai_scan, rai_compliance, rai_memory_write_check), a routing sentence would be expected.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_redteam_analyzeAnalyze Red Team ResponsesB
Read-onlyIdempotent

Analyse model responses to red team attack payloads. Returns a security report with vulnerability findings, severity breakdown, and an overall security score.

ParametersJSON Schema
NameRequiredDescriptionDefault
providerYes
responsesYesMap of attack_name → model_response_text
model_nameYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral value by stating the return shape: vulnerability findings, severity breakdown, and an overall security score, which matters because no output schema exists. It still omits any note on cost, latency, or whether the analysis is deterministic across runs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and closed with the return contents; every clause carries information and there is no padding. It could be marginally tighter by combining the report contents, but it is appropriately sized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly spends its budget describing the returned security report, which is the most important omission it could fill. However, for a three-parameter tool with a nested input object and no parameter documentation, the definition leaves the agent guessing about what provider and model_name should contain and where the responses are sourced from.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, and the description adds no information about any of the three required parameters. The nested responses map is documented in the schema, but provider and model_name are unexplained in both places, so the description fails to compensate for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: analyse model responses to red team attack payloads, and it names the artifact being analyzed (responses to attack payloads), which separates it from sibling tools like rai_redteam_payloads. It stops short of explicitly naming which sibling to use instead or when this differs from rai_scan or rai_bias_evaluate, so it is clear but not fully differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no statement of prerequisites (e.g., that responses must first come from rai_redteam_payloads or rai_scan), and no conditions under which this tool should be preferred over alternatives such as rai_trust_score. The agent must infer usage entirely from the tool name and description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_redteam_payloadsGet Red Team Attack PayloadsA
Read-onlyIdempotent

Return adversarial attack payloads to probe an AI model for security vulnerabilities. Categories: prompt_injection, jailbreak, data_leakage, role_confusion, delimiter_attack.

ParametersJSON Schema
NameRequiredDescriptionDefault
categoriesNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds that output is offensive-security payload content, which is meaningful context, but it says nothing about auth requirements, rate limits, or what happens when no categories are supplied.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the outcome and followed by the supported categories. The category list is slightly redundant against the schema enum, but it costs little and reads efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden; it identifies the payload content and category space but not the shape or volume of the response. Combined with annotations that cover the read-only/idempotent profile, it is adequate for a simple one-parameter lookup, with only the default-behavior gap remaining.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and there is a single optional 'categories' array parameter. The description enumerates the same values the schema already lists in the item enum, so it confirms the valid set but adds no new semantics — notably it does not state that omitting the parameter returns all categories.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource — 'Return adversarial attack payloads' — and states the purpose ('to probe an AI model for security vulnerabilities'), which separates it from analysis-oriented siblings like rai_redteam_analyze. It stops short of explicitly naming a sibling or contrasting with one, so it is clear but not fully differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The stated purpose ('probe an AI model for security vulnerabilities') implies when the tool is relevant, but there is no explicit when-to-use/when-not-to-use guidance and no alternative tool is named. An agent must infer that this is the payload-source tool and rai_redteam_analyze handles the results.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_scanScan for PII & Harmful ContentA
Read-onlyIdempotent

Scan text for PII (email, phone, SSN, credit card, IP address) and harmful content (hate speech, violence, self-harm). Returns findings and a redacted copy.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesText to scan
redactNoReplace detected PII with [REDACTED]

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, so the safety profile is covered. The description adds value by disclosing the response shape ('Returns findings and a redacted copy') and the redaction convention ([REDACTED]), which the annotations do not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler: the detection scope comes first and the return behavior second. Nothing is redundant or padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must carry return-value information, and it does so at least in outline ('findings and a redacted copy'), though it does not say what a 'finding' contains (type, spans, severity). For a simple 2-parameter tool this is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented at the field level. The description reinforces the redact behavior with the [REDACTED] placeholder, but adds no syntax, format, or edge-case meaning beyond the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Scan') plus resource ('text') with explicit enumerated detection categories (email, phone, SSN, credit card, IP; hate speech, violence, self-harm). No sibling is named for contrast, so the differentiation from rai_stream_scan or rai_pii_report is only implied by the 'single text' scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no alternatives named among the many rai_* siblings that overlap (rai_pii_report, rai_stream_scan, rai_policy_check). The agent must infer usage entirely from the verb 'Scan'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_stream_scanScan Streaming Text ChunksA
Read-onlyIdempotent

Scan a list of text chunks (as would arrive from an LLM streaming response) for PII and harmful content. Simulates the StreamingScanner guardrail without a live stream. Returns per-chunk scan results and an aggregated summary with stop recommendation.

ParametersJSON Schema
NameRequiredDescriptionDefault
chunksYesOrdered list of text chunks from LLM output stream
hard_stopNoStop processing after first PII detection
scan_windowNoScan every N chunks

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safe read-only, idempotent, closed-world profile, so the bar is lower. The description still adds real value by disclosing that the tool is a simulation of StreamingScanner requiring no live stream, and by naming the return shape (per-chunk results plus an aggregated summary with a stop recommendation) even though no output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: what is scanned, the simulation caveat, and the return shape. The core resource and scope are front-loaded with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully summarizes the return value, and all three parameters are fully covered by the schema. It is nearly complete for a read-only simulation tool; only the relationship to the sibling rai_scan is left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so chunks, hard_stop, and scan_window are already documented in the schema including defaults. The description adds nothing about parameter behavior (e.g., how hard_stop interacts with the stop recommendation), so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource ('Scan a list of text chunks ... for PII and harmful content') and scopes it to streaming LLM output, which is the key differentiator from the non-streaming sibling rai_scan. It does not name that sibling explicitly, so an agent must infer the routing from the word 'streaming'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Simulates the StreamingScanner guardrail without a live stream' implies a testing/simulation context, but the description never states when to choose this over rai_scan or any other scan tool, nor any prerequisites. Usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_trust_scoreCompute AI Trust ScoreB
Read-onlyIdempotent

Compute a composite AI Trust Score (0-100) across six governance dimensions: fairness, privacy, security, robustness, compliance, authenticity. Returns score, letter grade (A-F), and risk tier (LOW/MEDIUM/HIGH/CRITICAL).

ParametersJSON Schema
NameRequiredDescriptionDefault
privacyNo
fairnessNo
securityNo
complianceNo
robustnessNo
authenticityNo

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety and determinism profile is covered. The description adds useful context by disclosing the output shape (0-100 score, A-F grade, LOW-to-CRITICAL risk tier), but says nothing about weighting, determinism of the composite, or how missing dimensions are treated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences: the first states the computation and its dimensional scope, the second the return values. No filler and the core purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-output-schema tool the description does cover the return contract, and annotations cover safety. What is missing is the input scale/default behavior for six undocumented parameters and any differentiation from the sibling rai_check_trust, which is the main thing an agent needs to choose correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the burden, and it partially does by naming the same six dimensions as the parameters. However, it never explains that inputs are 0-1 normalized (while the output is 0-100), that all six default to 0.5, or how the dimensions are weighted, so the semantics of the actual inputs remain under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Compute a composite AI Trust Score') and enumerates the six governance dimensions and the output artifacts (score, letter grade, risk tier). It is clear what the tool does, but it never distinguishes itself from the close sibling rai_check_trust, leaving the agent to guess which trust-scoring tool to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of alternatives. With a near-identical sibling (rai_check_trust) in the toolset, the absence of routing criteria is a real gap rather than a minor omission.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rai_webhook_statusCheck Webhook Delivery HealthB
Read-onlyIdempotent

Check webhook delivery health and generate a structured status report. Takes delivery statistics and returns health grade, failure analysis, dead-letter queue status, and recommended remediation actions. Used by Security Engineers feeding SIEM systems and Platform Engineers debugging webhook pipelines.

ParametersJSON Schema
NameRequiredDescriptionDefault
failedNo
endpointsNo
successfulNo
avg_latency_msNo
total_deliveriesNo
dead_letter_countNo

TDQS

B3.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, and closed-world behavior. The description adds what the report contains (health grade, failure analysis, dead-letter queue status, remediation actions), which is useful given no output schema, but does not describe permissions, rate limits, or other operational traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, then input/output, then audience. No redundant or filler content; every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has six parameters with no schema descriptions and no output schema. While the description covers the high-level purpose and output, it fails to explain the parameters an agent must supply, leaving a significant gap for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry the full burden of explaining the six parameters. It only says 'Takes delivery statistics' generically and does not clarify the meaning, format, or role of any parameter such as 'endpoints' or 'dead_letter_count'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (check) and resource (webhook delivery health) plus the output (structured status report). It is clearly distinct from general health tools like rai_health by virtue of 'webhook', but does not explicitly name or contrast with siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides intended users (Security/Platform Engineers) and contexts (SIEM, debugging pipelines), which implies when it is useful, but offers no explicit 'use this instead of X when Y' guidance or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 30 tool updatesv1.2.6
    • First observedrai_audit_summary
    • First observedrai_benchmark
    • First observedrai_benchmark_prompts
    • First observedrai_bias_evaluate
    • First observedrai_budget_check
    • First observedrai_causal_influence_check
    • First observedrai_check_trust
    • First observedrai_compare_models
    • First observedrai_compliance
    • First observedrai_cost_estimate
    • First observedrai_drift_check
    • First observedrai_eu_ai_act_classify
    • First observedrai_executive_summary
    • First observedrai_hallucination
    • First observedrai_health
    • First observedrai_incident_log
    • First observedrai_iso42001_gap
    • First observedrai_memory_read_check
    • First observedrai_memory_write_check
    • First observedrai_model_route
    • First observedrai_org_status
    • First observedrai_passport_generate
    • First observedrai_pii_report
    • First observedrai_policy_check
    • First observedrai_redteam_analyze
    • First observedrai_redteam_payloads
    • First observedrai_scan
    • First observedrai_stream_scan
    • First observedrai_trust_score
    • First observedrai_webhook_status

TDQS

A3.7/5.0

Scored across 30 tools

Disambiguation5/5

Despite covering overlapping governance themes, descriptions carry explicit 'use X instead of Y' guidance (e.g., rai_compliance vs rai_eu_ai_act_classify, rai_trust_score vs rai_check_trust), and the scan-family (rai_scan/rai_stream_scan/rai_pii_report) and memory-family tools are clearly differentiated. Each tool has a distinct resource+action target with no true duplicates.

Naming Consistency5/5

Every tool uses a uniform rai_ prefix with a consistent snake_case convention (rai_trust_score, rai_policy_check, rai_bias_evaluate). No mixing of camelCase, no erratic verb styles.

Tool Count3/5

30 tools is heavy and pushes past the comfortable ceiling, even if the governance domain is genuinely broad (privacy, security, compliance, cost, bias, audit). Most tools earn their place, but a set this large increases selection burden for agents.

Completeness5/5

The surface covers an unusually full governance lifecycle: scanning, trust scoring, bias evaluation, compliance (NIST/EU AI Act/ISO 42001), red-teaming, cost/budget, drift, passports, incident logging, and memory/provenance gating. No obvious dead ends for the stated purpose.

Maintenance

ActivityActive
ResponsivenessResponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers