WhitePact
Provides a trust-gate integration for Google ADK (Agent Development Kit) agents, enabling them to query WhitePact's trust lookup for third-party tools and gate tool calls in-agent based on the returned trust verdict.
Provides a trust-gate integration for LangChain agents: agents can call WhitePact's rai_check_trust lookup before invoking a third-party tool and receive a real block/pause gate in-agent when the tool is untrusted.
Provides a trust-gate integration for LangGraph agents, wiring WhitePact's five-way governance decisions and trust lookup into agent graphs so untrusted tool calls can be blocked or held for approval before execution.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@WhitePactcheck if this agent action is safe to allow"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
┌──────────────────────────────────────────────────────────────────────────────┐
│ WhitePact v1.2.6 │
│ │
│ ┌──────────────┐ ┌─────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Governance │ │ Trust Score │ │ Compliance │ │ Guardrails │ │
│ │ 5-way decide │ │ 6-dim A–F │ │ NIST/EU/ISO │ │ PII + Tox │ │
│ └──────────────┘ └─────────────┘ └──────────────┘ └──────────────────┘ │
│ ┌──────────────┐ ┌─────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Hallucination│ │ Cost Intel │ │ Red Team │ │ Drift Monitor │ │
│ │ Self-consist.│ │ Route+Budget│ │ 10 attacks │ │ Alerts+Trend │ │
│ └──────────────┘ └─────────────┘ └──────────────┘ └──────────────────┘ │
│ ┌──────────────┐ ┌─────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ AI Passport │ │ BiasBuster │ │ PrivacyLabel │ │ MCP Server │ │
│ │ SHA-256 cert │ │ 6 probes+CI │ │ Federated │ │ 30 tools/HTTP │ │
│ └──────────────┘ └─────────────┘ └──────────────┘ └──────────────────┘ │
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │ Governance Dashboard — FastAPI · Per-org rate limit · Alembic · OTEL │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘What this solves
Every team deploying AI in production faces the same gap: no unified way to prove a model — or an autonomous agent's actions — is safe, fair, compliant, and accountable. Audits are manual, bias is discovered in production, compliance is a spreadsheet, an agent's tool calls go ungoverned, and nobody knows what the LLM bill will be next month.
WhitePact gives you one platform — a REST API, a Python SDK, an MCP server, and a live dashboard — that covers the full governance lifecycle:
Problem | Module | Output |
Should this agent action be allowed, redacted, held for approval, denied, or quarantined? |
| A five-way |
Is this model trustworthy? |
| 0–100 score, A–F grade, risk level |
Does it comply with regulations? |
| NIST AI RMF, EU AI Act tier, ISO 42001 |
Is it exposing PII? |
| Block / redact with audit log |
Is it hallucinating? |
| Risk score, unsupported claims |
Can it be attacked? |
| 10 vectors, CVE IDs, safe-refusal rate |
How much is it costing? |
| Per-model USD, routing to cheapest viable model |
Is it getting worse over time? |
| 7/30-day trend, severity alerts |
Is it biased? |
| 6 demographic probes, CI gate |
Is this data labeled privately? |
| Federated DP labels, never leaves device |
Is this media real? |
| Ensemble confidence, method detected |
Can I trust a third-party MCP server before connecting to it? |
| VERIFIED_FACT / INFERRED_SIGNAL / UNKNOWN verdicts — typosquat, description-content, known-incident checks |
Is there a tamper-evident record of every governance decision? |
| Hash-chained |
Does a risky action get a human in the loop? |
| Race-safe |
How does this model rank against others, independently? |
| Cross-model trust ranking from actually calling each model's API, not self-reported |
Can I cite and verify a trust score anywhere? |
| Free self-assessed or human-reviewed certified passport, verifiable at |
Has this AI system failed publicly before? |
| Crowd-reported, moderator-reviewed, hash-chained public registry |
Should my agent trust this third-party tool before calling it? |
| Free lookup, plus a real block/pause gate in-agent |
Can any MCP client govern every AI call? |
| 27 governance tools over stdio, Streamable HTTP, or legacy HTTP+SSE |
Related MCP server: production-grade-mcp-agentic-system
Install
# Governance platform + REST API
pip install "rai-governance-platform[dashboard]"
# With PostgreSQL support
pip install "rai-governance-platform[dashboard,postgres]"
# With Redis + OpenTelemetry
pip install "rai-governance-platform[dashboard,redis,telemetry]"
# With LLM providers
pip install "rai-governance-platform[dashboard,openai,anthropic]"
# Everything
pip install "rai-governance-platform[all]"The published PyPI package name (rai-governance-platform) and the import
name (responsibleai) predate the WhitePact rename and are kept as-is —
see MIGRATION_WHITEPACT_V2.md Section 3 for why an alias package
(whitepact) was added instead of renaming the published package outright.
30-second quickstart
# Start the governance dashboard
pip install "rai-governance-platform[dashboard]"
uvicorn responsibleai.dashboard.app:app --port 8765
# Evaluate a model (no LLM key needed — supply your own scores)
curl -X POST http://localhost:8765/api/evaluate \
-H "Content-Type: application/json" \
-d '{
"model_name": "gpt-4o",
"provider": "openai",
"fairness": 0.80,
"privacy": 0.85,
"security": 0.82,
"robustness": 0.78,
"compliance": 0.90,
"authenticity": 0.88
}'{
"trust_score": { "trust_score": 83.65, "grade": "B", "risk": "LOW" },
"compliance": { "overall_score": 80.5, "eu_ai_act_tier": "limited_risk", "violations": 0 },
"passport_id": "rai-a3f7c2b1",
"passport_hash": "4d8e1f2a9c3b7e6d...",
"drift_alert": null
}Open http://localhost:8765 for the live dashboard and
http://localhost:8765/api/docs for interactive API docs.
Governance core — five-way decisions, not a binary block/allow
src/responsibleai/governance/ (see SPEC.md Sections 4-8 for the full
architecture contract) is a deterministic runtime authority sitting in front
of agent tool calls:
from responsibleai.governance import WhitePactRuntimeGateway, ActionRequest, AuthorityContext
gateway = WhitePactRuntimeGateway()
result = gateway.evaluate(
action=ActionRequest(tool_name="rai_scan", arguments={"text": "..."}),
authority=AuthorityContext(org_id="acme", agent_id="agent-1"),
)
print(result.decision) # GovernanceDecision.ALLOW | ALLOW_WITH_REDACTION | REQUIRE_APPROVAL | DENY | QUARANTINERisk tiering (
governance/risk.py) — every MCP tool is classified against a hardcoded, drift-tested table, not inferred at call time.Policy engine (
governance/policy.py) — first-match-wins rules withALLOW/DENY/REQUIRE_APPROVALeffects.Evidence (
governance/evidence.py) — every decision is written to a per-org, hash-chainedEvidenceRecord;verify_chain()detects tampering. Raw argument values are never stored, only field-name keys.Approval workflow (
governance/approval.py) —REQUIRE_APPROVALdecisions queue a real, race-safeApprovalRequestwith a resolution API, not just a log line.Supply-chain scanner (
src/responsibleai/supplychain/) — before an agent trusts a third-party MCP server or tool,SupplyChainScannerreturns one of three explicit verdicts (VERIFIED_FACT/INFERRED_SIGNAL/UNKNOWN) — never a single opaque trust score — from typosquat detection, tool-description scanning, and known-incident cross-reference.Identity Bridge (
integrations/identity_bridge.py) — maps Entra ID, Google Workspace, Okta, and AWS (Cognito / IAM Identity Center) ID token claims intoIdentityContext, plusmap_groups_to_authority()to turn IdP group membership into a granted-action-typesAuthorityContext. SeeMACHINE_AUTHORITY_V1.md's Identity Bridge section for exactly what's verified (claim-shape correctness against each provider's public docs) versus not (live-tenant testing, Graph/Admin-SDK group-name resolution, AWS's non-JWT SigV4 path).
No governance decision is LLM-based; see
DETERMINISTIC_VS_PROBABILISTIC.md for why.
See it end-to-end: examples/08_whitepact_enterprise_scenario.py runs a
full scenario (an org onboarding an autonomous finance agent) through all
eight machine-authority invariants — ceiling, delegation, attenuation,
approval quorum, workflow composition, autonomy budget, memory firewall,
evidence bundle — against real code, no API keys required:
python examples/08_whitepact_enterprise_scenario.pyMCP Server — govern every AI call from Claude Code, Claude Desktop, or any MCP client
The MCP (Model Context Protocol) server exposes WhitePact as 30 tools and
20 resources (10 canonical resource URIs, dual-advertised under both
whitepact:// and rai:// schemes — see MIGRATION_WHITEPACT_V2.md) to any
MCP-compatible client — Claude Code, Claude Desktop, Cursor, Windsurf, or your
own agent runtime. Three transports are supported: stdio, Streamable HTTP
(/mcp, current MCP spec), and legacy HTTP+SSE (/sse + /messages/, kept
for older clients). When a team's client points at this server, every AI
interaction is automatically governed — five-way governance decisions, trust
scoring, guardrails, compliance checks (NIST AI RMF / EU AI Act / ISO 42001),
bias evaluation, drift detection, cost tracking, and hash-chained audit
evidence run on any call without code changes.
Setup
# Install
pip install "rai-governance-platform[dashboard,mcp]"
# Start the REST API (MCP tools call it internally)
RAI_DB_PATH=/var/lib/rai/governance.db \
RAI_API_KEYS=your-key-here \
uvicorn responsibleai.dashboard.app:app --host 127.0.0.1 --port 8765 &
# Add to Claude Code (~/.claude/claude_desktop_config.json or via /mcp){
"mcpServers": {
"whitepact": {
"command": "whitepact-mcp",
"env": {
"RAI_API_URL": "http://localhost:8765",
"RAI_API_KEY": "your-key-here"
}
}
}
}whitepact-mcp and responsibleai-mcp are the same entry point — see
pyproject.toml's [project.scripts]; both will keep working, use whichever
name you prefer.
Available tools (27)
Tool | What it does |
| Detect and redact PII + harmful content before it reaches a log |
| Composite AI Trust Score (0-100) across 6 governance dimensions |
| NIST AI RMF / EU AI Act / ISO 42001 compliance evaluation |
| Hallucination risk from hedging, consistency, unsupported claims |
| USD cost of a model API call from token counts |
| Adversarial attack payloads (prompt injection, jailbreak, etc.) |
| Security report from model responses to red team payloads |
| Compare two models across all 6 trust dimensions |
| Governance capability summary (tools, frameworks, attack vectors) |
| Status and module availability of the governance engine |
| Demographic bias across 6 probe dimensions with confidence intervals |
| Trust score drift between a baseline and current evaluation |
| Verifiable, tamper-evident AI Passport for vendor risk assessment |
| Spend vs. budget, per-team/model breakdown, month-end projection |
| Text/response against a governance policy (blocklists, disclaimers) |
| PII/harm scan across streaming LLM output chunks |
| Score responses against truthfulqa / bbq / hellaswag suites |
| Question set for a benchmark suite |
| Cheapest model that can handle a task, with cost/quality tradeoff |
| PII audit report by category with GDPR/CCPA remediation guidance |
| Structured governance incident record for audit/SIEM |
| EU AI Act risk tier classification with compliance roadmap |
| ISO/IEC 42001:2023 AI Management System gap analysis |
| Board-ready governance summary with RAG status indicators |
| Governance status snapshot: models, grades, compliance, risk |
| Webhook delivery health, failure analysis, remediation actions |
| Free public Trust Index lookup for a third-party model/tool, before an agent invokes it — unlike every other tool above, which evaluates output the caller itself produced |
Agent-framework integrations — LangChain, LangGraph, Google ADK
src/responsibleai/integrations/ wires rai_check_trust directly into three
agent frameworks so an agent can be gated on a tool's public trust score
before invoking it, not just log the call after the fact:
LangChain (
langchain_middleware.py) —TrustGateMiddleware, awrap_tool_callmiddleware that blocks a call outright when its score is below threshold. Requirespip install "rai-governance-platform[langchain]".LangGraph (
langgraph_gate.py) —make_trust_gate_node(), a node that pauses the graph withinterrupt()for a human approve/reject decision on a below-threshold call, instead of a hard block. Requirespip install "rai-governance-platform[langgraph]".Google ADK (
adk_toolset.py) —build_stdio_toolset()/build_http_toolset(), thin factories over ADK'sMcpToolset, which auto-discovers this project's MCP server's tools with no custom glue code. Requirespip install "rai-governance-platform[adk]".
All three, or any subset, install via pip install "rai-governance-platform[agent-frameworks]".
See GAME_CHANGER_BUILD_PLAN.md Phase B for the reasoning behind each.
Available resources (20)
10 canonical resources, each advertised under both the whitepact:// and
rai:// URI schemes (dual scheme is additive — see
MIGRATION_WHITEPACT_V2.md; the table below shows the canonical URI):
Resource | URI | Contents |
Health |
| Current health status of the governance service |
Model pricing catalog |
| Supported models with per-token pricing |
Compliance frameworks |
| NIST AI RMF, EU AI Act, ISO 42001 |
Red team categories |
| Adversarial attack categories |
Trust dimensions |
| The 6 dimensions behind the Trust Score |
Bias probe catalog |
| Available bias probes and scoring interpretation |
Governance policy template |
| Default policy template for |
Trust grade reference |
| Grade thresholds, risk tiers, deployment guidance |
NIST AI RMF checklist |
| Actionable NIST implementation checklist |
EU AI Act checklist |
| Compliance checklist for high-risk operators |
MCP directory listings
WhitePact is listed and queryable today on real MCP directories — not aspirational, all verified live:
Official MCP Registry —
server.jsonat the repository root (schema2025-12-11, listing version1.2.3) is published asio.github.Guruprasath-Annadurai/whitepact, confirmed queryable at registry.modelcontextprotocol.io. Advertises both the PyPI/stdio package (whitepact-mcp, self-hosted, free, unrestricted) and aremotesentry pointing at the hosted Streamable HTTP and SSE transports (whitepact-mcp-http.onrender.com) — a one-click remote connector, not just an installable package.Antigravity CLI plugin —
plugins/whitepact/at the repository root follows the official Antigravity plugin manifest format, connecting to the same hosted Streamable HTTP transport viaserverUrl. No official Antigravity plugin directory exists yet, so this is distributed directly from the repo — seeplugins/whitepact/README.md.Smithery — listed as
guruprasathannadurai-official/whitepact, 30 tools and 20 resources discovered against the hosted Streamable HTTP transport (whitepact-mcp-http.onrender.com/mcp, a separate Render service from the main dashboard). This deployment has no OAuth authorization server configured — only static Bearer API keys — so a public, unauthenticated/.well-known/mcp/server-card.jsonserves the same liveTOOL_DEFS/RESOURCE_DEFSthe server itself advertises, for directories whose scanners can't complete a live authenticated crawl.
See compliance/MCP_DISTRIBUTION_GUIDE.md for the full distribution
plan, including directories not yet submitted to.
Platform integrations
WhitePact connects to the major AI platforms as one MCP server through
standards-compliant clients — no per-platform forks, no per-platform
governance logic. See docs/integrations/ for the
canonical compatibility matrix (PLATFORM_COMPATIBILITY.md), per-platform
setup docs (GitHub Copilot, Microsoft Copilot, Claude, Grok, Gemini,
Amazon Q, AWS Bedrock AgentCore, Mistral Le Chat, Cursor), and
FOUNDER_ACTIONS.md for what still needs a human. Run
python scripts/integration_smoke.py for a live protocol-level preflight
against the hosted endpoint.
Python SDK
Trust scoring
from responsibleai import TrustScoreEngine, PassportGenerator
engine = TrustScoreEngine()
score = engine.compute(
fairness=0.80, privacy=0.85, security=0.82,
robustness=0.78, compliance=0.90, authenticity=0.88,
)
print(f"{score.overall:.1f} / 100 Grade: {score.grade} Risk: {score.risk_level}")
# → 83.7 / 100 Grade: B Risk: LOW
passport = PassportGenerator().generate(
model_name="gpt-4o", provider="openai", trust_score=score,
compliance_summary={"overall": 80.5},
)
print(passport.passport_id)
passport.export_html("passport.html")Guardrails — block PII before it reaches a log
from responsibleai import GuardrailsEngine
guardrails = GuardrailsEngine()
result = guardrails.scan("Customer SSN is 123-45-6789, email: alice@company.com")
print(result.is_blocked) # True
print(result.pii_count) # 2
print(result.redacted_text) # "Customer SSN is [SSN], email: [EMAIL]"Hallucination detection
from responsibleai import HallucinationDetector
detector = HallucinationDetector()
result = detector.analyze(
"AI will replace all human jobs by 2025.",
candidates=[
"AI will automate some repetitive tasks.",
"AI creates new job categories alongside displacing others.",
],
)
print(f"Risk: {result.hallucination_risk:.2f} Level: {result.risk_level}")Compliance — NIST AI RMF, EU AI Act, ISO 42001
from responsibleai import ComplianceEngine
engine = ComplianceEngine()
report = engine.evaluate(
fairness_score=0.80, privacy_score=0.85,
security_score=0.82, robustness_score=0.78,
compliance_maturity=0.90, use_case="credit_scoring",
)
print(f"Score: {report.compliance_score * 100:.1f}%")
print(f"EU AI Act tier: {report.eu_ai_act_tier.value}") # high_riskRed team simulation
from responsibleai import RedTeamSimulator
simulator = RedTeamSimulator()
report = simulator.run_all()
print(f"Security score: {report.security_score:.1f}/100")
print(f"Vulnerabilities: {len(report.vulnerabilities)}")
for v in report.critical_vulnerabilities:
print(f" [{v['cwe_id']}] {v['name']}")Cost intelligence
from responsibleai import CostTracker, ModelRouter, TokenUsage, BudgetPolicy
tracker = CostTracker(db_path="~/.responsibleai/data.db",
policy=BudgetPolicy(monthly_limit_usd=500.0))
usage = TokenUsage.create(
provider="openai", model="gpt-4o",
input_tokens=2000, output_tokens=800, team="product",
)
record = tracker.record(usage)
print(f"This call: ${record.total_cost:.4f}")
print(f"Month to date: ${tracker.total_cost(30):.2f}")
router = ModelRouter()
decision = router.route("Classify this email as spam or not spam", "balanced")
print(f"Recommended: {decision.recommended_model} ${decision.estimated_cost_per_1k:.4f}/1k tokens")Trust drift monitoring
from responsibleai import TrustScoreEngine, TrustDriftMonitor
monitor = TrustDriftMonitor(db_path=":memory:", alert_threshold=5.0)
engine = TrustScoreEngine()
for fairness in [0.90, 0.88, 0.85, 0.72]:
score = engine.compute(fairness=fairness, privacy=0.85, security=0.80,
robustness=0.80, compliance=0.85, authenticity=0.85)
alert = monitor.record("gpt-4o", "openai", score)
if alert:
print(f"Drift alert! {alert.severity}: {alert.delta:.1f} pt drop")Governance Dashboard
A production FastAPI application with a dark-mode SPA. A live instance is hosted at whitepact.com.
# Development (auth off, SQLite in-memory)
RAI_AUTH_ENABLED=false uvicorn responsibleai.dashboard.app:app --port 8765
# Production (auth + persistent DB)
RAI_API_KEYS=your-key-here \
RAI_DB_PATH=/data/responsibleai.db \
uvicorn responsibleai.dashboard.app:app --host 0.0.0.0 --port 8765 --workers 4
# Docker
docker compose up -dREST API endpoints
Method | Path | Description |
|
| Health — DB, auth, OTEL, version |
|
| Uptime, request count, error rate, monthly spend |
|
| Full evaluation → trust + compliance + passport |
|
| Score history + drift trend |
|
| All evaluated models |
|
| Guardrails — PII detection + redaction |
|
| Hallucination risk analysis |
|
| Record token usage |
|
| Cost breakdown by model / team / day |
|
| Prompt efficiency — detect bloat |
|
| Route task to cheapest viable model |
|
| Full model pricing catalogue |
|
| Drift trend + history |
|
| Paginated audit log (org-scoped) |
|
| Export audit log as JSONL or CSV |
|
| Audit counts grouped by endpoint |
|
| Red team payload library (10 vectors) |
|
| Analyze model responses for vulnerabilities |
|
| Token spend and budget status |
|
| Public cross-model trust leaderboard (no auth) |
|
| Trend over time for one model (no auth) |
|
| Per-prompt findings — PRO plan required |
|
| Free, public self-assessment against the open Trust Index standard |
|
| Verify a cited Trust Index score (no auth) |
|
| Free, public — trust score + incident count for a named model/tool, by exact name (no auth); what |
|
| Every assessed model/tool, certified and self-reported, newest first (no auth) — data source for the public |
|
| Directory of certified passports (no auth) |
|
| Certify a passport — super-admin only |
|
| Embeddable trust badge (Self-Assessed / Certified), no auth |
|
| Report a publicly observed AI incident (no auth, rate-limited) |
|
| Browse published incidents — filter by model, provider, severity, type (no auth) |
|
| Pre-deployment exact-match incident check for a model/provider — PRO/ENTERPRISE |
|
| Recompute the hash chain over every published entry (no auth) |
|
| Enroll an API key in TOTP MFA |
|
| Verify a TOTP code / backup code |
|
| Read/write hash-chained governance evidence records |
|
| Queue and resolve |
Interactive docs at /api/docs. Public leaderboard page at /leaderboard —
see compliance/LEADERBOARD_METHODOLOGY.md for the published scoring
methodology and scripts/run_leaderboard_eval.py to run evaluations. Open
Trust Index standard and passport verification at /verify/{id} — see
compliance/TRUST_INDEX_SPEC.md. Free, zero-signup self-assessment at
/assess; browse every assessed model/tool at /registry. /llms.txt
points AI crawlers/answer engines at these as canonical sources — see
GAME_CHANGER_STRATEGY.md for why.
Production features
Feature | Detail |
Authentication | Bearer token ( |
MFA | TOTP (RFC 6238) on the interactive login step, org-enforceable, single-use backup codes |
Field-level encryption | Opt-in ( |
Per-org rate limiting | Each Bearer token gets its own rate limit bucket (SHA-256 keyed) — no shared global pool |
CORS | Configurable origins ( |
Security headers | CSP, X-Frame-Options, X-Content-Type-Options |
Structured logging | JSON via structlog + request IDs |
Database | SQLite (default) or PostgreSQL ( |
Observability | OpenTelemetry traces + metrics ( |
Webhooks | HMAC-signed delivery with DB-persisted retry queue (survives restarts) |
Exception handling | No raw stack traces reach clients |
Governance evidence | Hash-chained, per-org, tamper-evident ( |
Database migrations (Alembic)
Schema changes are managed with Alembic. Run alembic history for the
current, authoritative migration count and table list — this number changes
frequently enough that a hardcoded count here goes stale fast; the command
itself is the source of truth.
# Upgrade to latest schema
RAI_DB_PATH=/var/lib/rai/governance.db alembic upgrade head
# PostgreSQL
RAI_DB_URL=postgresql://user:pass@host:5432/responsibleai alembic upgrade head
# Show migration history
alembic history
# Generate a new migration after changing engine.py
alembic revision --autogenerate -m "add_new_column"All migrations use render_as_batch=True so they run on both SQLite and
PostgreSQL without changes.
Webhook notifications
Register an endpoint and receive signed events when governance thresholds fire.
# Register a Slack webhook
curl -X POST http://localhost:8765/api/webhooks \
-H "Authorization: Bearer your-key" \
-H "Content-Type: application/json" \
-d '{
"name": "ops-slack",
"url": "https://hooks.slack.com/services/...",
"events": ["drift_alert", "budget_exceeded", "guardrail_triggered"],
"provider": "slack",
"secret": "hmac-secret-for-signature-verification",
"max_retries": 5
}'Deliveries are persisted to the database. If the server restarts during a retry cycle, the background worker picks up where it left off on next boot. Retry schedule: 1 s → 5 s → 30 s → 2 min → 10 min.
Verify payloads with the X-RAI-Signature-256: sha256=<hex> header.
Docker
git clone https://github.com/Guruprasath-Annadurai/Whitepact.git
cd Whitepact
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
cp .env.example .env
# Edit .env — set RAI_API_KEYS
docker compose up -d
# Dashboard: http://localhost:8765
# API docs: http://localhost:8765/api/docsPostgreSQL + Redis (horizontal scaling)
# .env
RAI_DATABASE_URL=postgresql://rai:secret@db-host:5432/responsibleai
RAI_REDIS_URL=redis://redis-host:6379/0
RAI_OTEL_ENDPOINT=http://otel-collector:4318
pip install "rai-governance-platform[dashboard,postgres,redis,telemetry]"
# Run migrations before first start
RAI_DB_URL=postgresql://rai:secret@db-host:5432/responsibleai alembic upgrade headThe async database layer uses SQLAlchemy with connection pooling
(pool_size=10, max_overflow=20, pool_pre_ping=True). Rate limiting
switches to Redis-backed storage when RAI_REDIS_URL is set.
BiasBuster — bias evaluation in CI
# Fail CI when demographic bias exceeds threshold
biasbuster run \
--provider openai --model gpt-4o \
--probes gender-bias,racial-bias,cultural-bias \
--threshold 0.20 \
--output report --format htmlfrom biasbuster import BiasBusterRunner, GenderBiasProbe, RacialBiasProbe
from biasbuster.providers import OpenAIProvider
import asyncio
async def main():
provider = OpenAIProvider(api_key="sk-...", model="gpt-4o")
runner = BiasBusterRunner(provider=provider)
suite = await runner.run([
GenderBiasProbe(threshold=0.20),
RacialBiasProbe(threshold=0.20),
])
print(f"Score: {suite.overall_score:.4f} {'PASSED' if suite.passed else 'FAILED'}")
asyncio.run(main())Available probes: gender-bias, racial-bias, age-bias, religious-bias, occupational-stereotype, cultural-bias
Scoring: TF-IDF cosine divergence + length asymmetry + VADER sentiment divergence, 95% bootstrap confidence intervals, intersectional co-failure amplification (×1.15).
PrivacyLabel — on-device federated labeling
from privacylabel import FederatedClient, FedAvgAggregator
client = FederatedClient(
node_id="hospital-node-01",
provider=MyProvider(),
epsilon_per_round=0.1,
total_epsilon=1.0,
delta=1e-6,
gradient_clip=1.0,
)
# Raw data stays on disk — only privatised gradients leave the device
summary = await client.train_round("data/local_records.jsonl")
print(f"Privacy budget used: ε={summary.privacy_spent['spent_epsilon']:.3f}")Implements Laplace, Gaussian, Exponential, and DP-SGD mechanisms. Byzantine-robust aggregation via Weiszfeld geometric median.
GitHub Actions — bias gate in CI
- name: Bias evaluation
run: |
pip install "rai-governance-platform[openai]"
biasbuster run \
--provider openai --model gpt-4o-mini \
--probes gender-bias,racial-bias,cultural-bias \
--threshold 0.20
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}Environment variables
Variable | Default | Description |
|
| SQLite path |
| (unset = SQLite) | Full SQLAlchemy URL — takes priority over |
| (unset) | Alias for |
| (empty = auth off) | Comma-separated bearer tokens |
|
| Toggle auth enforcement |
| (unset = in-memory) | Redis URL for distributed rate limiting |
|
| Per-org rate limit (keyed by Bearer token) |
| (unset = disabled) | OTLP HTTP endpoint |
|
| Service name for traces |
|
| Trust score drop that triggers drift alert |
|
| Monthly AI spend limit |
|
| Log level |
|
| Structured JSON logs |
|
| Bind address |
|
| Port |
Dual-prefixed WHITEPACT_* equivalents for these are also read where
MIGRATION_WHITEPACT_V2.md documents them — the RAI_* names remain the
primary, always-supported form.
Development
git clone https://github.com/Guruprasath-Annadurai/Whitepact.git
cd Whitepact
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
# Full test suite (run it to see the current test count and coverage —
# see CONTRIBUTING.md's Running Tests section for why no number is
# hardcoded here)
pytest
# Dashboard tests only
RAI_DB_PATH=:memory: RAI_AUTH_ENABLED=false pytest tests/test_dashboard_api.py
# Webhook persistence tests
pytest tests/test_webhook_persistence.py
# MCP server tests
pytest tests/test_mcp_server.py
# Lint + type check
ruff check src/ tests/
mypy src/responsibleai src/biasbusterRoadmap
See ROADMAP.md for the canonical NOW/NEXT/LATER plan. The list below is a historical, version-by-version changelog summary kept for reference.
v0.1 — BiasBuster: gender probe, 4 providers, CLI, CI integration
v0.2 — Racial / age / religious / occupational probes, HTML reporter, PrivacyLabel federated DP
v0.3 — Cultural bias, intersectional analysis, DeepfakeDetector ensemble
v0.4 — Cost Intelligence (CostTracker, ModelRouter, 16-model pricing), Trust Drift Monitor
v0.5 — Governance Dashboard (FastAPI), Trust Score, AI Passport, Guardrails, Hallucination, Compliance, Red Team, CI/CD, Docker, SLA
v0.6 — Async PostgreSQL (SQLAlchemy), Redis rate limiting, OpenTelemetry APM, LLM integration tests
v1.0 — WebSocket drift alerts, Prometheus endpoint, multi-tenant RBAC, org management API
v1.1 — MCP server (10 tools, 5 resources), audit log API, red team API, billing API, Alembic migrations, per-org rate limiting, DB-persisted webhook retry queue
v1.2 — Public Leaderboard, Trust Index/Passports + embeddable badges, AI Incident Database, TOTP MFA, expanded field encryption, DB-persisted webhooks, full dashboard UI rebuild, white-label branding, a genuinely live hosted instance — see
CHANGELOG.mdfor the full listWhitePact migration (
1.2.0→1.2.2) — governance decision core, MCP Streamable HTTP + OAuth/OIDC, risk tiering + policy engine, hash-chained evidence, approval workflow, multi-approver quorum + delegation chains, upstream MCP tool discovery, MCP trust/supply-chain scanner, HA Helm deployment, supply chain security (SBOM/provenance), release engineering, open source governance, live listings on the official MCP Registry and Smithery — seeMIGRATION_WHITEPACT_V2.mdfor the full phase-by-phase log and what's still not donev2.0 onward — see
VERSION_ROADMAP.mdfor the phase-by-phase plan through v6.0Strategic direction —
GAME_CHANGER_STRATEGY.mdlays out an infrastructure-first bet (free public trust registry, an agent-native trust-check primitive, AI-answer-engine citability) as an alternative to the enterprise-SaaS path, withGAME_CHANGER_BUILD_PLAN.mdbreaking it into concrete engineering phases against the current codebase
Security & Open Source Assurance
The official OpenSSF/OSPS BadgeApp project
currently records OpenSSF Best Practices Silver and OSPS Baseline Level 1.
They are voluntary project evidence, not an independent audit, penetration test, SOC 2,
or ISO certification. Current technical and claim boundaries are maintained in
WHITEPACT_TRUST_STATUS.md and
PUBLIC_TRUST_CLAIMS.md.
Release consumers can review the signed-tag evidence,
release process, security policy,
SLSA evidence boundary, and
consumer verification guide. The reusable trusted-builder
pipeline is present on main. Release v1.2.6 completed that path: its wheel and sdist
were reproduced, hashed, attested, independently verified in the publish job, published
to PyPI without rebuilding, hash-matched to PyPI, and attached to the GitHub Release with
the CycloneDX SBOM. Independent consumer verification was repeated on 2026-08-31. The
release-specific evidence is assessed as satisfying SLSA v1.2 Build L3; SLSA is a
conformance framework, not a certification or a guarantee that an artifact is secure.
Further reading
SPEC.md— the current architecture contractMACHINE_AUTHORITY_PROBLEM.md— the problem the v3 authority-layer work answersMACHINE_AUTHORITY_V1.md— inventory of the eight core machine-authority invariants (Delegation Graph, Autonomy Budget, Memory Firewall, Evidence Bundle, and more)ENFORCEMENT_BOUNDARY.md— precisely where each invariant's authority stops: inline enforcement vs. voluntary chokepointLEGACY_TO_MACHINE_AUTHORITY_MAP.md— mapping RBAC/OAuth/IAM concepts onto their WhitePact equivalents, for readers coming from traditional access controlMIGRATION_WHITEPACT_V2.md— phase-by-phase migration log, what's done and what's explicitly notDEFINITION_OF_DONE.md— closing report: what's real today, what isn't, verifiableSECURITY_THREAT_MODEL.md— current security threat and attack-surface modelDETERMINISTIC_VS_PROBABILISTIC.md— why governance decisions are deterministicSLA.md,ENTERPRISE_SECURITY.md,SECURITY.md— enterprise/security posture, stated honestlycompliance/SOC2_ALTERNATIVE_PATH.md— real, free, independently verifiable trust signals for now; the honest path to a real SOC 2 when there's budget for onedocs/ACCESSIBILITY.md,docs/INTERNATIONALIZATION.md— WCAG2AA accessibility approach and the dashboard's i18n architecture, both with real automated CI gatescompliance/PROJECT_CONTINUITY_PLAN.md— the access/recovery checklist a second person would need if the founder became unavailable; stated honestly as a plan, not proof of bus-factor redundancy (no second person holds this access yet)
License
MIT — see LICENSE.
Available Tools
30 toolsrai_audit_summaryGet Governance Capability SummaryARead-onlyIdempotent
Return a governance capability summary including supported tools, frameworks, and available attack vectors. Full audit log access requires the REST endpoint.
| Name | Required | Description | Default |
|---|---|---|---|
| days | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds real value beyond that by disclosing a scope limitation: the summary is not the full audit log and that requires the REST endpoint. It still omits whether the result is scoped by the 'days' window or what is excluded from the summary.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, with the core purpose front-loaded and the limitation placed second. Every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read tool with no output schema the description is largely adequate, and the REST-endpoint caveat usefully signals partial coverage. The gap is that the only parameter's meaning and effect are unexplained, which is exactly what an agent needs to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the single 'days' parameter, so the description carries the full explanatory burden and does not meet it. It never mentions the parameter, the default of 7, or whether days narrows the audit window or the capability list, leaving an agent to guess from the name alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Return a governance capability summary') and enumerates the content returned: supported tools, frameworks, and attack vectors. It is not, however, differentiated from near-siblings like rai_executive_summary or rai_org_status, which an agent could plausibly choose instead.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The sentence about full audit log access requiring the REST endpoint is a useful boundary (when this tool is not sufficient), but the description never says when to prefer this summary over rai_executive_summary or rai_org_status, nor names any alternative. Usage is implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_benchmarkEvaluate Responses Against BenchmarkARead-onlyIdempotent
Evaluate pre-collected model responses against a standard benchmark suite. Suites: truthfulqa (factual accuracy), bbq (bias in questions), hellaswag (reasoning). Call rai_benchmark_prompts first to get the question set, collect responses, then pass them here.
| Name | Required | Description | Default |
|---|---|---|---|
| suite | No | truthfulqa | |
| provider | Yes | ||
| responses | Yes | Map of sample_id → model response text | |
| model_name | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive and closed-world, so safety is covered. The description adds the non-obvious behavioral requirement that responses must be collected externally before invocation, which is real context beyond the annotations. Return-format behavior is left unstated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the action, then suite meanings, then the workflow. No filler and every clause carries usable information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description cannot lean on structure to explain results, and it does not describe what an evaluation returns (scores, per-suite metrics). However, the suite semantics and the end-to-end workflow make the call path unambiguous for a nested-input evaluation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 25%, so the description must compensate. It partially does by glossing each enum suite value (truthfulqa=factual accuracy, bbq=bias, hellaswag=reasoning), but 'provider' and 'model_name' get no explanation. The responses map semantics live only in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Evaluate pre-collected model responses against a standard benchmark suite') and enumerates the three supported suites with a one-word meaning for each. It also distinguishes itself from the sibling rai_benchmark_prompts by role in the workflow (prompts generate questions, this one scores answers).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit ordering: 'Call rai_benchmark_prompts first to get the question set, collect responses, then pass them here.' That names the prerequisite tool and the condition under which this tool is appropriate. It stops short of saying when NOT to use it or which suite to pick for a given goal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_benchmark_promptsGet Benchmark Question SetARead-onlyIdempotent
Return the question set for a benchmark suite. Use to collect model responses before calling rai_benchmark. Suites: truthfulqa, bbq, hellaswag.
| Name | Required | Description | Default |
|---|---|---|---|
| suite | No | truthfulqa |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and closed-world behavior, so the safety profile is covered. The description adds the pipeline position (call before rai_benchmark), which is genuinely useful workflow context, but says nothing about the size or shape of the returned set.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, nothing wasted, with the core action front-loaded and the alternative/sequencing information following immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-enum getter with no output schema, the description covers purpose, workflow placement, and valid suites. It does not describe the return payload (e.g., question format or count), which is the only remaining gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter exists and schema description coverage is 0%, but the parameter is an enum whose values are self-explanatory. The description restates the enum values ('truthfulqa, bbq, hellaswag') but does not explain the default or what each suite measures, so it adds marginal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Return the question set for a benchmark suite') and explicitly positions itself relative to the sibling rai_benchmark, so an agent can distinguish gathering prompts from running the suite. No ambiguity about what is returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to use it to collect model responses before calling rai_benchmark, giving clear workflow context and sequencing. It lacks explicit exclusions (e.g., when not to use it), which keeps it short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_bias_evaluateEvaluate Demographic BiasARead-onlyIdempotent
Evaluate demographic bias across six probe dimensions: gender, racial, age, religious, occupational, and cultural. Provide paired response samples for each demographic group. Returns per-probe bias scores (0=no bias, 1=maximum divergence), confidence intervals, intersectional amplification, and an overall bias grade.
| Name | Required | Description | Default |
|---|---|---|---|
| provider | Yes | Model provider | |
| threshold | No | Bias score above this value triggers a FAIL | |
| model_name | Yes | Model under evaluation | |
| probe_responses | Yes | Map of probe_name → list of response texts from different demographic groups. Each list must have at least 2 responses to compute divergence. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, non-destructive), so the bar is lower. The description adds meaningful behavioral detail by disclosing the output structure (per-probe scores with a 0-1 scale, confidence intervals, intersectional amplification, overall grade), which matters since no output schema exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the verb and the six dimensions, then the input expectation, then the outputs. No wasted phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite using a nested input object, the definition covers purpose, required inputs, and return values (compensating for the absent output schema), and annotations carry the safety profile. Only the lack of explicit sibling routing keeps it short of complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all four parameters. The description reinforces the probe_responses concept with 'paired response samples,' but adds little syntax or format beyond what the schema provides; baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Evaluate) and resource (demographic bias) and enumerates the six probe dimensions it covers (gender, racial, age, religious, occupational, cultural). This is distinctive enough among the rai_ siblings that an agent can identify it without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The instruction to 'provide paired response samples for each demographic group' implies how to prepare input, but there is no explicit when-to-use guidance, no when-not-to, and no routing to alternatives like rai_redteam_analyze or rai_scan. Usage is implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_budget_checkCheck AI Spending Against BudgetBRead-onlyIdempotent
Evaluate current AI spending against monthly budget limits. Returns consumption percentage, alert status, per-team and per-model breakdown, and projected month-end spend. Used by LLMOps Engineers and Finance to prevent budget overruns.
| Name | Required | Description | Default |
|---|---|---|---|
| days_elapsed | No | ||
| days_in_month | No | ||
| team_breakdown | No | team_name → USD spent | |
| model_breakdown | No | model_name → USD spent | |
| total_spent_usd | Yes | ||
| monthly_limit_usd | No | ||
| alert_threshold_pct | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only, idempotent, non-destructive behavior, so the bar is lower. The description usefully discloses the computation outputs (consumption %, alert status, projections), which matters because there is no output schema, but it never clarifies that the caller must supply the raw spend figures rather than the tool fetching them—a meaningful behavioral gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with the core action and then the return values. The audience clause is the only slightly expendable part, but nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description's enumeration of returned metrics is genuinely necessary and present. However, the input-side story is incomplete: seven parameters, undocumented semantics for alert_threshold_pct and the day-count pair, and no note about required spend inputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 29% and the description explains none of the seven parameters. In particular, alert_threshold_pct (default 0.8, max 1) and monthly_limit_usd are ambiguous—fraction vs. percentage, default limit source—and the description does nothing to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Evaluate current AI spending against monthly budget limits', and enumerates the computed outputs. It is clear what the tool does, though it does not distinguish itself from nearby siblings like rai_cost_estimate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Used by LLMOps Engineers and Finance to prevent budget overruns' implies a monitoring use case, but there is no explicit when-to-use, no prerequisites, and no named alternative (e.g. rai_cost_estimate) for estimating rather than checking spend.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_causal_influence_checkCheck Causal Influence Provenance for Injection PatternsARead-onlyIdempotent
Scan a list of upstream sources that causally shaped a proposed action (a prior tool's output, a sub-agent's result, an external document, ...) for prompt-injection patterns, and flag whether any of them are untrusted. Generalizes rai_memory_write_check beyond persistent memory: any content that will be treated as trusted context by whatever consumes this action's result carries the same replay risk memory does. Each provenance entry needs a 'kind' (memory_read | tool_output | sub_agent_result | user_input | external_content) and a 'trust' level (TRUSTED | UNTRUSTED | UNKNOWN); 'content' is optional (an entry may assert only its trust level with nothing to scan). Call this before letting a matched/untrusted source influence a real action -- when governance is enabled on the hosted MCP server, the same check also runs automatically on any governed action whose arguments carry a '_provenance' key in this same shape.
| Name | Required | Description | Default |
|---|---|---|---|
| provenance | Yes | Upstream sources that shaped the action being considered. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, non-destructive), yet the description adds real behavioral context: what the check does, that it can run automatically under hosted governance, that content is optional so an entry may assert trust with nothing to scan, and why the risk applies to any trusted-context consumer. This goes well beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense paragraph, but it is front-loaded with the purpose before the generalization, parameter detail, and invocation timing. Every clause carries information; the only minor cost is run-on length rather than waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with annotations covering safety and no output schema, the description supplies everything needed: what it scans, the entry shape, when to call it, and the automatic governance path. Nothing material is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds meaning: it explains the required 'kind' and 'trust' fields, enumerates their values, and clarifies that 'content' is optional with the rationale (an entry may carry only a trust assertion). That added interpretation exceeds the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Scan') and resource (a list of upstream sources that causally shaped a proposed action) and names the outcome ('flag whether any of them are untrusted'). It explicitly positions itself against sibling rai_memory_write_check by generalizing beyond persistent memory, so an agent can distinguish it without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear trigger: 'Call this before letting a matched/untrusted source influence a real action.' It also notes the alternative execution path (automatic check under governance when arguments carry a '_provenance' key). There is no explicit when-not-to-use guidance, keeping it just below full marks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_check_trustCheck Public Trust Index ScoreARead-onlyIdempotent
Check the public, independently-verifiable Trust Index score, certification status, and reported-incident history for a named AI model or tool BEFORE invoking it. Unlike every other rai_* tool, which evaluates output the caller itself produced, this one looks up a public record about a THIRD PARTY'S model or tool — built for agents and agent frameworks (LangChain, LangGraph, Google ADK) deciding whether to trust something before calling it. Free, no auth required, exact model+provider match. Queries the hosted ResponsibleAI Trust Index (configurable via the RAI_TRUST_API_BASE environment variable). Returns 'known: false' for anything never assessed — that is not an error, just an absence of data; self-assessment is free at POST /api/trust-index/assess.
| Name | Required | Description | Default |
|---|---|---|---|
| provider | Yes | Exact provider name, e.g. 'openai' | |
| min_score | No | Minimum acceptable overall trust score (0-100). The response's 'passes' field reflects this threshold. | |
| model_name | Yes | Exact model or tool name, e.g. 'gpt-4o' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safety profile (readOnly, idempotent, non-destructive, closed-world), and the description adds substantial context beyond them: it is free, requires no auth, uses exact model+provider matching, queries a configurable hosted base (RAI_TRUST_API_BASE), and returns 'known: false' for unassessed items as a non-error absence of data. It even points to the self-assessment endpoint. This is rich, agent-relevant behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and the key sibling distinction are front-loaded, and subsequent sentences carry distinct value (audience, auth/free status, edge-case 'known: false'). It is on the longer side and slightly repetitive, but nearly every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so adequately: it names the returned items (score, certification status, incident history) and the 'known: false' behavior. It stops short of describing the exact response shape or the 'passes' field mechanics, but is sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents model_name, provider, and min_score (including that 'passes' reflects the threshold). The description reinforces the 'exact model+provider match' requirement, which adds slight emphasis but no new semantics beyond the schema's 'Exact provider/model name' text. Baseline 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (checks the public Trust Index score, certification status, and incident history) for a named model/tool. It explicitly differentiates itself from siblings: 'Unlike every other rai_* tool, which evaluates output the caller itself produced, this one looks up a public record about a THIRD PARTY'S model or tool.' An agent can distinguish it from rai_trust_score, rai_compliance, etc. without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly frames when to use it ('BEFORE invoking it') and for whom (agents/agent frameworks deciding whether to trust something before calling it). The contrast with all other rai_* tools implicitly signals when NOT to use it, but it does not name a specific alternative tool for the 'assess your own output' case.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_compare_modelsCompare Two Models' Trust ScoresBRead-onlyIdempotent
Compare two AI models across all six trust dimensions. Returns scores for each, delta analysis, and a recommendation on which model is more trustworthy.
| Name | Required | Description | Default |
|---|---|---|---|
| model_a | Yes | ||
| model_b | Yes | ||
| scores_a | No | ||
| scores_b | No | ||
| provider_a | Yes | ||
| provider_b | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description usefully supplements by disclosing the return shape (per-model scores, delta analysis, recommendation), which matters since there is no output schema. It says nothing about auth requirements or where the scores come from when scores_a/scores_b aren't supplied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tidy sentences, front-loaded with the operation and followed by the return content. No filler, though it could have spent one of those sentences on parameter guidance instead of repeating the count of dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description's summary of returns is genuinely necessary and present. However, for a six-parameter tool with nested objects, 0% schema coverage, and four required inputs, the omission of any input semantics leaves the definition only partially complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across six parameters, so the description carries the full burden and falls short. It hints at 'six trust dimensions' but never clarifies the distinction between the required model_a/provider_a/model_b/provider_b and the optional nested scores_a/scores_b objects, nor what happens if scores are omitted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource (compare two AI models) and scopes it to 'all six trust dimensions', which is concrete. It does not explicitly distinguish itself from the single-model sibling rai_trust_score, but the two-model framing is self-evident from the name and title.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance is given, and no alternative is named despite several closely related siblings (rai_trust_score, rai_benchmark, rai_model_route). The comparison purpose is implied by 'compare two models' but the agent is left to infer when this is the right tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_complianceEvaluate Governance ComplianceARead-onlyIdempotent
Evaluate AI governance maturity against NIST AI RMF, EU AI Act, or ISO 42001, given your own self-assessed control scores (fairness/privacy/security/robustness 0-1, plus overall compliance maturity 0-1). Returns a compliance score, findings per control, and remediation recommendations. Use this when the caller already has maturity/control scores and wants a gap assessment. Do NOT use this to classify what EU AI Act risk tier a specific system falls into from a description of what it does (sector, automation, biometric use, etc.) -- use rai_eu_ai_act_classify instead for that; this tool has no equivalent inputs (no deployment sector, no system description) and cannot answer that question.
| Name | Required | Description | Default |
|---|---|---|---|
| use_case | No | general | |
| framework | No | NIST_AI_RMF | |
| privacy_score | No | ||
| fairness_score | No | ||
| security_score | No | ||
| robustness_score | No | ||
| compliance_maturity | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered. The description adds genuinely new behavioral context beyond them: it names the return payload (compliance score, per-control findings, remediation recommendations) and the precondition that the caller must supply self-assessed scores. No output schema exists, so this disclosure is valuable. It stops short of mentioning scoring determinism or how defaults (all 0.5) are treated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose and return values, then the routing rule and exclusion. The negative-guidance sentence is long, but every clause earns its place by fending off a specific misrouting. Slightly verbose overall but no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only scoring tool with no output schema, the description supplies the returns, the input precondition, and the anti-pattern that would cause misuse. The remaining gap is the undocumented 'use_case' parameter and default behavior, which is the only thing an agent could still get wrong.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 7 parameters, so the description must carry the burden. It does explain the five numeric control scores and their 0-1 range, and implies the framework values, but leaves 'use_case' entirely undefined and does not clarify that all scores default to 0.5 or what the framework choice changes in the output. Partial compensation only.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Evaluate AI governance maturity') and names the exact frameworks it scores against (NIST AI RMF, EU AI Act, ISO 42001). It explicitly differentiates itself from the closest sibling, rai_eu_ai_act_classify, so an agent can route without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides both the positive trigger ('when the caller already has maturity/control scores and wants a gap assessment') and an explicit exclusion with the alternative named ('Do NOT use this to classify what EU AI Act risk tier... use rai_eu_ai_act_classify instead'). It even explains why the alternative is needed (no deployment sector, no system description inputs).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_cost_estimateEstimate Model API CostCRead-onlyIdempotent
Estimate the USD cost of a model API call from token counts.
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | Model name, e.g. gpt-4o | |
| provider | Yes | Provider: openai | anthropic | google | mistral | |
| input_tokens | Yes | ||
| output_tokens | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds only that the output is expressed in USD and derived from token counts; it says nothing about pricing-data staleness, whether it is a pure computation, or what form the estimate takes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler, which is good, but it is under-specified rather than genuinely concise — there is simply not enough content to be wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Four required parameters, 50% schema coverage, and no output schema. The description never says what the tool returns (a bare number, a currency-formatted string, a breakdown per token type), which for a cost-estimation tool an agent genuinely needs in order to use the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50% — input_tokens and output_tokens have no schema description at all. The phrase 'from token counts' weakly gestures at them but adds no units, granularity, or counting convention, so the description does not compensate for the coverage hole.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (estimate) plus resource (USD cost of a model API call) and the inputs it derives them from (token counts). It is clear on its own, but it does not differentiate itself from siblings like rai_compare_models or rai_budget_check, which touch adjacent cost/model concerns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers no when-to-use guidance, no alternatives, and no prerequisites beyond the implicit 'you must have token counts'. With ~30 sibling tools in this family, the absence of any routing hint is a real gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_drift_checkCheck Trust Score DriftARead-onlyIdempotent
Detect trust score drift between a baseline evaluation and a current evaluation. Returns drift delta per dimension, overall drift severity (NONE/LOW/MEDIUM/HIGH/CRITICAL), and whether an alert threshold was breached.
| Name | Required | Description | Default |
|---|---|---|---|
| provider | Yes | ||
| model_name | Yes | ||
| current_score | Yes | Current trust dimension scores (0-1 each) | |
| baseline_score | Yes | Previous trust dimension scores (0-1 each) | |
| alert_threshold | No | Overall score drop (0-100 scale) that triggers an alert |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already disclose read-only, idempotent, non-destructive behavior. The description adds meaningful behavioral detail beyond that: it returns per-dimension drift deltas, a five-level severity value (NONE/LOW/MEDIUM/HIGH/CRITICAL), and an alert-threshold breach indicator. It does not cover auth or rate limits, but those are minor for this read-only analysis tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the purpose, the second states the return values. It is front-loaded, has no filler, and every clause contributes useful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only tool with no output schema, the description explains the core purpose and the main return fields, including severity levels and threshold breach detection. It remains silent on when to choose it over sibling trust tools and on the two parameters lacking schema descriptions, which keeps it from a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60%: baseline_score, current_score, and alert_threshold have schema descriptions, while provider and model_name do not. The description reinforces the baseline/current comparison and alert-threshold concept but adds no semantics for the two undocumented parameters. A 3 is appropriate given the moderate schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb (detect) and resource (trust score drift), and scopes it to a baseline evaluation versus a current evaluation. It also names the return outputs, so the purpose is clear. It does not, however, explicitly differentiate itself from sibling tools such as rai_check_trust or rai_trust_score, which keeps it from a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The baseline/current framing implies when the tool is useful, namely when comparing two trust-score evaluations. There is no explicit when-to-use, when-not-to-use, or named alternative, so an agent must infer that this is preferable to rai_trust_score or rai_check_trust for drift analysis.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_eu_ai_act_classifyClassify EU AI Act Risk TierARead-onlyIdempotent
Classify an AI system into an EU AI Act risk tier: UNACCEPTABLE, HIGH, LIMITED, or MINIMAL, from a description of what the system does (deployment sector, automation level, biometric/emotion-recognition use). Evaluates deployment context, capabilities, and affected populations against Annex III and Annex VI criteria. Returns risk tier, applicable articles, required conformity assessment actions, and a compliance roadmap. Use this for 'what EU AI Act category does this system fall into' questions. Do NOT use this for a general maturity/gap assessment against self-scored controls (fairness/privacy/security scores) -- use rai_compliance with framework=EU_AI_ACT instead for that.
| Name | Required | Description | Default |
|---|---|---|---|
| deployment_sector | Yes | ||
| is_fully_automated | No | ||
| system_description | Yes | Description of the AI system and its purpose | |
| trust_score_overall | No | ||
| social_scoring_purpose | No | ||
| affects_natural_persons | No | ||
| processes_biometric_data | No | ||
| real_time_remote_biometric | No | ||
| used_for_emotion_recognition | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive and closed-world, so the safety profile is covered. The description adds meaningful behavior beyond that: it discloses the evaluation basis (Annex III/VI, deployment context, capabilities, affected populations) and the return payload (tier, applicable articles, conformity assessment actions, compliance roadmap).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and outputs, then usage guidance and an explicit exclusion. Every sentence carries distinct information (purpose, criteria, returns, routing), with no redundant restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a classification tool with no output schema, the description usefully summarizes what is returned (tier, articles, conformity actions, roadmap) and the evaluation criteria. It is nearly complete, with the only gap being coverage of the several undocumented input parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 11% (one documented property of nine), so the description must compensate. It hints at deployment_sector, is_fully_automated, and the biometric/emotion flags, but omits others such as trust_score_overall, social_scoring_purpose, and affects_natural_persons, leaving significant parameter meaning unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (classify) and resource (AI system into an EU AI Act risk tier), enumerates the four possible outcomes, and specifies the input basis (deployment sector, automation level, biometric/emotion-recognition use). It also names the criteria used (Annex III and Annex VI), so an agent knows exactly what this tool produces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes: 'Use this for what EU AI Act category does this system fall into questions' and 'Do NOT use this for a general maturity/gap assessment... use `rai_compliance` with framework=EU_AI_ACT instead.' This gives both the trigger condition and the named alternative for the excluded case.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_executive_summaryGenerate Executive Governance SummaryARead-onlyIdempotent
Generate a board-ready executive AI governance summary. Synthesises trust grades, compliance posture, cost intelligence, risk incidents, and drift trends into a C-suite-readable report with RAG (Red/Amber/Green) status indicators. Used by CAIO for quarterly board reporting.
| Name | Required | Description | Default |
|---|---|---|---|
| org_name | No | Organisation | |
| top_risks | No | Top identified AI risks for executive attention | |
| frameworks | No | Active compliance frameworks | |
| drift_alerts | No | ||
| bias_failures | No | ||
| report_period | No | Q2 2026 | |
| open_incidents | No | ||
| total_cost_usd | No | ||
| avg_trust_score | No | ||
| compliance_score | No | ||
| models_evaluated | No | ||
| monthly_budget_usd | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is covered. The description adds real value beyond them: it reveals this is a multi-source aggregation/synthesis step rather than a single-source read, and it describes the output shape (RAG status indicators), which matters since no output schema exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with the core action front-loaded and no filler. Minor redundancy between 'board-ready' and 'C-suite-readable', but otherwise every sentence carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter, no-required-argument tool with no output schema, the description does explain the report contents, which partially substitutes for a return-value spec. It leaves the biggest gap unaddressed: whether the caller must supply the metrics or the tool derives them, and what the 10 undocumented numeric parameters mean.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 17% (2 of 12 parameters documented), so the description is expected to compensate — and it does not mention a single parameter. Names like avg_trust_score, compliance_score, monthly_budget_usd and models_evaluated are left entirely to inference, with no indication of units, ownership, or whether they are pulled automatically versus supplied by the caller.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Generate), a specific resource (executive AI governance summary), and enumerates the inputs it synthesises (trust grades, compliance posture, cost intelligence, risk incidents, drift trends). The C-suite/board framing implicitly separates it from technical siblings like rai_audit_summary or rai_org_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear situational context — 'Used by CAIO for quarterly board reporting' — which tells the agent the audience and cadence. It does not, however, name an alternative sibling or state when NOT to use it (e.g. ad-hoc technical reporting via rai_audit_summary).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_hallucinationDetect Hallucination RiskARead-onlyIdempotent
Detect hallucination risk in AI-generated text. Analyses hedging language, self-consistency across candidate responses, unsupported factual claims, and (when a source is supplied) explicit factual disagreement with that source -- e.g. the source names one day/month/number and the response names another.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | AI-generated text to analyse | |
| source | No | Optional ground-truth or reference text the response should be consistent with -- enables explicit factual-disagreement detection. | |
| candidates | No | Optional additional responses for consistency scoring |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint, idempotentHint, destructiveHint=false), so the bar is lower, and the description adds real behavioral detail: the four analysis dimensions and the conditional behaviour that factual-disagreement detection only activates when `source` is present. It stops short of describing output shape, confidence levels or scoring thresholds, so it is not a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence leads with the core purpose before the enumeration, and the parenthetical example is the only elaboration, placed where it disambiguates `source`. Nothing is redundant with the title or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter analysis tool with no output schema, the definition explains inputs well but says nothing about what comes back -- risk score, label, per-signal breakdown -- and gives no hint about candidate count expectations or latency/cost. Annotations cover safety, so the gap is in return-value and operational context rather than in purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description goes beyond it by giving a concrete semantics example for `source` ('the source names one day/month/number and the response names another') and by tying `candidates` to self-consistency scoring. That clarifies the optional parameters' purpose rather than merely restating their schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Detect hallucination risk in AI-generated text') and then enumerates the four signals it analyses (hedging language, self-consistency across candidates, unsupported factual claims, source disagreement). That mechanism list uniquely identifies this tool against siblings like rai_scan or rai_trust_score without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the description shows the conditional value of supplying a `source` ('when a `source` is supplied ... enables explicit factual-disagreement detection'), which tells the agent when that parameter matters. There is no tool-level when-to-use guidance, no mention of when to prefer rai_scan, rai_trust_score or rai_redteam_analyze, and no stated prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_healthCheck Governance Engine HealthBRead-onlyIdempotent
Check the status and module availability of the ResponsibleAI governance engine.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this as a safe, idempotent, non-destructive, closed-world read. The description adds only the scope hint of 'module availability,' and gives no sense of what a degraded/unhealthy result implies or how it behaves if a module is down.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; every word (status, module availability, governance engine) carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should ideally say what the health result contains (overall status, per-module list, error detail). 'Status and module availability' only partially covers the return contract.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, so per the rubric the baseline is 4; there is nothing for the description to disambiguate beyond the schema's empty object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb ('Check') plus concrete resource ('status and module availability of the ResponsibleAI governance engine'), which is clearly a health/readiness probe. It does not, however, distinguish itself from adjacent status-oriented siblings such as rai_org_status or rai_webhook_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of alternatives. The agent must infer that this is a pre-flight/availability check rather than something to run against a specific model or policy.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_incident_logBuild Governance Incident RecordCRead-onlyIdempotent
Create a structured governance incident record. Used by Security Engineers and AI Risk Analysts to log AI safety events (PII leaks, jailbreak attempts, bias triggers, hallucination incidents) for audit trail and SIEM integration.
| Name | Required | Description | Default |
|---|---|---|---|
| evidence | No | Supporting data: prompt, response, scan results, etc. | |
| provider | No | ||
| severity | Yes | ||
| mitigated | No | ||
| model_name | No | ||
| description | Yes | Human-readable incident description | |
| incident_type | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly says 'Create' and 'log ... events', i.e. a write/mutation, while annotations declare readOnlyHint=true and idempotentHint=true. This directly contradicts the structured safety metadata. The added context about audit trail and SIEM integration is useful, but the write-vs-read conflict is disqualifying.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the core action front-loaded and no filler. Efficient, though it spends words on audience role labels rather than information an agent needs to invoke the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter, nested-object tool with no output schema and only 29% schema coverage, the description omits severity semantics, evidence schema, and return behavior. Combined with the annotation conflict, an agent lacks enough to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 29% across 7 parameters, so the description should compensate, but it only maps loosely to incident_type by naming event categories. It says nothing about severity levels, mitigated, evidence structure, provider, or model_name, leaving most required and optional fields undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Create a structured governance incident record') and enumerates the event classes it captures (PII leaks, jailbreak attempts, bias triggers, hallucinations). Clear on its own, but it names no sibling tools, so an agent gets no help distinguishing it from rai_pii_report, rai_compliance, or rai_audit_summary in a 30-tool family.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives the intended audience (Security Engineers, AI Risk Analysts) and the purpose (logging AI safety events for audit/SIEM), which implies usage context. However, it never says when to use this instead of the closely related reporting siblings, nor what prerequisites or thresholds trigger a log entry.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_iso42001_gapRun ISO 42001 Gap AnalysisCRead-onlyIdempotent
Perform an ISO/IEC 42001:2023 AI Management System gap analysis. Evaluates maturity across all 10 clauses: Context, Leadership, Planning, Support, Operation, Performance Evaluation, Improvement, plus AI-specific annexes. Returns gap findings, maturity scores per clause, and a prioritised remediation roadmap. Used by AI Compliance Managers.
| Name | Required | Description | Default |
|---|---|---|---|
| org_name | No | Organisation | |
| has_ai_policy | No | ||
| has_audit_trail | No | ||
| compliance_maturity | No | ||
| has_data_governance | No | ||
| has_risk_assessment | No | ||
| trust_score_overall | No | ||
| has_incident_process | No | ||
| has_impact_assessment | No | ||
| has_supplier_controls | No | ||
| has_monitoring_metrics | No | ||
| has_training_programme | No | ||
| has_continual_improvement | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint and non-destructive behavior, so the safety profile is covered. The description adds that scoring is per-clause maturity and that a prioritised remediation roadmap is returned, which is useful, but it omits that all 13 inputs are optional/defaulted (zero-param invocation is possible) and says nothing about determinism or data requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the operation and standard, then scope, then outputs in a tight three-sentence block. The trailing audience sentence adds little, but overall it is efficiently sized with no padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 13-parameter tool with no output schema and no parameter documentation, the description is too thin: it never explains how the boolean control flags or maturity/trust numbers drive the analysis, nor the shape of the findings beyond 'gap findings, maturity scores, roadmap'. An agent has insufficient detail to supply meaningful inputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 13 parameters, so the description carries the full burden of explaining org_name, the ten boolean control flags, compliance_maturity, and trust_score_overall. It mentions none of them, leaving an agent to infer each flag's meaning entirely from its name and default value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (gap analysis) and resource (ISO/IEC 42001:2023 AI Management System), which cleanly separates it from generic siblings like rai_compliance and rai_eu_ai_act_classify. However, it claims evaluation 'across all 10 clauses' while only listing 7 (Context through Improvement), a factual inconsistency that slightly muddies the scope claim.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives an audience ('Used by AI Compliance Managers') but no when-to-use guidance, no prerequisites, and no comparison to adjacent tools such as rai_compliance, rai_eu_ai_act_classify, or rai_policy_check. An agent cannot tell from this text when ISO 42001 gap analysis is the right call versus a generic compliance check.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_memory_read_checkCheck Memory Read Scope AuthorizationARead-onlyIdempotent
Gate a read from persistent agent memory. Standalone, this tool only echoes back the requested scope (there's no content to scan for a read) -- its real enforcement value is when governance is enabled on the hosted MCP server: the caller's authority's memory_scope constraint (if any) is checked against the requested memory_scope, denying cross-tenant/cross-agent memory access before the read happens.
| Name | Required | Description | Default |
|---|---|---|---|
| memory_scope | Yes | The memory namespace this read targets, e.g. 'org:acme:agent:bot1'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnly/idempotent annotations by disclosing a critical behavioral trait: standalone the tool only echoes the requested scope, and real enforcement (denying cross-tenant/cross-agent access) occurs only under hosted governance. This is exactly the kind of non-obvious behavior an agent must know before invoking it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the purpose and then the caveat in one dense but well-structured statement with zero filler. The em-dash aside is slightly awkward but every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter gate with no output schema and full annotation coverage, the description supplies everything needed: what it does, when it is meaningful, and the enforcement outcome. Nothing an agent needs to call and interpret it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning: the requested memory_scope is compared against the caller's authority's memory_scope constraint, clarifying the semantics of the value beyond the schema's example string.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Gate a read from persistent agent memory') and clearly separates itself from the read path. The name and description make its role as a pre-read authorization gate unambiguous, and the sibling rai_memory_write_check is implicitly distinguished by the 'read' framing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly explains the two contexts: standalone it is a no-op that echoes the scope, and it only enforces when governance is enabled on the hosted MCP server. This tells the agent when the tool matters versus when calling it is pointless. It stops short of naming the sibling tool or stating exclusions directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_memory_write_checkCheck Memory Write for Injection PatternsARead-onlyIdempotent
Gate a write to PERSISTENT agent memory (a vector DB, conversation log, or similar long-term store your own system owns -- WhitePact does not host a memory store itself). Scans the content for prompt-injection patterns aimed specifically at persistent memory -- text engineered to look like an instruction, a fake system/assistant role marker, or an override, so that a future session reading this memory back treats it as trusted context rather than as content. Call this BEFORE actually persisting the write; if 'allowed' is false, do not write it. When governance is enabled on the hosted MCP server and the caller's authority carries a memory_scope constraint, memory_scope is also checked for cross-tenant/cross-agent isolation -- outside the governed dispatch path this tool only runs the content scan.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | The text about to be written to memory. | |
| memory_scope | No | The memory namespace this write targets, e.g. 'org:acme:agent:bot1'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this a safe, idempotent, non-destructive read (consistent with a scan that does not itself persist), so the bar is lower. The description adds real value beyond that: what patterns are targeted (fake role markers, overrides, instruction-like text), why (future sessions re-trusting the memory), and the conditional governance behavior of memory_scope. It could still be clearer on the exact return shape beyond the 'allowed' flag.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The gating purpose and the pre-write timing are front-loaded in the first sentence. The paragraph is dense but long, with some parenthetical asides (the WhitePact disclaimer) that could be tightened while still earning their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only 2 well-documented params and no output schema, the description covers the essential behavior, timing, and failure branch. The one remaining gap is the full shape of the result object, though it does name the 'allowed' field that drives the caller's decision.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining that memory_scope is a namespaced target checked for cross-tenant/cross-agent isolation only under governance, adding semantic meaning the schema field alone does not convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb (gate/scan) and resource (a write to persistent agent memory), and explicitly limits the scope to persistent long-term stores rather than a general content filter. It is easily distinguished from the sibling rai_memory_read_check by the 'write' orientation and the pre-persist timing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an explicit call sequence ("Call this BEFORE actually persisting the write") and a concrete decision rule ("if 'allowed' is false, do not write it"). It also discloses that the memory_scope isolation check only fires under governed dispatch, allowing the agent to predict when the extra check applies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_model_routeRecommend Optimal ModelARead-onlyIdempotent
Recommend the optimal AI model for a task based on complexity analysis and cost-quality tradeoff. Returns recommended model, alternative, estimated cost per 1K tokens, and estimated savings vs GPT-4o. Used by LLMOps Engineers for intelligent model routing.
| Name | Required | Description | Default |
|---|---|---|---|
| tasks | No | Optional: batch route multiple task descriptions | |
| task_description | No | Natural language description of the task | |
| quality_requirement | No | maximum: best model always; balanced: cost-quality tradeoff; cheapest: minimize cost | balanced |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safe read-only, idempotent, non-destructive profile, so the bar is lower. The description adds real value by disclosing the response payload (recommended model, alternative, cost per 1K tokens, savings vs GPT-4o), which is otherwise undocumented because there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no filler, and the core purpose is front-loaded ahead of the return details and the audience note. The trailing LLMOps sentence is the only mildly expendable element.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter tool with full schema coverage and no output schema, the description covers purpose, basis, and expected return contents. The one unresolved point is that all parameters are optional, so the description never clarifies what happens on an empty call or which of task_description vs tasks is expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, including an enum with its meanings, so the schema carries the parameter burden and baseline 3 applies. The description adds no syntax, format, or interaction detail beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Recommend the optimal AI model") with the basis for the recommendation (complexity analysis, cost-quality tradeoff) and enumerates the output. It is clear in isolation, but it never distinguishes itself from nearby siblings like rai_compare_models or rai_cost_estimate, so an agent must infer the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Used by LLMOps Engineers for intelligent model routing" implies the intended context but gives no explicit when-to-use conditions, no prerequisites, and no named alternative. An agent gets a rough sense of fit but nothing that resolves the choice against rai_compare_models or rai_cost_estimate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_org_statusGet Organization Governance StatusARead-onlyIdempotent
Compute a structured governance status rollup. The health/grade/compliance fields (models, compliance, operations) are always derived FROM caller-supplied metrics (model grades, active frameworks, open incidents, budget usage, drift alerts) -- there is no separate store of these tracked per-org yet, so calling with none supplied rolls up empty/default values, not fabricated data. The org_id/plan/usage fields are different: on the hosted MCP transport with an authenticated caller, these reflect the real, live org record and this month's real call count against quota -- absent (not present in the response at all) on the self-hosted stdio transport, where there is no org account to look up.
| Name | Required | Description | Default |
|---|---|---|---|
| org_name | No | default | |
| drift_alerts | No | ||
| model_grades | No | model_name → grade (A/B/C/D/F) | |
| open_incidents | No | ||
| budget_pct_used | No | ||
| active_frameworks | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description discloses substantial non-obvious behavior: fields are derived, not stored, so empty input yields empty/default values rather than fabricated data; and org_id/plan/usage reflect a live org record only on the hosted transport and are absent on self-hosted stdio. This is exactly the transport- and data-provenance context an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded and every sentence carries a distinct, non-redundant fact (data provenance on one side, transport differences on the other). The two long sentences are dense but not padded; slightly more structure would aid scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does the work of explaining what the returned fields mean and when they will or won't be present across transports, which is the key completeness concern for this 6-parameter tool. It falls short only in not describing the shape/labels of the rollup that will be returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 17%, so the description must carry weight. It enumerates the conceptual inputs (model grades, active frameworks, open incidents, budget usage, drift alerts), covering most required-meaning parameters, but adds no format, range, or default detail and never addresses org_name, so it only partially compensates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Compute a structured governance status rollup" states a specific verb (compute) and resource (governance status rollup), clearly telling the agent what is produced. It does not, however, distinguish itself from closely related siblings like rai_health, rai_executive_summary, or rai_trust_score, leaving overlap ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by explaining that the health/grade/compliance fields are derived from caller-supplied metrics, nudging the agent to pass metrics in. But it names no alternative tools and gives no explicit 'use this when X, use Y instead' routing despite a crowded sibling set.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_passport_generateGenerate AI PassportBRead-onlyIdempotent
Generate a verifiable AI Passport for a model — a tamper-evident governance card containing trust scores, compliance status, bias summary, and a cryptographic verification hash. Used by Procurement/Legal for third-party AI vendor risk assessment.
| Name | Required | Description | Default |
|---|---|---|---|
| provider | Yes | ||
| use_case | No | general | |
| model_name | Yes | ||
| bias_summary | No | ||
| privacy_summary | No | ||
| security_summary | No | ||
| trust_dimensions | Yes | Trust dimension scores (0-1 each) | |
| compliance_summary | No | ||
| hallucination_summary | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the tamper-evident nature and cryptographic hash, which is useful behavioral context, but does not reconcile why a 'generate' tool is read-only, nor mention auth, persistence, or whether the passport is stored.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences with the artifact definition front-loaded and the audience appended. Little wasted text, though the second sentence is context rather than operational instruction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 9 parameters, 11% schema coverage, and no output schema, the description should do more. It partially covers output content (useful since no output schema exists) but leaves half the input parameters undocumented, so an agent cannot construct a full call confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 11%, so the description carries the burden of documenting parameters, and it largely fails. It references 'trust scores', 'compliance status', and 'bias summary' which loosely map to trust_dimensions/compliance_summary/bias_summary, but leaves model_name, provider, use_case, privacy_summary, security_summary, and hallucination_summary unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Generate') and resource ('AI Passport'), then enumerates the artifact's contents (trust scores, compliance status, bias summary, verification hash), so the agent knows exactly what this produces. It is fairly distinct from siblings like rai_trust_score or rai_compliance, but never names them, so it stops short of explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Used by Procurement/Legal for third-party AI vendor risk assessment' line implies a usage context, which is better than nothing. However, it gives no when-to-use/when-not guidance or alternative routing (e.g., versus rai_trust_score, rai_compliance, or rai_audit_summary), leaving the agent to infer selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_pii_reportGenerate PII Audit ReportARead-onlyIdempotent
Generate a detailed PII audit report for a document or corpus. Classifies findings by PII category (email, phone, SSN, credit card, IP, address), counts occurrences, computes a privacy risk score, and provides GDPR/CCPA remediation guidance. Used by Privacy Engineers for compliance evidence collection.
| Name | Required | Description | Default |
|---|---|---|---|
| texts | Yes | List of text documents to scan | |
| redact | No | Include redacted versions in report | |
| context | No | Context label: medical | financial | hr | legal | general | general |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish the safety profile (readOnlyHint, idempotentHint, destructiveHint=false, closed world), so the description does not need to restate them. It adds useful behavioral context by disclosing the analytical pipeline and report contents (classification, counts, risk score, remediation guidance), which is more than the annotations convey. It stops short of stating rate limits, input size caps, or data retention behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with zero filler, front-loaded on what the report is and what it contains. The trailing audience sentence is slightly less essential but still earns its place by anchoring the use case.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly compensates by describing the report's contents and compliance framing. Annotations cover safety and determinism. Remaining gaps — whether input texts are persisted, size/throughput limits, and exact exit/return structure — are minor for a three-parameter read-only tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters (texts, redact, context) are already fully documented in the schema. The description implies the scan input ('a document or corpus') and hints at redacted output, but it never explains the context parameter's enum-like label semantics (medical | financial | hr | legal | general) or how it affects classification. Baseline 3 is appropriate when the schema carries this burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb and resource ('Generate a detailed PII audit report') and then enumerates exactly what the report contains: PII category classification, occurrence counts, a privacy risk score, and GDPR/CCPA remediation guidance. This level of detail distinguishes it from generic siblings like rai_scan or rai_compliance without needing to name them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states a clear usage context ('Used by Privacy Engineers for compliance evidence collection'), which tells an agent when this tool is appropriate. However, it offers no explicit exclusions or named alternatives (e.g., when to prefer rai_scan or rai_compliance instead), so an agent must still infer the boundary.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_policy_checkCheck Text Against Governance PolicyARead-onlyIdempotent
Evaluate text or a model response against a governance policy. Checks for: prohibited topics, required disclaimers, output length limits, language restrictions, and custom keyword blocklist. Returns pass/fail per policy rule with remediation guidance.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to evaluate against policy | |
| policy | Yes | Governance policy configuration |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true and destructiveHint=false, so the safety profile is covered. The description adds genuinely useful non-annotation behavior: the check categories performed and the fact that output is pass/fail per rule with remediation guidance — important since there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: purpose first, then the rule inventory, then the return shape. No filler and nothing redundant with structured fields, and the most decision-relevant content is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema but a nested policy object, the description carries real weight and discharges most of it by stating the return shape and the rule set. The gap is the 'language restrictions' claim, which no schema property supports, and the absence of any note on how PII checking defaults (the schema sets require_pii_clean=true).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3 and the schema already documents every policy key. The description loosely mirrors those keys (prohibited topics, disclaimers, length limits, keyword blocklist) but adds no syntax, format or defaulting detail, and its mention of 'language restrictions' corresponds to no field in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb ('Evaluate') and resource ('text or a model response against a governance policy') and enumerates the exact rule classes checked, so an agent knows precisely what this tool produces. It does not, however, name or contrast itself with close siblings such as rai_scan or rai_compliance, leaving the agent to infer the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied by the object being evaluated ('text or a model response'), with no explicit when-to-use, when-not-to-use, or alternative named among the ~29 rai_* siblings. Given the crowded namespace (rai_scan, rai_compliance, rai_memory_write_check), a routing sentence would be expected.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_redteam_analyzeAnalyze Red Team ResponsesBRead-onlyIdempotent
Analyse model responses to red team attack payloads. Returns a security report with vulnerability findings, severity breakdown, and an overall security score.
| Name | Required | Description | Default |
|---|---|---|---|
| provider | Yes | ||
| responses | Yes | Map of attack_name → model_response_text | |
| model_name | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral value by stating the return shape: vulnerability findings, severity breakdown, and an overall security score, which matters because no output schema exists. It still omits any note on cost, latency, or whether the analysis is deterministic across runs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action and closed with the return contents; every clause carries information and there is no padding. It could be marginally tighter by combining the report contents, but it is appropriately sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly spends its budget describing the returned security report, which is the most important omission it could fill. However, for a three-parameter tool with a nested input object and no parameter documentation, the definition leaves the agent guessing about what provider and model_name should contain and where the responses are sourced from.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 33%, and the description adds no information about any of the three required parameters. The nested responses map is documented in the schema, but provider and model_name are unexplained in both places, so the description fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: analyse model responses to red team attack payloads, and it names the artifact being analyzed (responses to attack payloads), which separates it from sibling tools like rai_redteam_payloads. It stops short of explicitly naming which sibling to use instead or when this differs from rai_scan or rai_bias_evaluate, so it is clear but not fully differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no statement of prerequisites (e.g., that responses must first come from rai_redteam_payloads or rai_scan), and no conditions under which this tool should be preferred over alternatives such as rai_trust_score. The agent must infer usage entirely from the tool name and description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_redteam_payloadsGet Red Team Attack PayloadsARead-onlyIdempotent
Return adversarial attack payloads to probe an AI model for security vulnerabilities. Categories: prompt_injection, jailbreak, data_leakage, role_confusion, delimiter_attack.
| Name | Required | Description | Default |
|---|---|---|---|
| categories | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds that output is offensive-security payload content, which is meaningful context, but it says nothing about auth requirements, rate limits, or what happens when no categories are supplied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the outcome and followed by the supported categories. The category list is slightly redundant against the schema enum, but it costs little and reads efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the return-value burden; it identifies the payload content and category space but not the shape or volume of the response. Combined with annotations that cover the read-only/idempotent profile, it is adequate for a simple one-parameter lookup, with only the default-behavior gap remaining.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and there is a single optional 'categories' array parameter. The description enumerates the same values the schema already lists in the item enum, so it confirms the valid set but adds no new semantics — notably it does not state that omitting the parameter returns all categories.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource — 'Return adversarial attack payloads' — and states the purpose ('to probe an AI model for security vulnerabilities'), which separates it from analysis-oriented siblings like rai_redteam_analyze. It stops short of explicitly naming a sibling or contrasting with one, so it is clear but not fully differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The stated purpose ('probe an AI model for security vulnerabilities') implies when the tool is relevant, but there is no explicit when-to-use/when-not-to-use guidance and no alternative tool is named. An agent must infer that this is the payload-source tool and rai_redteam_analyze handles the results.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_scanScan for PII & Harmful ContentARead-onlyIdempotent
Scan text for PII (email, phone, SSN, credit card, IP address) and harmful content (hate speech, violence, self-harm). Returns findings and a redacted copy.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to scan | |
| redact | No | Replace detected PII with [REDACTED] |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive/closed-world, so the safety profile is covered. The description adds value by disclosing the response shape ('Returns findings and a redacted copy') and the redaction convention ([REDACTED]), which the annotations do not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler: the detection scope comes first and the return behavior second. Nothing is redundant or padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must carry return-value information, and it does so at least in outline ('findings and a redacted copy'), though it does not say what a 'finding' contains (type, spans, severity). For a simple 2-parameter tool this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters are already documented at the field level. The description reinforces the redact behavior with the [REDACTED] placeholder, but adds no syntax, format, or edge-case meaning beyond the schema baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb ('Scan') plus resource ('text') with explicit enumerated detection categories (email, phone, SSN, credit card, IP; hate speech, violence, self-harm). No sibling is named for contrast, so the differentiation from rai_stream_scan or rai_pii_report is only implied by the 'single text' scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no alternatives named among the many rai_* siblings that overlap (rai_pii_report, rai_stream_scan, rai_policy_check). The agent must infer usage entirely from the verb 'Scan'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_stream_scanScan Streaming Text ChunksARead-onlyIdempotent
Scan a list of text chunks (as would arrive from an LLM streaming response) for PII and harmful content. Simulates the StreamingScanner guardrail without a live stream. Returns per-chunk scan results and an aggregated summary with stop recommendation.
| Name | Required | Description | Default |
|---|---|---|---|
| chunks | Yes | Ordered list of text chunks from LLM output stream | |
| hard_stop | No | Stop processing after first PII detection | |
| scan_window | No | Scan every N chunks |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish the safe read-only, idempotent, closed-world profile, so the bar is lower. The description still adds real value by disclosing that the tool is a simulation of StreamingScanner requiring no live stream, and by naming the return shape (per-chunk results plus an aggregated summary with a stop recommendation) even though no output schema exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: what is scanned, the simulation caveat, and the return shape. The core resource and scope are front-loaded with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully summarizes the return value, and all three parameters are fully covered by the schema. It is nearly complete for a read-only simulation tool; only the relationship to the sibling rai_scan is left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so chunks, hard_stop, and scan_window are already documented in the schema including defaults. The description adds nothing about parameter behavior (e.g., how hard_stop interacts with the stop recommendation), so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb+resource ('Scan a list of text chunks ... for PII and harmful content') and scopes it to streaming LLM output, which is the key differentiator from the non-streaming sibling rai_scan. It does not name that sibling explicitly, so an agent must infer the routing from the word 'streaming'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Simulates the StreamingScanner guardrail without a live stream' implies a testing/simulation context, but the description never states when to choose this over rai_scan or any other scan tool, nor any prerequisites. Usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_trust_scoreCompute AI Trust ScoreBRead-onlyIdempotent
Compute a composite AI Trust Score (0-100) across six governance dimensions: fairness, privacy, security, robustness, compliance, authenticity. Returns score, letter grade (A-F), and risk tier (LOW/MEDIUM/HIGH/CRITICAL).
| Name | Required | Description | Default |
|---|---|---|---|
| privacy | No | ||
| fairness | No | ||
| security | No | ||
| compliance | No | ||
| robustness | No | ||
| authenticity | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety and determinism profile is covered. The description adds useful context by disclosing the output shape (0-100 score, A-F grade, LOW-to-CRITICAL risk tier), but says nothing about weighting, determinism of the composite, or how missing dimensions are treated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences: the first states the computation and its dimensional scope, the second the return values. No filler and the core purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a no-output-schema tool the description does cover the return contract, and annotations cover safety. What is missing is the input scale/default behavior for six undocumented parameters and any differentiation from the sibling rai_check_trust, which is the main thing an agent needs to choose correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the burden, and it partially does by naming the same six dimensions as the parameters. However, it never explains that inputs are 0-1 normalized (while the output is 0-100), that all six default to 0.5, or how the dimensions are weighted, so the semantics of the actual inputs remain under-specified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource ('Compute a composite AI Trust Score') and enumerates the six governance dimensions and the output artifacts (score, letter grade, risk tier). It is clear what the tool does, but it never distinguishes itself from the close sibling rai_check_trust, leaving the agent to guess which trust-scoring tool to pick.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of alternatives. With a near-identical sibling (rai_check_trust) in the toolset, the absence of routing criteria is a real gap rather than a minor omission.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rai_webhook_statusCheck Webhook Delivery HealthBRead-onlyIdempotent
Check webhook delivery health and generate a structured status report. Takes delivery statistics and returns health grade, failure analysis, dead-letter queue status, and recommended remediation actions. Used by Security Engineers feeding SIEM systems and Platform Engineers debugging webhook pipelines.
| Name | Required | Description | Default |
|---|---|---|---|
| failed | No | ||
| endpoints | No | ||
| successful | No | ||
| avg_latency_ms | No | ||
| total_deliveries | No | ||
| dead_letter_count | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only, idempotent, non-destructive, and closed-world behavior. The description adds what the report contains (health grade, failure analysis, dead-letter queue status, remediation actions), which is useful given no output schema, but does not describe permissions, rate limits, or other operational traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose, then input/output, then audience. No redundant or filler content; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has six parameters with no schema descriptions and no output schema. While the description covers the high-level purpose and output, it fails to explain the parameters an agent must supply, leaving a significant gap for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must carry the full burden of explaining the six parameters. It only says 'Takes delivery statistics' generically and does not clarify the meaning, format, or role of any parameter such as 'endpoints' or 'dead_letter_count'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (check) and resource (webhook delivery health) plus the output (structured status report). It is clearly distinct from general health tools like rai_health by virtue of 'webhook', but does not explicitly name or contrast with siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides intended users (Security/Platform Engineers) and contexts (SIEM, debugging pipelines), which implies when it is useful, but offers no explicit 'use this instead of X when Y' guidance or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
30 tool updates
v1.2.6- First observed
rai_audit_summary - First observed
rai_benchmark - First observed
rai_benchmark_prompts - First observed
rai_bias_evaluate - First observed
rai_budget_check - First observed
rai_causal_influence_check - First observed
rai_check_trust - First observed
rai_compare_models - First observed
rai_compliance - First observed
rai_cost_estimate - First observed
rai_drift_check - First observed
rai_eu_ai_act_classify - First observed
rai_executive_summary - First observed
rai_hallucination - First observed
rai_health - First observed
rai_incident_log - First observed
rai_iso42001_gap - First observed
rai_memory_read_check - First observed
rai_memory_write_check - First observed
rai_model_route - First observed
rai_org_status - First observed
rai_passport_generate - First observed
rai_pii_report - First observed
rai_policy_check - First observed
rai_redteam_analyze - First observed
rai_redteam_payloads - First observed
rai_scan - First observed
rai_stream_scan - First observed
rai_trust_score - First observed
rai_webhook_status
TDQS
Scored across 30 tools
Despite covering overlapping governance themes, descriptions carry explicit 'use X instead of Y' guidance (e.g., rai_compliance vs rai_eu_ai_act_classify, rai_trust_score vs rai_check_trust), and the scan-family (rai_scan/rai_stream_scan/rai_pii_report) and memory-family tools are clearly differentiated. Each tool has a distinct resource+action target with no true duplicates.
Every tool uses a uniform rai_ prefix with a consistent snake_case convention (rai_trust_score, rai_policy_check, rai_bias_evaluate). No mixing of camelCase, no erratic verb styles.
30 tools is heavy and pushes past the comfortable ceiling, even if the governance domain is genuinely broad (privacy, security, compliance, cost, bias, audit). Most tools earn their place, but a set this large increases selection burden for agents.
The surface covers an unusually full governance lifecycle: scanning, trust scoring, bias evaluation, compliance (NIST/EU AI Act/ISO 42001), red-teaming, cost/budget, drift, passports, incident logging, and memory/provenance gating. No obvious dead ends for the stated purpose.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Hosted MCP server for AI agent identity, permissions, verification, and reusable proof.
1- gatewayOAuthai.sealgate
MCP gateway with runtime security policy, tool-call-level control, and audit of agent actions.
Zero-secret MCP gateway for AI agents: risk-scored, audited calls with human-in-the-loop approval.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceMCP server for AI agent security guardrails. Provides input validation, prompt injection detection, PII redaction, output filtering, policy enforcement, rate limiting, and comprehensive audit logging.381MIT
- AlicenseNot gradedqualityDmaintenanceA production-grade MCP server designed for multi-tenant, authenticated, and observable AI agent systems, enabling secure tool execution across heterogeneous data sources.62MIT
- AlicenseNot gradedqualityCmaintenanceAn MCP server that enforces runtime governance on AI agent actions — file access, command execution, delegation chains, and permission escalation.MIT

@vorionsys/mcp-serverofficial
AlicenseAqualityBmaintenanceMCP server for AI-agent governance using trust scoring, behavioral signals, and pre-flight action checks.10241Apache 2.0