Skip to main content
Glama

website-search

by com.zeltser
OverviewDiscussionsAdmin

aidefense_evaluate_program

Read-onlyIdempotent

Get the AI Defense Matrix evaluation playbook for assessing an AI security program: per-cell prompts, gap-inventory template, and a workflow that walks each asset class first and rolls findings up to the Govern column. Supports mode='gate' for binary deployment-gate decisions (returns the deployment-gate workflow plus gate-tier prompts only) and consumerPattern for scoping to consumed-vs-built AI deployments. The AI applies these prompts against your program documentation locally, and no program details leave your client. This server never requests your program docs or product roadmap and instructs your AI to keep them local—the matrix, framework alignments, and playbooks flow to your AI for local analysis.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
modeNoassessment (default): full program assessment — all maturity tiers. gate: binary deployment-gate decision — returns the deploymentGateWorkflow plus only gate-tier prompts (drops 90-day and mature prompts).
assetNoOptional: focus the prompts on one asset class.
assetsNoOptional: focus the prompts on multiple asset classes (e.g., for a deployment that touches orchestration + runtime data + agent identities). Takes precedence over `asset` if both are set.
functionNoOptional: focus the prompts on one NIST CSF function.
frameworkNoOptional: scope cellPrompts to those whose 'sources' field cites the named framework. Accepts a bare slug ('iso-42001') for any prompt citing that framework, or a 'framework:concept-id' form ('mitre-atlas:AML.T0051') to match an exact technique. Composes with mode, consumer_pattern, asset, and assets.
consumer_patternNoconsumed: organization consumes a third-party model (GPT-4 via API) — drops AI-Workload Platforms, Training Data, AI-Generated Code rows, and AI Model identify/protect/detect/respond/recover (keeps ai-model.govern). built: organization hosts/trains its own model — all rows in scope. hybrid (default behavior when omitted): all rows in scope.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changed
    • changedInput schema / properties / asset / enum
      Previous value: -[
-  "ai-workload-platforms",
-  "ai-orchestration-tools",
-  "ai-generated-code",
-  "ai-gateways-routers",
-  "ai-model",
-  "training-data",
-  "runtime-ai-data",
-  "ai-agent-identities"
-]New value: +[
+  "ai-workload-platforms",
+  "ai-coding-orchestration-tools",
+  "ai-generated-code",
+  "ai-traffic",
+  "ai-model",
+  "training-data",
+  "runtime-ai-data",
+  "ai-agent-identities",
+  "ai-gateways-routers",
+  "ai-orchestration-tools"
+]
    • changedInput schema / properties / assets / items / enum
      Previous value: -[
-  "ai-workload-platforms",
-  "ai-orchestration-tools",
-  "ai-generated-code",
-  "ai-gateways-routers",
-  "ai-model",
-  "training-data",
-  "runtime-ai-data",
-  "ai-agent-identities"
-]New value: +[
+  "ai-workload-platforms",
+  "ai-coding-orchestration-tools",
+  "ai-generated-code",
+  "ai-traffic",
+  "ai-model",
+  "training-data",
+  "runtime-ai-data",
+  "ai-agent-identities",
+  "ai-gateways-routers",
+  "ai-orchestration-tools"
+]
  2. Changed1 schema field changed
    • addedInput schema / properties / framework
      Added value: +{
+  "description": "Optional: scope cellPrompts to those whose 'sources' field cites the named framework. Accepts a bare slug ('iso-42001') for any prompt citing that framework, or a 'framework:concept-id' form ('mitre-atlas:AML.T0051') to match an exact technique. Composes with mode, consumer_pattern, asset, and assets.",
+  "pattern": "^(nist-ir-8596|csa-aicm|iso-42001|google-saif|mitre-atlas|owasp-ai-exchange|owasp-llm-top10|owasp-asi)(:[A-Za-z0-9._-]+)?$",
+  "type": "string"
+}
  3. Changed3 schema fields changed
    • addedInput schema / properties / assets
      Added value: +{
+  "description": "Optional: focus the prompts on multiple asset classes (e.g., for a deployment that touches orchestration + runtime data + agent identities). Takes precedence over `asset` if both are set.",
+  "items": {
+    "enum": [
+      "ai-workload-platforms",
+      "ai-orchestration-tools",
+      "ai-generated-code",
+      "ai-gateways-routers",
+      "ai-model",
+      "training-data",
+      "runtime-ai-data",
+      "ai-agent-identities"
+    ],
+    "type": "string"
+  },
+  "type": "array"
+}
    • addedInput schema / properties / consumer_pattern
      Added value: +{
+  "description": "consumed: organization consumes a third-party model (GPT-4 via API) — drops AI-Workload Platforms, Training Data, AI-Generated Code rows, and AI Model identify/protect/detect/respond/recover (keeps ai-model.govern). built: organization hosts/trains its own model — all rows in scope. hybrid (default behavior when omitted): all rows in scope.",
+  "enum": [
+    "consumed",
+    "built",
+    "hybrid"
+  ],
+  "type": "string"
+}
    • addedInput schema / properties / mode
      Added value: +{
+  "description": "assessment (default): full program assessment — all maturity tiers. gate: binary deployment-gate decision — returns the deploymentGateWorkflow plus only gate-tier prompts (drops 90-day and mature prompts).",
+  "enum": [
+    "assessment",
+    "gate"
+  ],
+  "type": "string"
+}
  4. Added

TDQS

A3.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, and the description adds substantial behavioral context beyond that: 'The AI applies these prompts against your program documentation locally, and no program details leave your client.' It also states that the server never requests program docs or product roadmap, giving the agent important data-handling expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence is informative and front-loaded, but the description becomes verbose with overlapping data-locality statements repeated across two sentences ('no program details leave your client' and 'instructs your AI to keep them local'). It could be tightened to two concise sentences without losing meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's core purpose, key mode behavior, and the privacy model, which is largely sufficient given the absence of an output schema. The main gaps are the consumerPattern naming inconsistency with the schema and a lack of detail on the gap-inventory template's structure, but overall the agent has the context needed to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all six parameters, setting the baseline at 3. The description adds minimal extra parameter meaning—mainly reinforcing mode='gate' behavior—but it calls the consumer_pattern parameter 'consumerPattern,' which is inconsistent with the schema and could mislead the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb and resource: 'Get the AI Defense Matrix evaluation playbook for assessing an AI security program' and enumerates the deliverables (per-cell prompts, gap-inventory template, workflow). It is not a tautology and clearly conveys the tool's function, though it does not explicitly differentiate itself from siblings like aidefense_get_matrix.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when the tool is used ('for assessing an AI security program') and explains the gate mode for binary deployment-gate decisions. However, it does not explicitly say when not to use this tool or name alternatives among the many sibling tools, so usage selection is mostly implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All website-search toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources