llm-assure-mcp

run_assessment

Run automated red-team assessments against chatbots, RAG apps, or AI agents and generate HTML, JSON, JUnit, or Markdown reports with scores, findings, and evidence.

Instructions

Run an automated red-team assessment and write HTML/JSON/JUnit/Markdown reports.

Args: target_file: path to a target YAML file (preferred). target: inline target config (same keys as the YAML) if no file is given. suites: suite ids to run; omit for all. tags: only run probes with these tags (e.g. ["baseline"] for a quick scan). pack_dirs: extra suite directories (premium/industry packs). use_judge: enable the Claude LLM-judge detector (needs ANTHROPIC_API_KEY). output_dir: where reports are written. concurrency: parallel requests to the target. Returns score, grade, counts, findings with evidence, and report paths.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "run_assessment"
  }
}

Input Schema

Name	Required	Default
`tags`	No
`suites`	No
`target`	No
`pack_dirs`	No
`use_judge`	No
`output_dir`	No	llm-assure-report
`concurrency`	No
`target_file`	No

Output Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

First observedv0.1.0Nov 13, 2026

TDQS

A4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden well: it discloses that reports are written to disk, that the LLM judge requires ANTHROPIC_API_KEY, and that concurrency issues parallel requests to the target. It does not say whether existing reports in output_dir are overwritten, how long a run takes, or that it sends live traffic to the target — gaps that matter for an unattended mutating run.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is front-loaded and the Args list is dense with no filler; each line adds information the schema does not. The docstring-style layout is functional but slightly mechanical, and the trailing Returns line partly duplicates the output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter orchestration tool, the definition covers inputs, environment prerequisites, and side effects; the existence of an output schema means return values needn't be detailed in depth. Remaining gaps are the target-config key structure and overwrite behavior for the report directory.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, yet the Args block documents all eight parameters, including non-obvious ones like pack_dirs ('premium/industry packs') and use_judge's key requirement. It stops short of explaining what keys a target config accepts or the practical range for concurrency.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Run an automated red-team assessment') plus the concrete side effect ('write HTML/JSON/JUnit/Markdown reports'). It is clearly distinct from the list/get/send siblings, though it never names or contrasts them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives real routing guidance at the argument level: target_file is 'preferred', target is the fallback 'if no file is given', omit suites to run all, and tags ['baseline'] for a quick scan. It lacks any when-not-to-use guidance relative to siblings such as list_suites or send_probe, so it is short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server