Skip to main content
Glama
io.github.justrunme/agent-security-contract

io.github.justrunme/agent-security-contract

by justrunme

verify_scenario

Verifies a single corpus scenario against an ASC policy by running its trace through the policy evaluator and returning whether the actual verdict matches the expected result, with failure reasons.

Instructions

Verify a single corpus scenario file against an ASC policy.

Loads the scenario JSON, runs the trace through the policy evaluator, and returns whether the actual verdict matches the expected verdict.

Args: policy_path: Path to the ASC policy YAML file scenario_path: Path to a single scenario JSON file

Returns: Mapping with match (bool), expected, actual, reason, and scenario metadata

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "policy_path": "<policy_path>",
      "scenario_path": "<scenario_path>"
    },
    "name": "verify_scenario"
  }
}

Input Schema

NameRequiredDescriptionDefault
policy_pathYes
scenario_pathYes

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.3.1

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does explain the internal flow (loads JSON, runs the trace through the evaluator, compares verdicts). However it omits whether the operation is read-only, what happens on malformed JSON or a missing policy file, and any failure semantics — meaningful gaps for an unannotated tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded purpose sentence followed by a compact flow sentence, with Args/Returns sections that are easy to scan. The Returns block partly duplicates the existing output schema, which is minor waste but not harmful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema already documents return values, so the Returns section is redundant rather than compensating. What is missing is the behavioral envelope: no annotations, no statement about side effects, error handling, or how a non-matching verdict is reported beyond the field list.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does: it identifies policy_path as an ASC policy YAML file and scenario_path as a single scenario JSON file, giving file formats the bare schema lacks. It stops short of path conventions or relative-vs-absolute guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Verify a single corpus scenario file against an ASC policy') and the word 'single' implicitly contrasts with the sibling verify_corpus. An agent can tell what it does without opening the schema, though the differentiation from verify_corpus is implied rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: 'single scenario file' suggests this is the per-file counterpart to a bulk verifier, but the description never says when to choose this over verify_corpus or check_tool_call. No prerequisites or exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.