Skip to main content
Glama
SweetKenneth

Counterfactual Immune Forge

by SweetKenneth

adjudicate_defensive_mutation

Evaluate proposed defensive mutations against recorded attack scenarios, verifying replay, regression, and fitness improvement before sealing verdicts as Merkle evidence roots.

Instructions

Adjudicate one defensive-mutation episode from recorded observations: impact screen, same-scenario replay, mandatory regression gates and positive fitness delta, then seal the decision as a Merkle evidence root. Executes nothing and promotes nothing on its own.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
policyNo
baselineYes
scenarioYes
diagnosisNo
candidatesNo
baselineReplayYes
baselineFitnessNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed23 schema fields changedv0.2.1
    • addedInput schema / additionalProperties
      Added value: +false
    • addedInput schema / properties / baseline / additionalProperties
      Added value: +false
    • addedInput schema / properties / baseline / properties / id / maxLength
      Added value: +4096
    • addedInput schema / properties / baseline / properties / id / minLength
      Added value: +1
    • addedInput schema / properties / baseline / properties / version / maxLength
      Added value: +4096
    • addedInput schema / properties / baseline / properties / version / minLength
      Added value: +1
    • addedInput schema / properties / baselineFitness
      Added value: +{
      +  "type": "number"
      +}
    • addedInput schema / properties / baselineReplay / additionalProperties
      Added value: +false
    • addedInput schema / properties / baselineReplay / properties / scenarioId
      Added value: +{
      +  "maxLength": 4096,
      +  "minLength": 1,
      +  "type": "string"
      +}
    • changedInput schema / properties / baselineReplay / required
      Previous value: -[
      -  "reproduced",
      -  "attackSucceeded",
      -  "securityScore"
      -]New value: +[
      +  "scenarioId",
      +  "reproduced",
      +  "attackSucceeded",
      +  "securityScore"
      +]
    • addedInput schema / properties / candidates / items / additionalProperties
      Added value: +false
    • addedInput schema / properties / candidates / items / properties
      Added value: +{
      +  "defense": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "id": {
      +        "maxLength": 4096,
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      "state": {},
      +      "version": {
      +        "maxLength": 4096,
      +        "minLength": 1,
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "id",
      +      "version"
      +    ],
      +    "type": "object"
      +  },
      +  "fitnessScore": {
      +    "type": "number"
      +  },
      +  "impact": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "reasons": {
      +        "items": {
      +          "type": "string"
      +        },
      +        "type": "array"
      +      },
      +      "riskScore": {
      +        "type": "number"
      +      },
      +      "safe": {
      +        "type": "boolean"
      +      }
      +    },
      +    "required": [
      +      "safe",
      +      "reasons"
      +    ],
      +    "type": "object"
      +  },
      +  "mutation": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "description": {
      +        "maxLength": 4096,
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      "id": {
      +        "maxLength": 4096,
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      "patch": {}
      +    },
      +    "required": [
      +      "description",
      +      "patch"
      +    ],
      +    "type": "object"
      +  },
      +  "regression": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "failures": {
      +        "items": {
      +          "type": "string"
      +        },
      +        "type": "array"
      +      },
      +      "passed": {
      +        "type": "boolean"
      +      },
      +      "score": {
      +        "type": "number"
      +      }
      +    },
      +    "required": [
      +      "passed",
      +      "failures"
      +    ],
      +    "type": "object"
      +  },
      +  "replay": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "attackSucceeded": {
      +        "type": "boolean"
      +      },
      +      "reproduced": {
      +        "type": "boolean"
      +      },
      +      "scenarioId": {
      +        "maxLength": 4096,
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      "securityScore": {
      +        "type": "number"
      +      },
      +      "state": {},
      +      "trace": {}
      +    },
      +    "required": [
      +      "scenarioId",
      +      "reproduced",
      +      "attackSucceeded",
      +      "securityScore"
      +    ],
      +    "type": "object"
      +  }
      +}
    • addedInput schema / properties / candidates / items / required
      Added value: +[
      +  "mutation",
      +  "defense",
      +  "impact"
      +]
    • addedInput schema / properties / candidates / maxItems
      Added value: +256
    • addedInput schema / properties / policy / additionalProperties
      Added value: +false
    • removedInput schema / properties / policy / properties / requireAttackReproduction
      Removed value: -{
      -  "type": "boolean"
      -}
    • addedInput schema / properties / policy / properties / requiredFitnessMargin / minimum
      Added value: +0
    • addedInput schema / properties / scenario / additionalProperties
      Added value: +false
    • addedInput schema / properties / scenario / properties / expectedSecurityProperty / maxLength
      Added value: +4096
    • addedInput schema / properties / scenario / properties / expectedSecurityProperty / minLength
      Added value: +1
    • addedInput schema / properties / scenario / properties / kind / maxLength
      Added value: +4096
    • addedInput schema / properties / scenario / properties / kind / minLength
      Added value: +1
    • changedInput schema / properties / scenario / required
      Previous value: -[
      -  "kind",
      -  "expectedSecurityProperty"
      -]New value: +[
      +  "kind",
      +  "payload",
      +  "expectedSecurityProperty"
      +]
  2. First observedv0.1.0

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. It discloses the internal evaluation stages—impact screen, same-scenario replay, mandatory regression gates, positive fitness delta—and explicitly states the tool 'Executes nothing and promotes nothing on its own.' It also reveals that a Merkle evidence root is produced. This is substantial transparency for a non-annotated tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense sentence that front-loads the core action and then lists the evaluation criteria. It is appropriately sized and contains no filler, though the long colon-list structure makes it slightly heavy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of annotations, an output schema, and any parameter-level documentation, this description is too incomplete for an agent to safely invoke the tool. It gives a good process overview but omits required-input semantics, expected outputs, and the meaning of key fields like policy, diagnosis, and baselineFitness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, yet the description does not explain any parameter names or required inputs. It references high-level concepts like 'impact' and 'replay,' which map loosely to schema fields such as impact, replay, and baselineReplay, but it does not clarify scenario, baseline, candidates, policy, fitnessScore, or baselineFitness. This is insufficient for a tool with seven nested parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Adjudicate one defensive-mutation episode from recorded observations.' It then enumerates the exact adjudication steps and distinguishes itself from siblings by noting it 'Executes nothing and promotes nothing on its own,' which clearly separates adjudication from execution and promotion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The purpose is clear enough to imply when this tool should be used: when a defensive-mutation episode needs adjudication. However, it does not explicitly mention alternative tools such as verify_episode_evidence or explain when to prefer this over them, so usage guidance is left mostly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.