Skip to main content
Glama
aethis-mcp

aethis-mcp

Official
by Aethis-ai

aethis_set_tests

Destructive

Replace a project's complete reviewed test suite after field discovery, overwriting prior tests without changing sources, fields, or guidance. Confirm replacement support first.

Instructions

Replace the complete reviewed test suite for an existing project after field discovery. Requires 1 to 100 legacy cases, or up to 500 with contract_version: 1, and replaces prior tests without creating a project or changing its sources, fields, or guidance. This is destructive. The target API must advertise replacement support before any write. If the response is interrupted, inspect the project before approving another replacement.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "project_id": "<project_id>",
      "test_cases": "<test_cases>"
    },
    "name": "aethis_set_tests"
  }
}

Input Schema

NameRequiredDescriptionDefault
project_idYesExisting project ID whose complete test suite will be replaced
test_casesYesThe complete authoritative reviewed suite (1-500 cases); this replaces existing tests
contract_versionNoRequired when acceptance expectations, expected_review_bindings or retained_passages are supplied.
retained_passagesNoOptional list of source passages that must be retained in the rule notes. Each is {id, text} with text at most 2000 characters; matching ignores case, whitespace runs and typographic quotes and checks literal retention, not meaning.
expected_review_bindingsNoOptional review-binding catalogue; omit for no assertion or use {} to assert zero bindings.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.24.0
    • changedInput schema / properties / test_cases / items / properties / expectations / anyOf
      Previous value: -[
      -  {
      -    "additionalProperties": false,
      -    "properties": {
      -      "pending_reviews": {
      -        "additionalProperties": false,
      -        "properties": {
      -          "resolution_fields": {
      -            "items": {
      -              "type": "string"
      -            },
      -            "maxItems": 500,
      -            "type": "array"
      -          },
      -          "unmapped_count": {
      -            "maximum": 500,
      -            "minimum": 0,
      -            "type": "integer"
      -          }
      -        },
      -        "required": [
      -          "unmapped_count"
      -        ],
      -        "type": "object"
      -      },
      -      "useful_unknown_fields": {
      -        "items": {
      -          "$ref": "#/properties/test_cases/items/properties/expectations/anyOf/0/properties/pending_reviews/properties/resolution_fields/items"
      -        },
      -        "maxItems": 500,
      -        "type": "array"
      -      }
      -    },
      -    "type": "object"
      -  },
      -  {
      -    "type": "null"
      -  }
      -]New value: +[
      +  {
      +    "additionalProperties": false,
      +    "properties": {
      +      "field_errors": {
      +        "additionalProperties": {
      +          "type": "string"
      +        },
      +        "description": "Exact message the engine is expected to report for each field when an input invariant rejects the inputs. Only valid when expected_outcome is undetermined.",
      +        "type": "object"
      +      },
      +      "pending_reviews": {
      +        "additionalProperties": false,
      +        "properties": {
      +          "resolution_fields": {
      +            "items": {
      +              "type": "string"
      +            },
      +            "maxItems": 500,
      +            "type": "array"
      +          },
      +          "unmapped_count": {
      +            "maximum": 500,
      +            "minimum": 0,
      +            "type": "integer"
      +          }
      +        },
      +        "required": [
      +          "unmapped_count"
      +        ],
      +        "type": "object"
      +      },
      +      "useful_unknown_fields": {
      +        "items": {
      +          "$ref": "#/properties/test_cases/items/properties/expectations/anyOf/0/properties/pending_reviews/properties/resolution_fields/items"
      +        },
      +        "maxItems": 500,
      +        "type": "array"
      +      }
      +    },
      +    "type": "object"
      +  },
      +  {
      +    "type": "null"
      +  }
      +]
  2. Changed2 schema fields changedv0.23.0
    • changedInput schema / properties / contract_version / description
      Previous value: -"Required when acceptance expectations or expected_review_bindings are supplied."New value: +"Required when acceptance expectations, expected_review_bindings or retained_passages are supplied."
    • addedInput schema / properties / retained_passages
      Added value: +{
      +  "description": "Optional list of source passages that must be retained in the rule notes. Each is {id, text} with text at most 2000 characters; matching ignores case, whitespace runs and typographic quotes and checks literal retention, not meaning.",
      +  "items": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "id": {
      +        "description": "Unique identifier for the passage",
      +        "type": "string"
      +      },
      +      "match": {
      +        "const": "normalized_contains",
      +        "type": "string"
      +      },
      +      "text": {
      +        "description": "Source text that must be retained",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "id",
      +      "text"
      +    ],
      +    "type": "object"
      +  },
      +  "maxItems": 100,
      +  "type": "array"
      +}
  3. First observedv0.22.0

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructiveHint=true and idempotentHint=false, and the description still adds real substance: it states the exact blast radius (replaces prior tests, leaves sources/fields/guidance untouched), a required server-side capability check before writing, and an interruption/retry hazard. That is behavior an agent could not infer from the annotation set alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Five short sentences, front-loaded with purpose, then limits, then the destructive warning, then two operational guards. Nothing is repeated from the schema or annotations, and the terse "This is destructive" is well placed right after the blast-radius statement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, open-world, 5-parameter tool with nested objects and no output schema, the description covers preconditions, limits, blast radius, and retry safety. The remaining gap is what a successful or failed response actually looks like — it only hints at this indirectly via the "response is interrupted" case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns above that by adding a cross-parameter constraint absent from the schema — 1-100 legacy cases normally, extended to 500 only with contract_version: 1. It does not explain expectations/retained_passages/expected_review_bindings semantics, leaving that entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb and resource ("Replace the complete reviewed test suite"), scopes it to an "existing project", and sequences it "after field discovery". It also carves out what it does NOT do ("without creating a project or changing its sources, fields, or guidance"), which separates it from generation and field-editing siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear precondition (run after field discovery), a hard gate ("The target API must advertise replacement support before any write"), and a recovery rule ("If the response is interrupted, inspect the project before approving another replacement"). What it lacks is an explicit named alternative for when replacement isn't the right call, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.