Skip to main content
Glama
Mipiti
by Mipiti

Submit Functional Test Assertions

submit_functional_test_assertions

Attach machine-verifiable evidence assertions to an existing functional test so CI can verify it. Bind test existence and passing claims to a repository for independent verification.

Instructions

Attach machine-verifiable evidence assertions to one already-existing functional test so CI can verify it. Mutating.

This submits EVIDENCE for a test that already exists (identified by functional_test_id) — it does not create or register the test. It is the functional-conformance analog of submit_assertions (which covers security controls): it binds assertions such as "the test exists" and "the test passes" to the functional test, and an independent CI run against the named repo is what turns an operator's "verified" claim into verified state.

To bulk-register test DEFINITIONS from your codebase instead, use import_functional_tests; to hand-author a single test use add_functional_test. Call this after the test is implemented (e.g. following get_scan_prompt (kind="functional")), then read the resulting state via get_functional_coverage or get_functional_test_sufficiency.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
model_idYesID of the threat model.
server_versionYes
assertions_jsonYesJSON array of assertion objects, each {"type": "test_attested" | "test_exists" | ..., "params": {...}, "description": "...", "repo": "<owner>/<repo>"}. Every assertion must carry an explicit repo, or the "no_repo" sentinel when the check is not tied to a repository. These assertions count toward functional conformance; a ``covers`` declaration is refused here, because a binding to a control clause is declared on ``submit_assertions``.
functional_test_idYesThe already-existing functional test the assertions prove.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.77.0
    • changedInput schema / properties / assertions_json / description
      Previous value: -"JSON array of assertion objects, each {\"type\": \"test_attested\" | \"test_exists\" | ..., \"params\": {...}, \"description\": \"...\", \"repo\": \"<owner>/<repo>\"}. Every assertion must carry an explicit repo, or the \"no_repo\" sentinel when the check is not tied to a repository."New value: +"JSON array of assertion objects, each {\"type\": \"test_attested\" | \"test_exists\" | ..., \"params\": {...}, \"description\": \"...\", \"repo\": \"<owner>/<repo>\"}. Every assertion must carry an explicit repo, or the \"no_repo\" sentinel when the check is not tied to a repository. These assertions count toward functional conformance; a ``covers`` declaration is refused here, because a binding to a control clause is declared on ``submit_assertions``."
  2. Changed1 schema field changedv0.75.0
    • changedInput schema / properties / assertions_json / description
      Previous value: -"JSON array of assertion objects, each {\"type\": \"test_passes\" | \"test_exists\" | ..., \"params\": {...}, \"description\": \"...\", \"repo\": \"<owner>/<repo>\"}. Every assertion must carry an explicit repo, or the \"no_repo\" sentinel when the check is not tied to a repository."New value: +"JSON array of assertion objects, each {\"type\": \"test_attested\" | \"test_exists\" | ..., \"params\": {...}, \"description\": \"...\", \"repo\": \"<owner>/<repo>\"}. Every assertion must carry an explicit repo, or the \"no_repo\" sentinel when the check is not tied to a repository."
  3. Addedv0.68.2

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations to lean on, the description carries the full burden. It clearly labels the tool as 'Mutating' and explains the effect: assertions are bound to the functional test and an independent CI run converts an operator's claim into verified state. It also clarifies the distinction from submit_assertions. However, it does not disclose edge behaviors like idempotency, whether existing assertions are replaced, or error conditions, leaving minor gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is more than a single sentence but each part earns its place: the opening one-liner states purpose, the middle clarifies the workflow and CI verification, and the closing routes to alternatives. It is logically ordered (purpose → distinction → usage) and front-loaded with the core verb and resource. While it is a bit longer than necessary, the detail is justified given the complexity and the need to differentiate among many siblings.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity, the presence of an output schema (covering return values), and the large sibling set, the description is remarkably complete. It explains prerequisites (test must already exist), the relationship to CI, the distinction from security-control assertions, and the correct sequencing with other tools. It even references specific functional-test follow-ups. An agent can decide when and how to call this tool without recourse to external documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes three of four parameters with meaningful detail (model_id, functional_test_id, assertions_json), and server_version is left undescribed. The description adds conceptual context (e.g., that assertions_json is evidence and mentions the 'covers' refusal via the parameter description) but does not compensate for the missing server_version or provide additional parameter semantics beyond the schema. With 75% schema coverage, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise verb-resource pairing: 'Attach machine-verifiable evidence assertions to one already-existing functional test.' It further clarifies the negative scope ('does not create or register the test') and differentiates from siblings by naming specific alternatives (import_functional_tests, add_functional_test, submit_assertions). This leaves no ambiguity about what the tool does and how it is distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage timing ('Call this after the test is implemented'), pairs it with a concrete preceding step (get_scan_prompt), and names follow-up read tools (get_functional_coverage, get_functional_test_sufficiency). It also gives explicit alternatives with conditions ('To bulk-register test DEFINITIONS... use import_functional_tests; to hand-author a single test use add_functional_test'). This is textbook guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools