Compass by 402rates

Start or repeat a synthetic agent test

start_agent_test

Create a free bounded synthetic run without an account. An optional finalized previous run and its token link a repeat of the same case version and scenario. No real purchase. Caller model costs remain external.

Input Schema

TableJSON Schema

Name	Required	Description
`case_id`	No	Versioned synthetic case identifier.
`variant`	No	Budget case: budget_first or attempts_first. Linked repeats retain the previous variant.
`access_token`	No	Secret short-lived run token. Omitted from reports. MCP client history may retain this argument.
`case_version`	No	Exact case version. No implicit migration.
`previous_run_id`	No	Optional finalized run to repeat. Its access token authenticates this start.
`configuration_changed`	No	Optional caller-declared change since the previous run; not independently verified.

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedSep 10, 2026

addedInput schema / properties / variant

Added value: +{
+  "description": "Budget case: budget_first or attempts_first. Linked repeats retain the previous variant.",
+  "maxLength": 32,
+  "type": "string"
+}

First observedSep 9, 2026

TDQS

A3.7/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite annotations indicating a mutating tool, the description adds valuable behavioral nuance: the run is free, bounded, requires no account, and caller model costs remain external. This goes beyond the annotations and helps the agent anticipate consequences of invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief, front-loads the primary purpose, and every sentence contributes useful information about account, cost, and repetition. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description conveys the core intent and cost behavior, but because there is no output schema, it should clarify what the tool returns (e.g., token/run identifier) for the caller to use in subsequent steps. This gap matters for an agent that needs to consume the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are well-defined. The description adds modest context about repeat behavior aligning with previous_run_id and variant, but does not substantially add beyond the schema definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates a synthetic agent test run and can repeat a previous run, using a specific verb and resource. It distinguishes itself from read-only siblings (describe_agent_test, inspect_test_state), though it does not explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives good context (free, no account, repeat via previous_run_id) but does not explicitly state when to prefer this tool over siblings like list_agent_test_cases or execute_test_action. All required information is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Compass by 402rates tools Inspect

Try in Browser

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?