ML Referee
Server Details
Independent statistical feature evaluation for machine learning and forecasting. ML Referee tests up to 1,000 candidate features in one run against a declared baseline, helping researchers and AI agents decide which inputs to pursue, which bounded feature repairs merit further testing, and where more evidence is needed—without a separate ablation run for every feature.
Use the MCP tools to discover input schemas, check access and credits, preflight a dataset, obtain a quote, submit an authorized evaluation, and retrieve structured results and reports. Results are scoped to the submitted evidence and evaluation design; they do not guarantee future model performance.
Invite-only beta with manual approval. Public tool discovery requires no authentication; account and evaluation tools require OAuth through approved client connections and available evaluation credits. Connect directly to https://api.mlreferee.com/mcp.
Website: https://mlreferee.com Request beta access: https://mlreferee.com/beta Agent setup and documentation: https://mlreferee.com/agents
- Status
- Healthy
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
TDQS
Scored across 26 tools
Most tools target a clearly distinct resource+action, and descriptions distinguish the three parallel families (baseline audit, repair, campaign) and their upload/preflight/quote steps. The main risk is misselecting between the three similarly-named families (e.g. quote_audit vs quote_repair vs quote_campaign, or the three upload_* tools), though the payload descriptions help separate them.
Every name follows a strict verb_noun snake_case pattern (upload_private_dataset, preflight_repair, quote_campaign, get_audit_result, delete_audit). Prefixes are used predictably across get_, delete_, upload_, preflight_, quote_ and verify_ operations.
26 tools is on the heavy side and sits at the threshold where the surface starts to feel sprawling. The three full lifecycle families with their own upload/preflight/quote variants justify many of them, but consolidation (e.g. unified upload/preflight/quote with a mode parameter) would reduce load.
Coverage is strong: upload, preflight, quote, submit, status, result, receipt, verification, deletion, plus credits, beta access, schemas, verticals, usage, readiness and followup. Minor gaps like no explicit cancel/abort for a submitted audit or update operation are workable around.
Available Tools
26 toolsdelete_auditADestructiveIdempotentInspect
Delete retained inputs and results for an audit owned by this tenant.
| Name | Required | Description | Default |
|---|---|---|---|
| audit_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, idempotentHint=true and openWorldHint=false, so the safety profile is covered. The description adds genuine value by naming what is destroyed ('retained inputs and results') and the tenant ownership constraint, but is silent on whether the audit record itself survives, whether deletion is recoverable, or what happens on a non-owned audit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the ownership constraint comes after the destructive action so the agent reads the risk first.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter destructive tool with rich annotations and no output schema, the description covers target and scope adequately. Minor gaps about post-deletion state of the audit record remain, but nothing blocks correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one parameter with 0% schema description coverage; the schema only supplies a 32-hex pattern. The description identifies the audit as the deletion target, which maps to audit_id, but adds no format or identity details beyond that implicit link.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Delete retained inputs and results for an audit') plus scope ('owned by this tenant'). It is clearly distinct from delete_upload, though it does not explicitly name that sibling as the alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The tenant-ownership clause implies context (only audits this tenant owns can be deleted), but there is no guidance on when to use this rather than delete_upload, or on preconditions such as audit state or whether results must be retrieved first.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_uploadADestructiveIdempotentInspect
Delete a tenant-private upload that is no longer needed.
| Name | Required | Description | Default |
|---|---|---|---|
| upload_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the safety profile is covered. The description adds the tenant-private scoping detail but says nothing about permanence, whether associated data is destroyed alongside the upload, or authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence, front-loaded with the verb and resource. Nothing is wasted, though the brevity leaves no room for the routing or consequence information an agent would benefit from.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter destructive delete whose annotations already carry the safety profile and whose schema carries the ID pattern, this is close to complete. The main remaining gap is whether deletion is permanent or recoverable, which the annotations cover only implicitly via destructiveHint.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning for upload_id beyond the schema, which conveys only a pattern (^[a-f0-9]{32}$) with no description text. With a single parameter and 0% schema description coverage, a 3 is the minimum-viable result – the description neither explains where the ID comes from nor what form it takes.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Delete) and resource (tenant-private upload), which distinguishes it from the sibling upload_* tools that create rather than remove uploads. The 'tenant-private' qualifier narrows scope, though no explicit sibling is named for contrast.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The clause 'that is no longer needed' implies a usage condition but gives no explicit when/when-not guidance, no prerequisites, and names no alternative. In a family with three upload_* creators, an agent gets only a weak hint about when deletion is the right move.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
describe_refereeBRead-onlyIdempotentInspect
Discover capabilities, privacy, and the approved public catalog.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so the safety profile is fully covered by structured data. The description adds only that the content is about capabilities, privacy, and an 'approved public catalog', giving no sense of format, size, or whether the catalog is static or filtered — modest added value given the annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short, front-loaded sentence with no filler or repetition. It is terse to the point of underspecification rather than verbose, which is the lesser sin here.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and no output schema, the description carries the full burden of telling the agent what the return looks like, and it only gestures at three topic areas without describing structure or whether the response is static reference data. Adequate for a no-arg meta tool, but not sufficient to route confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, so the baseline is 4; there is nothing for the description to disambiguate. The description correctly implies the tool is argument-free and self-contained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description has a verb ('Discover') and names three content areas (capabilities, privacy, approved public catalog), so it is not a pure tautology. However, 'Referee' is never defined, and the phrasing is abstract enough that an agent cannot confidently predict what comes back versus a sibling like list_verticals or get_referee_readiness.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no indication of when to call this tool, whether it should be called first on onboarding, or how it differs from nearby discovery tools such as get_referee_readiness, get_referee_usage, or list_verticals. The agent must infer placement in a 26-tool namespace from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_audit_receiptARead-onlyIdempotentInspect
Read a tenant-private report fingerprint retained after report expiry or deletion. Older reports may have no record. This does not retrieve report contents.
| Name | Required | Description | Default |
|---|---|---|---|
| audit_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this read-only, idempotent and non-destructive, so the description need not restate safety. It adds genuine context the annotations lack: that the record survives expiry/deletion as a fingerprint only, that older reports may have no record at all, and that contents are explicitly out of scope.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences with zero filler, and the scope and the negative constraint ('does not retrieve report contents') land first. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, a 'report fingerprint' is never defined – an agent does not know what it will actually receive back, which is a notable gap for a read tool. The behavioural side (retention, expiry, possible absence) is covered well, so the definition is adequate but incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% – the only parameter is a bare audit_id with a 32-hex pattern and no prose. The description does nothing to compensate: it never explains what an audit_id is, where an agent obtains one, or what the pattern represents, so the sole calling parameter is left unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Read') and a specific resource ('tenant-private report fingerprint'), and explicitly disambiguates it from the report-contents tools by stating 'This does not retrieve report contents.' It never names the nearest sibling (get_audit_result) outright, but the negation does the distinguishing work.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case (after report expiry or deletion) and warns 'Older reports may have no record,' which conditions expectations. It does not, however, state explicitly when to call this instead of get_audit_result or verify_audit_report, leaving routing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_audit_resultBRead-onlyIdempotentInspect
Read a completed tenant-private result and signed output manifest.
| Name | Required | Description | Default |
|---|---|---|---|
| audit_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so safety is covered. The description adds meaningful context with "tenant-private" and "signed output manifest," but it does not explain access requirements, error behavior if the audit is incomplete, or return format details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The definition is a single front-loaded sentence with no filler. Every word contributes to identifying the resource and the completed-state constraint.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and annotations cover its read-only safety profile, but with no output schema the description should say more about what the "result" and "signed output manifest" contain or how they differ from a receipt. It also omits any routing guidance against similar sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one parameter, audit_id, with 0% schema description coverage and only a regex pattern. The description never mentions the parameter or clarifies that it identifies the audit whose result is being read, so it does little to compensate for the missing schema semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb ("Read") and a specific resource ("completed tenant-private result and signed output manifest"), so the agent knows what is being fetched. However, it does not distinguish this from closely named siblings like get_audit_receipt, get_audit_status, or verify_audit_report, leaving some ambiguity about the exact artifact returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The adjective "completed" implies this tool should be used only after an audit has finished, which is useful implied guidance. But the description never states when to prefer it over get_audit_status or get_audit_receipt, nor does it give explicit when-not conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_audit_statusBRead-onlyIdempotentInspect
Read the state of an audit owned by the authenticated tenant.
| Name | Required | Description | Default |
|---|---|---|---|
| audit_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds the tenant-ownership constraint, which is useful authorization context, but says nothing about what 'state' contains or error behavior when the audit is not found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single efficient, front-loaded sentence with no padding. It is appropriately sized but so terse that it omits information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-param, read-only tool with no output schema, the description should at least clarify what 'state' means (e.g., lifecycle stages). It establishes scope but leaves the return semantics and parameter format to inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one required audit_id parameter with 0% schema description coverage, so the description carries the burden of explaining it. The description never mentions audit_id or its 32-char hex format, leaving the parameter undocumented outside the raw pattern.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (read) and resource (state of an audit), so the agent knows exactly what it does. It does not, however, differentiate itself from close siblings like get_audit_result, get_audit_receipt, or verify_audit_report, which all sound related.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'owned by the authenticated tenant' implies the auth/ownership scope under which this applies, but there is no explicit when-to-use or when-not guidance and no pointer to the sibling tools that might be more appropriate for results or receipts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_beta_request_statusARead-onlyIdempotentInspect
Read the authenticated tenant's beta-access request status.
| Name | Required | Description | Default |
|---|---|---|---|
| request_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since annotations already define the tool as read-only and idempotent, the description adds crucial operational context regarding scope: the request belongs to the 'authenticated tenant'. This clarifies the data boundary without contradicting the structured hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, highly efficient sentence that front-loads the function and scope, avoiding any filler or unnecessary jargon.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool is a simple GET request with a single ID parameter, the combination of the description and the provided annotations provides all necessary functional and safety context to execute the call successfully.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, requiring the description to add semantic value to the parameter. While it mentions the parameter is related to the request ID, it does not elaborate on constraints or expected data patterns beyond what the input schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action ('Read') and the specific resource ('beta-access request status'). It includes context ('authenticated tenant's'), making its purpose highly unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case—tracking status—but does not explicitly detail the prerequisite steps or compare it against alternative ways a user might track their progress in the beta program.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_creditsBIdempotentInspect
Read beta or paid audit-credit availability and usage.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description frames the operation as a read ('Read beta or paid audit-credit availability and usage'), but the annotations declare readOnlyHint=false. This is a direct inconsistency between the description and structured annotations, so behavioral transparency is scored at the contradiction level.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no filler. It conveys the core purpose immediately and does not waste space on redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and no output schema, the description covers the basic operation but does not explain what the returned credit availability or usage information looks like. Given the annotation inconsistency and lack of return-value context, it is minimally adequate rather than complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to explain parameter syntax, and the empty schema is consistent with a no-argument call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb+resource: 'Read beta or paid audit-credit availability and usage.' It is specific enough to distinguish this from sibling audit tools, though it does not explicitly name or contrast with the closest alternative, get_referee_usage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit when-to-use guidance, no when-not-to-use conditions, and no alternatives. Usage is only implied by the tool name and the phrase 'audit-credit availability and usage.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_input_schemaCRead-onlyIdempotentInspect
Get neutral column roles and validation defaults for a vertical.
| Name | Required | Description | Default |
|---|---|---|---|
| vertical | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds that the payload is column roles and validation defaults, but says nothing about the shape, size, or whether results are stable across versions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler. It is efficient, though its brevity borders on under-specification rather than well-engineered concision.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description is the only place the return value could be characterized, and 'neutral column roles and validation defaults' is too thin for a schema-introspection tool in an audit pipeline. Nothing links it to the audit workflow or explains what an agent should do with the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must carry the meaning of 'vertical', and it does not — the reader must infer it from the regex pattern listing general_ml, demand_forecasting and bioinformatics_research. The description adds no semantics beyond the schema's own pattern constraint.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'Get' and the resource 'neutral column roles and validation defaults' are stated, and 'for a vertical' scopes it. But 'neutral column roles' is internal jargon that an agent may not decode, and there is no differentiation from the similarly named sibling get_repair_schema or from list_verticals.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description never says when to call this tool, what prerequisite state it assumes, or which sibling to use instead. With get_repair_schema and list_verticals in the same family, the agent is left to guess which schema-retrieval tool applies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_referee_readinessARead-onlyIdempotentInspect
Report connection, credential, audit-credit, and submission readiness with the next action.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false, and openWorldHint=false, so safety and repeatability are covered. The description adds that a recommended next action is returned, which is genuine behavioral value, but says nothing about how failures manifest or whether partial readiness is reported.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence listing the four readiness dimensions and the returned next action. Every clause carries information; no filler or restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, annotated, read-only diagnostic with no output schema, knowing the four checked areas and that a next action is returned is close to sufficient. It stops short of describing what a not-ready result looks like, which is the main remaining gap given the absence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty with zero parameters, so baseline is 4. There is nothing for the description to disambiguate, and it correctly does not pretend otherwise.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ("Report") and enumerates exactly what is assessed: connection, credential, audit-credit, and submission readiness, plus a next action. It is clearly distinguishable from quote_* or preflight_* siblings by its diagnostic, multi-area scope, though it never names an alternative to contrast with.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The mention of "submission readiness" implies a pre-submit/preflight check, but the description never says when to call this versus get_credits, describe_referee, or preflight_dataset. No prerequisites or exclusions are given, so the agent must infer the trigger condition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_referee_usageBRead-onlyIdempotentInspect
Read aggregate, privacy-preserving Referee activity and outcome evidence for the selected audience.
| Name | Required | Description | Default |
|---|---|---|---|
| window_days | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already confirm this is a safe read operation ('readOnlyHint: true'). The description adds critical behavioral context by specifying that the data is 'aggregate' and 'privacy-preserving,' which dictates the nature of the information the agent can expect to retrieve.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, dense sentence, making excellent use of space. It front-loads the most important constraints (aggregate, privacy-preserving) and is highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema, the description provides a reasonable high-level overview of the *type* of data being retrieved (evidence and outcomes). It is functionally complete for a usage-level summary tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the schema provides no explanatory text. The description mentions 'selected audience' but fails to provide any semantic detail or constraints regarding the single input parameter, 'window_days,' leaving the parameter function entirely undocumented by text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Read'), the resource ('Referee activity and outcome evidence'), and the nature of the data ('aggregate, privacy-preserving'). While clear, it doesn't explicitly differentiate this from a sibling tool like 'describe_referee' without external knowledge.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when summary or privacy-focused data is required ('aggregate, privacy-preserving'). However, it offers no guidance on when *not* to use this tool or how it compares to more granular sibling functions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_repair_schemaCRead-onlyIdempotentInspect
Get the feature-only contract for diagnosing degradation and testing bounded repairs.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this as a read-only, idempotent, non-destructive operation, so the safety profile is covered. The description adds context about the contract's intended use (diagnosing degradation and testing bounded repairs), but it does not describe the return format or any behavioral nuances beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, appropriately sized sentence with no wasted words. However, the key scoping information ('feature-only contract') is jargon-heavy rather than front-loaded with clear, actionable detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters, no output schema, and no annotations beyond the safety profile, the description carries the full burden of explaining what the tool returns and how it fits into the repair workflow. It only vaguely gestures at a 'feature-only contract' without clarifying its contents, structure, or relation to sibling tools like get_input_schema or preflight_repair.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and the schema is an empty object. Per the rubric, a parameterless tool receives a baseline of 4 because there are no parameter semantics to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a verb ('Get') and a resource ('feature-only contract'), but the term 'feature-only contract' is jargon that doesn't clarify what the schema actually contains. It does not distinguish this tool from the sibling get_input_schema or explain how the repair schema differs from other schema-like tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a vague purpose ('diagnosing degradation and testing bounded repairs') but provides no explicit guidance on when to use this tool versus alternatives like get_input_schema, preflight_repair, or quote_repair. No prerequisites, exclusions, or call-order hints are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_verticalsBRead-onlyIdempotentInspect
List the three supported evaluation profiles and their discovery tags.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, closed-world and non-destructive, so the description needn't re-litigate safety. It adds the useful fact that the result set is a fixed, small set of three profiles each carrying discovery tags, but says nothing about ordering or the tag format, which matters since there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler. The fixed cardinality ('three') is the most actionable detail and appears early.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description is the sole source of truth about the return value, and it stops at 'profiles and their discovery tags' without saying what a tag looks like or how to use one in a later call. For a tiny discovery tool that is close to adequate, but a single clause about the shape of the response would close the gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing to disambiguate and no risk of misuse; the baseline for a parameterless tool applies. The description correctly doesn't invent arguments.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('List') and resource ('the three supported evaluation profiles and their discovery tags'), and the cardinality ('three') makes the scope concrete. It is distinguishable from the audit/referee/upload siblings, though it never bridges the name 'verticals' to 'evaluation profiles', which is the one thing an agent may wonder about.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No statement of when to call this versus alternatives such as describe_referee or get_referee_usage, and no precondition or typical follow-up. The phrase 'discovery tags' weakly implies it is an entry point, but the agent must infer that.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
preflight_datasetBInspect
Validate a tenant-private upload and its baseline evidence/row identity without spending a credit.
| Name | Required | Description | Default |
|---|---|---|---|
| contract | Yes | ||
| vertical | Yes | ||
| upload_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The no-credit guarantee is useful non-obvious context beyond the annotations. However, with readOnlyHint=false the agent cannot tell whether preflight persists state, and the description says nothing about what is checked, what failure modes exist, or whether results are cached.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler, though its brevity reflects under-specification rather than tight editing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool taking a large nested audit contract with no output schema, the description omits what constitutes a valid upload, what the validation covers, and what the caller receives back. Far too thin for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the contract object carries ~25 nested fields, yet the description only gestures at 'baseline evidence/row identity' and says nothing about upload_id format, the vertical enum, or which contract fields are validated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Validate) and resource (a tenant-private upload plus its baseline evidence/row identity), which distinguishes it from submit_audit and quote_audit. It does not explicitly name those siblings, so an agent must infer the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'without spending a credit' implies this is a cheap pre-check to run before a paid audit, but the description never states when to call it versus submit_audit or quote_audit, nor what a failing preflight means for the next step.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
preflight_repairBIdempotentInspect
Validate a tenant-private feature-repair matrix without spending a credit.
| Name | Required | Description | Default |
|---|---|---|---|
| contract | Yes | ||
| upload_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=false, idempotentHint=true, openWorldHint=false, and readOnlyHint=false. The description adds the important cost-related detail that no credit is spent, but it does not explain what validation actually does or why the operation is not read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words. It communicates the core action and cost constraint immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The input schema is large and complex, with many nested contract fields and no output schema. Given that complexity and 0% schema description coverage, the one-sentence description is too sparse to tell an agent what the validation checks or what to expect back.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the upload_id or the nested contract structure. It only indirectly references the feature-repair matrix, leaving the two required parameters and their many nested fields undocumented in prose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: validate a tenant-private feature-repair matrix. It also adds a useful qualifier, 'without spending a credit,' but does not explicitly distinguish this tool from siblings such as quote_repair or upload_repair_matrix.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'preflight' framing and 'without spending a credit' imply this is a low-cost validation step before a paid repair operation. However, the description does not explicitly state when to use it versus quote_repair, upload_repair_matrix, or other repair-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
quote_auditBIdempotentInspect
Create a contract-bound quote for one audit credit; baseline mode and evidence cannot be substituted.
| Name | Required | Description | Default |
|---|---|---|---|
| contract | Yes | ||
| vertical | Yes | ||
| upload_id | Yes | ||
| idempotency_key | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose that this is a non-read-only but idempotent and non-destructive operation. The description adds meaningful behavioral constraints: the quote is contract-bound and the baseline mode and evidence cannot be substituted after the fact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words. The core action and its key constraint are immediately clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a complex quote-creation operation with a large nested contract schema and no output schema. The description does not explain prerequisites, how upload_id and vertical interact, what the resulting quote represents, or what happens on success. It is too thin for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description references baseline mode and evidence constraints, which adds some meaning around the nested contract. However, it says nothing about the four required top-level parameters: upload_id, vertical, contract, and idempotency_key. With 0% top-level schema description coverage, this leaves too much undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: creating a contract-bound quote for one audit credit. It also adds the scope constraint that baseline mode and evidence cannot be substituted. It is clear, though it does not explicitly distinguish itself from siblings such as quote_campaign or quote_repair.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'for one audit credit' implies this is the single-audit quoting tool, but the description does not explicitly say when to use it versus quote_campaign, quote_repair, or submit_audit. Usage is therefore suggested rather than fully specified.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
quote_campaignCIdempotentInspect
Preflight all private feature shards, time a pilot, and quote credits and ETA for up to 1,000 candidates in one corrected search family.
| Name | Required | Description | Default |
|---|---|---|---|
| shards | Yes | ||
| vertical | Yes | ||
| idempotency_key | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotency, non-destructiveness, and non-read-only behavior. The description adds useful context that the tool preflights all shards, times a pilot, and quotes credits/ETA, but it omits failure modes, side effects, and what preflight actually validates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no filler. It is efficient, though the phrase 'corrected search family' is jargon-heavy and adds ambiguity rather than clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a large nested schema with numerous contract fields and no output schema, yet the description is only one sentence. It leaves key required inputs and the quote-vs-submit workflow unexplained, making it incomplete for an agent to call confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it only loosely maps to the shards concept and candidate count. It does not explain the required vertical enum, the idempotency_key, or the shard structure and constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names specific actions (preflight, time a pilot, quote credits and ETA) and a clear resource scope (private feature shards and up to 1,000 candidates). It does not explicitly differentiate itself from sibling quoting tools like quote_audit or quote_repair, which keeps it below a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus quote_audit, quote_repair, or preflight_dataset. The only constraint-like information is a candidate limit, not a usage rule or prerequisite.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
quote_repairBIdempotentInspect
Preflight and quote label-based feature repair; large jobs return a measured ETA and credit requirement.
| Name | Required | Description | Default |
|---|---|---|---|
| contract | Yes | ||
| upload_id | Yes | ||
| idempotency_key | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false, so the safety profile is largely covered. The description adds useful context that large jobs return a measured ETA and credit requirement, but it leaves unclear what 'large' means and does not explain how the quote is persisted or whether it consumes credits at quote time.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no filler. Every clause adds either purpose or an output behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The schema is rich and annotations cover safety/idempotency, so the description does not need to repeat every field. However, for a tool requiring an upload_id, a complex contract, and an idempotency_key, the description omits important preconditions and usage context that would help an agent invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low at 0%, so the description should compensate by explaining the required upload_id, contract, and idempotency_key parameters. It mentions none of them and adds no parameter semantics beyond the tool's broad domain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb pair and resource: 'Preflight and quote label-based feature repair.' An agent can tell this is a quoting/preflight tool for feature repair, but it does not explicitly distinguish itself from siblings such as preflight_repair or get_repair_schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Preflight and quote' implies this should be used before running a repair, but there is no explicit when-to-use guidance, no when-not-to-use guidance, and no named alternative such as preflight_repair.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
record_audit_followupCIdempotentInspect
Record whether a completed audit changed a decision or was confirmed prospectively.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | ||
| value | Yes | ||
| audit_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=true and destructiveHint=false, so the safety profile is covered. The description adds the useful but implicit precondition that the audit must be 'completed', yet says nothing about what happens on re-record (overwrite vs. reject despite idempotency) or that the value recorded is a durable verdict attached to the audit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence, zero filler. It is short to the point of under-specification rather than padded, which is the lesser sin.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A three-parameter write tool with no output schema, no parameter descriptions, and two enums whose relationship is undocumented. The description carries none of the missing load, leaving the agent to infer valid kind/value combinations and audit lifecycle state from the schema alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description names none of the three parameters. Most critically it never explains that the eight 'value' enums are partitioned by 'kind' (decision_impact values vs. prospective_outcome values), which is exactly the information an agent needs to avoid an invalid pairing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Record') and resource (audit follow-up), and the phrase 'whether a completed audit changed a decision or was confirmed prospectively' maps onto the two enum kinds, so the agent can tell this is a write of a follow-up verdict rather than a read or a submission. It does not explicitly separate itself from verify_audit_report or submit_audit, but the resource noun is distinctive enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance, and no alternatives named among the 25 siblings. The only weak signal is the word 'completed', which implies a precondition but is never stated as one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
redeem_beta_inviteAIdempotentInspect
Redeem one operator-issued invite for free beta audit credits before January 1, 2027.
| Name | Required | Description | Default |
|---|---|---|---|
| invite_code | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose that this is a non-read-only, idempotent, non-destructive, closed-world write. The description adds useful behavior beyond that: the outcome (free beta audit credits), that it is a single invite, and the expiry date. It does not say whether the code is consumed on success, whether re-redeeming the same code re-grants credits (consistent with idempotentHint), or what failures look like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and outcome, with the deadline as the closing constraint. Every clause earns its place and nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, no-output-schema tool, the description covers the prerequisite, the effect, and the validity window, which is enough to call it correctly. The remaining gap — error behavior and post-success state (credits visible via get_credits) — is minor but real.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must carry the parameter meaning, and it only adds that the code is 'operator-issued' — it never explains where the agent obtains one or how to respond to a malformed code. The regex pattern in the schema does define the accepted format, which partially compensates, warranting the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (redeem), a specific resource (an operator-issued beta invite), and the resulting benefit (free beta audit credits), plus a hard deadline. It is clearly distinct from request_beta_access and get_beta_request_status by the fact that it consumes an already-issued code, though that contrast is implied rather than named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The prerequisite (an operator-issued invite) and the expiration window (before January 1, 2027) are stated, which tells the agent when this applies. There is no explicit guidance on what to do instead when the agent has no code (e.g., use request_beta_access) or how to handle an expired/invalid invite.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
request_beta_accessCIdempotentInspect
Request operator-reviewed beta access without purchasing or granting credits.
| Name | Required | Description | Default |
|---|---|---|---|
| lane | Yes | ||
| use_case | Yes | ||
| idempotency_key | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly=false, idempotent=true, destructive=false, so the safety profile is covered. The description adds one genuinely new fact: the request is 'operator-reviewed', implying asynchronous human approval rather than instant granting. It still doesn't say the outcome is a pending request needing later status checks.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tightly-worded sentence with the key constraint front-loaded and no filler. Appropriately sized for the text it contains, though that brevity reflects omission rather than density.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A non-read-only submission tool with three required enum parameters, no output schema, and no indication of what is returned or what the caller should do next (poll status, wait for approval). For a mutation creating an operator-reviewed request, this leaves major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and all three required parameters (lane, use_case, idempotency_key) are undocumented — two are enums whose option meanings are not explained. The description contributes nothing about what any parameter means or how to choose among values, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb + resource: 'Request operator-reviewed beta access'. The clause 'without purchasing or granting credits' rules out a plausible misreading that this tool spends credits. It doesn't name its closest siblings (redeem_beta_invite, get_beta_request_status), so a 4 rather than a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. An agent cannot tell from the description whether this is the right entry point versus redeem_beta_invite, or whether get_beta_request_status is the follow-up. The only implicit guide is the 'without purchasing' clause.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
submit_auditCIdempotentInspect
Accept a quote and atomically reserve its disclosed audit credits.
| Name | Required | Description | Default |
|---|---|---|---|
| quote_id | Yes | ||
| idempotency_key | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-read-only, idempotent, non-destructive, closed-world, so the safety profile is covered. The description usefully adds that credit reservation is atomic, implying all-or-nothing spending. It does not say what happens on failure, whether reserved credits are released, or how the idempotency_key governs replay.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence with no filler; the side effect is placed where the reader needs it. It is, however, terse to the point that the tool is under-described, so brevity here is borderline under-specification rather than clean economy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a mutating, credit-spending operation with no output schema and no parameter descriptions, yet the description omits the result (audit id, receipt, status), failure semantics, and credit-release behavior. Given sibling tools like get_audit_receipt and get_audit_status, an agent is left guessing what it gets back.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the full burden for two parameters, yet it never explains quote_id (a 32-hex quote reference) or why idempotency_key exists (replay-safe retries). The schema supplies patterns and length bounds but no meaning, and the description does not compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource pair ('accept a quote') plus the concrete side effect ('atomically reserve its disclosed audit credits'), which implicitly separates it from quote_audit and delete_audit. It does not name any sibling explicitly, so the differentiation is inferable rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to call this versus quote_audit, get_audit_receipt, or delete_audit, and no prerequisites spelled out (e.g., that a quote must first be obtained and not expired). The only hint is the required quote_id parameter, which leaves the ordering and retry policy to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
upload_campaign_shardCIdempotentInspect
Encrypt one of up to 64 private feature shards for a large campaign; retain for at least 4,200 seconds.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | ||
| format | No | csv | |
| retention_seconds | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnly=false, idempotent=true, destructive=false, so the safety profile is covered; the description should add context about what happens to data or what constraints apply. Instead, its 'retain for at least 4,200 seconds' statement conflicts with the schema's retention_seconds minimum of 60, which actively misleads an agent configuring a short-lived shard. The encryption claim is also unexplained (at-rest? in transit? returned key?).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single compact sentence with the scope constraint front-loaded and no filler. It is efficient, though efficiency here reflects under-specification rather than disciplined editing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A write tool with no output schema, 0% parameter coverage, and no annotation coverage of encryption or payload handling means the description carries the full burden, and it does not meet it. It omits payload size/format expectations, the shard-count limit's practical meaning, and the relationship between the stated retention floor and the schema's actual bounds.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and none of the three parameters are named or explained: the 5,000,000-char data payload, the csv/json format switch, and the 60-86400 retention range with a default of 86400. The one parameter the description does touch, retention, is described with a value (4,200s) that contradicts the schema's stated minimum.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Encrypt one of up to 64 private feature shards') with useful scope information about campaigns. However, it never reconciles the 'encrypt' framing with the tool name 'upload_campaign_shard', so an agent cannot tell whether the tool stores bytes, encrypts them, or both, and it never contrasts with the sibling upload_private_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives operational constraints (up to 64 shards, large campaigns), but no explicit when-to-use/when-not guidance and no pointer to alternatives such as upload_private_dataset or upload_repair_matrix, which sit right next to it. The agent must infer selection from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
upload_private_datasetBIdempotentInspect
Encrypt a de-identified CSV/JSON payload for a short tenant-private retention window.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | ||
| format | No | csv | |
| retention_seconds | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotent=true, destructive=false, and closed-world, so safety is covered. The description adds that data is encrypted and retained only for a short tenant-private window, which is useful context, but says nothing about auth requirements, what happens after retention expiry, or the response.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with zero filler, and the encryption/retention framing is front-loaded. Slightly under-specified rather than bloated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a write tool with no output schema and 0% parameter description coverage, one sentence is insufficient: it omits what the call returns, whether de-identification is enforced or merely assumed, and any sizing/retention limits. The agent knows what it does but not enough to call it confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It loosely maps to all three params — payload (data), CSV/JSON (format), retention window (retention_seconds) — but omits concrete constraints like the 5MB data cap and the 60–86400 second retention bounds that the schema enforces.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action (encrypt) on a specific resource (de-identified CSV/JSON payload) with a stated retention scope. It is distinguishable from sibling uploads like upload_campaign_shard and upload_repair_matrix, though 'Encrypt' rather than 'upload/store' slightly blurs the primary verb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to choose this over the other upload_* siblings, no prerequisites, and no exclusions. The agent must infer the use case from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
upload_repair_matrixCIdempotentInspect
Encrypt one complete de-identified repair matrix of up to 100 MB and 1,000 diagnostic features.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | ||
| format | No | csv | |
| retention_seconds | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose a non-read-only, non-destructive, idempotent, closed-world write, so the safety profile is covered. The description adds useful context that data is encrypted and de-identified and is capped at 100 MB / 1,000 features, but says nothing about retention semantics, whether a re-upload replaces a previous matrix, or what happens on validation failure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the constraints arrive before anything else. It is tight but arguably over-compressed given the undocumented parameters that could have been covered in the same sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should say what an upload yields (an upload/matrix identifier usable by quote_repair or preflight_repair), and it should cover format and retention defaults. For a mutating ingest tool with 0% schema coverage, this is under-specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the full burden. It clarifies the size ceiling for 'data' (100 MB maps to the maxLength) but leaves 'format' (csv/json) and 'retention_seconds' (4200–86400, default 7200) completely unexplained — the agent gets no hint of the allowed file formats or retention window from either source.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource (a complete de-identified repair matrix) with concrete boundaries (100 MB, 1,000 diagnostic features), which separates it in spirit from upload_private_dataset and upload_campaign_shard. The verb 'Encrypt' diverges from the tool name 'upload_repair_matrix' and leaves mild ambiguity about whether encryption, upload, or both occur.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No indication of when this should be used versus upload_private_dataset, upload_campaign_shard, or preflight_repair, and no mention that a preflight check is the typical prerequisite. The agent must infer the entire workflow from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
verify_audit_reportARead-onlyIdempotentInspect
Compare a locally computed SHA-256 with the retained tenant-private receipt. Use the exact downloaded result JSON bytes (not the API envelope), or UTF-8 result.report text with representation report_text. Never copy the expected hash as the candidate: compute it from the supplied artifact. A digest match does not validate a summary, factual correctness, or absence of harm. No report is uploaded.
| Name | Required | Description | Default |
|---|---|---|---|
| sha256 | Yes | ||
| audit_id | Yes | ||
| representation | No | result_json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only, idempotent, non-destructive, and closed-world behavior. The description adds meaningful context beyond annotations by stating that no report is uploaded and that a digest match does not validate factual correctness or absence of harm.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four tight sentences, front-loaded with the core operation and followed by precise usage constraints and scope limits. Every sentence adds necessary information with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers privacy and verification limits well, but there is no output schema and the description never states what the tool returns, such as a boolean match result or mismatch details. It also leaves audit_id unexplained despite low schema description coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must carry parameter meaning and does so for the two ambiguous parameters: it explains that sha256 must be computed from the supplied artifact and that report_text selects UTF-8 result.report text. It does not explain audit_id beyond the schema pattern, which keeps it from a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: compare a locally computed SHA-256 against the retained tenant-private receipt. An agent can distinguish this from receipt-fetching siblings like get_audit_receipt, though the description does not explicitly name those alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear usage instructions: use the exact downloaded result JSON bytes or UTF-8 result.report text with representation report_text, and never copy the expected hash as the candidate. It lacks explicit when-not-to-use guidance relative to sibling tools such as get_audit_result.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
26 tool updates
- First observed
delete_audit - First observed
delete_upload - First observed
describe_referee - First observed
get_audit_receipt - First observed
get_audit_result - First observed
get_audit_status - First observed
get_beta_request_status - First observed
get_credits - First observed
get_input_schema - First observed
get_referee_readiness - First observed
get_referee_usage - First observed
get_repair_schema - First observed
list_verticals - First observed
preflight_dataset - First observed
preflight_repair - First observed
quote_audit - First observed
quote_campaign - First observed
quote_repair - First observed
record_audit_followup - First observed
redeem_beta_invite - First observed
request_beta_access - First observed
submit_audit - First observed
upload_campaign_shard - First observed
upload_private_dataset - First observed
upload_repair_matrix - First observed
verify_audit_report
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
Pay-per-use tool marketplace for AI agents. Search, price-check, and call APIs via MCP.
44-tool MCP server for AI compliance and trust checks. Pay-per-call, no signup.
The OpenRouter for tools. One MCP connection gives any AI agent 254 hosted tools, pay per call.
Related MCP Servers
- FlicenseAqualityDmaintenanceHigh-value scientific tools for AI agents — literature search (PubMed, arXiv, Semantic Scholar), chemical compound lookup (PubChem, ChEMBL), patent prior art search (USPTO, EPO), GPU spot prices across 4 providers, and real-time earth science data (USGS, NASA, OpenAQ). Per-call billing via API key. Keys issued instantly at https://mcp-site.com/keys/request6-
- AlicenseAqualityDmaintenanceDeterministic time-series statistics for AI agents. This MCP server gives any LLM agent unit-tested statistical tools — anomaly detection, changepoint detection, seasonal decomposition, stationarity/trend tests, data-quality audits, baseline forecasts — with schema-validated structured output and no arbitrary code execution.17MIT
- AlicenseAqualityCmaintenanceMCP server exposing statistical regression testing for LLM agents as a "run" tool: p-value, effect size, and confidence interval on whether agent behavior actually changed.162 PyPIApache 2.0

HumanJudgeofficial
AlicenseNot gradedqualityBmaintenanceHuman-evaluation infrastructure for AI quality. 25,000+ blind human reviews by 200+ verified reviewers across 58 AI models — query the data via five MCP tools (get_model_scores, compare_models, get_flags, check_content, get_latest).2MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.