Server Quality Checklist

Tool Scores

• search_indicesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral transparency burden. It states that the tool 'returns matching catalog entries', which indicates a read-only search behavior. However, it does not disclose potential error behavior, rate limits, or what happens on zero matches, so it provides only basic behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and the example is illustrative without being verbose. Every sentence adds value, with no fluff or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 params, no output schema), and the description covers the main purpose and usage context. However, it does not specify the exact return format (list of slugs vs. full entries) or how the limit parameter affects results, which are important for an agent to use it correctly without additional assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (q is described, limit is not). The description adds an example for q but entirely omits any explanation of 'limit', leaving its meaning and impact unexplained. With only partial coverage, the description should compensate but does not, leaving a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Resolve' with a clear resource ('free-text query' to 'canonical FLOPS index slugs'), and distinguishes itself from siblings like get_index and list_indices by focusing on free-text search. The example 'H100 spot' clarifies the intended use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use when you don't know the exact slug', which gives clear context for when to invoke this tool over alternatives. However, it does not name alternatives directly or provide explicit when-not-to-use guidance, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_indexA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses that the payload is 'PUBLIC' and 'source-opaque,' describes the exact return shape, and introduces the resource alias. While it doesn't discuss auth or rate limits, the non-mutating 'Resolve' semantics and public framing provide strong behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three dense sentences with no filler: it states the core action, lists the return fields, gives a usage preference, and mentions the MCP resource alias. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description is remarkably complete. It covers the return format, the intended use case, the public nature of the payload, and even the resource URI equivalent. No critical operational context appears missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers the single parameter fully with description and an example, so schema coverage is 100%. The description does not add parameter-level detail, which is acceptable because the schema handles it; the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Resolve') and clearly identifies the resource ('a FLOPS index to its source-opaque PUBLIC payload'), explicitly framing it as the 'public citation contract.' This distinguishes it from siblings like get_price and verify by focusing on the citation-oriented payload.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance: 'Prefer this when you intend to CITE the value.' It does not explicitly name alternative tools or provide when-not-to-use conditions, but it establishes a clear use case in context of sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_priceA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses important behavioral traits: values are 'delayed and indicative' and the envelope is returned 'as-is.' This goes beyond a simple fetch description, though it does not cover error handling or rate limits. The key user-relevant caveats are addressed, so it earns a 4.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each with a distinct purpose: stating the action, enumerating the return envelope, and pointing to the alternative. No filler or redundancy; information is front-loaded and efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description explicitly lists the 14 fields returned, which fully covers the return structure. It also adds context about data freshness (delayed, indicative) and directs to get_index for a lighter payload. For a simple one-parameter fetch tool, this is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage of the single 'slug' parameter with a clear description and examples. The description does not add additional parameter semantics beyond what the schema offers. Per the rubric, baseline 3 is appropriate when schema coverage is high and the description adds little.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Fetch the current published value for a FLOPS compute price index' with a specific verb and resource. It distinguishes itself from the sibling get_index by clarifying that get_price returns the full envelope while get_index returns a reduced, citation-only payload. This makes the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs the user to an alternative: 'Use get_index for the reduced, citation-only payload.' This establishes when to use get_price (when the full 14-field envelope is needed) versus get_index, providing clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_indicesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the full transparency burden. It discloses the return shape ({count, indices[]} with field details) and auth status ('No auth required'). It does not mention edge cases or rate limits, but for a read-only list operation the disclosure is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, then return format, usage guidance, and auth note. No wasted words; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one optional parameter and no output schema, the description is complete. It explains the return structure, usage context, and authentication, which is sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the one parameter; the schema already fully describes family_filter. The description does not add extra semantic meaning beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List all public FLOPS compute-price indices.' It clearly distinguishes this list-all tool from siblings like get_index or search_indices by stating its role as a discovery mechanism.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is given: 'Use this to discover available indices before calling get_index or verify.' This names specific alternatives and provides ordering context, making it clear when to pick this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verifyA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden. It discloses the exact response fields (`verified`, `expected`, `actual_value`, `delta_pct`), the meaning of `verified: null` ('not checked', NOT 'correct'), and the rounding/delay behavior of anonymous access. This goes far beyond a simple action statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet comprehensive. It front-loads the core action, then logically covers parameter behavior, response fields, and important caveats. Every sentence adds value; there is no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's inherent complexity (subtle null meaning, anonymous data rounding/delay, multiple response fields) and the absence of an output schema, the description provides a complete picture. It even includes an example value format and explains the delta percentage. No critical behavioral aspect is left unexplained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful semantics: it explains the optional `value`'s purpose ('a number you are about to cite'), the consequence of omitting it, and the resulting `verified` null behavior. This enhances the schema's minimal 'Optional. The number you want checked.'

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Check a FLOPS index value against the published index.' It clearly distinguishes the verification use case from a plain lookup, and its mention of omitting `value` to do a lookup differentiates it from sibling tools like get_index. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear conditional guidance: use `value` when about to cite a number, omit it for a plain lookup. It explains the anonymous public-value caveat. It does not explicitly name sibling alternatives (e.g., 'use get_index for lookup'), but the context is sufficient for an agent to decide when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.