Server Quality Checklist

Tool Scores

• search_3gpp_docsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the search functionality but lacks critical details: it doesn't specify if this is a read-only operation, what the output format looks like (though an output schema exists), whether there are rate limits, or how results are ranked. The description is minimal and doesn't compensate for the absence of annotations, leaving behavioral traits largely undefined.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of a single sentence that directly states the tool's function. It includes relevant examples (e.g., document types) without unnecessary elaboration. Every word earns its place, making it efficient and easy to parse for an AI agent, with no wasted information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, 1 required), 100% schema description coverage, and the presence of an output schema, the description is somewhat complete but has gaps. It adequately covers the basic purpose but lacks usage guidelines and behavioral details. The output schema likely handles return values, reducing the need for description there, but the absence of annotations and insufficient behavioral context lowers the score to a minimal viable level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, providing clear details for all three parameters (query, spec, maxResults). The description adds minimal value beyond the schema by listing example document types, but it doesn't elaborate on parameter usage, such as how the 'spec' filter interacts with the query or the implications of 'maxResults'. Given the high schema coverage, the baseline score of 3 is appropriate, as the description doesn't significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching 3GPP specification documents by keywords, with specific examples of document types (TS 24.008, TS 24.301, TS 24.501, TS 36.300). It distinguishes from sibling tools like 'get_emm_cause' (which likely retrieves specific EMM causes) and 'list_specs' (which likely lists available specifications) by focusing on keyword search functionality. However, it doesn't explicitly differentiate from potential overlapping search tools beyond the scope of 3GPP documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer this over 'get_emm_cause' (e.g., for broader searches vs. specific cause retrieval) or 'list_specs' (e.g., for content search vs. metadata listing). There's also no information about prerequisites, such as required authentication or access rights, leaving usage context implied but unspecified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_emm_causeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description states it 'gets' information, implying a read-only operation, but doesn't specify any behavioral traits like authentication needs, rate limits, error handling, or what 'detailed information' entails. For a tool with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, with every part contributing to understanding the tool's function. No waste or redundancy is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple lookup with two parameters), high schema coverage (100%), and the presence of an output schema (implied by context signals), the description is reasonably complete. It covers the core purpose and scope. However, without annotations, it lacks behavioral context, and the absence of usage guidelines reduces completeness for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any meaning beyond what the schema provides—it mentions 'cause number' and 'network type' generically but doesn't elaborate on semantics, valid ranges beyond examples, or usage nuances. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get detailed information about EMM cause (LTE) or 5GMM cause (5G) values'. It specifies the verb ('Get'), resource ('detailed information'), and scope (LTE/5G cause values), though it doesn't explicitly differentiate from sibling tools like list_specs or search_3gpp_docs, which appear to be more general documentation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or other contexts, leaving the agent to infer usage based on the tool name and parameters alone. There's no explicit when/when-not or alternative tool recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_specsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a listing operation, implying read-only behavior, but doesn't mention any constraints like pagination, rate limits, or what 'available' means (e.g., only active specs). This leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's purpose without any fluff or redundancy. It's front-loaded and efficiently communicates the essential information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters, 100% schema coverage, and an output schema exists, the description is minimally adequate. However, as a listing tool with no annotations, it lacks details on behavioral aspects like result format or limitations, which could be helpful despite the output schema covering return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, earning a high baseline score for not adding unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('available 3GPP specifications in the database'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'search_3gpp_docs', which likely offers filtering capabilities, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_3gpp_docs'. It lacks context about whether this lists all specifications without filtering or if it's the default listing tool, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.