Server Quality Checklist

Tool Scores

• iceberg_api_list_versionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so safety is covered. The description adds only a hint about what is listed (stable/nightly identities, optional revision) but no behavioral context such as configuration source, error behavior, or side effects. With annotations present, the description adds minimal new transparency value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that names the action first and then specifies the resource scope. Every phrase adds information: 'configured' indicates server-side state, 'stable/nightly' describes the version types, and 'optional local source revision' clarifies an additional return component. No filler or repetition of the title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only listing tool with an output schema and safety annotations, the description is nearly adequate. It covers what will be listed and that local revision is optional. The main gap is the absence of any connection to the broader workflow (e.g., why an agent would call this before browsing/searching), but this is a minor omission given the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema's description coverage is 100% (trivially satisfied), so no parameter documentation is needed. The description's mention of 'configured' and 'optional local source revision' clarifies that inputs come from configuration rather than call parameters, which is a useful, non-redundant cue. Baseline 4 is appropriate for a zero-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a concrete action ('List') and a specific resource ('configured stable/nightly Javadoc identities and optional local source revision'), making the tool's purpose discernible. It is distinguishable from sibling tools like iceberg_api_browse and iceberg_api_search because it targets version identity listing. However, 'Javadoc identities' is somewhat specialized and not fully self-explanatory, so not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use or when-not-to-use guidance is provided. The description does not reference any sibling or alternative, nor does it suggest a scenario in which an agent should choose this tool over iceberg_api_compare_versions or iceberg_api_get_type. The agent must infer applicability solely from the name and general wording.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_api_searchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds useful behavioral traits—'ranked' and 'case-insensitive'—but does not elaborate on pagination, ranking order, or result limits. It adds some value but lacks rich behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately communicates the core behavior ('Ranked case-insensitive search'). Every word earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema, annotations, and several self-describing parameters, the description is mostly complete for invoking the tool. The main gap is the absence of usage differentiation among several search/browse siblings, which would help an agent decide when to choose this tool. Otherwise, the search scope and behavior are adequately conveyed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67%, so the baseline is 3. The description does not explain individual parameters; 'packages, types, member names, signatures, and identities' loosely maps to query and scope meaning, but adds little beyond the schema's own parameter descriptions. No additional semantic depth is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('search') and resource ('published Iceberg packages, types, member names, signatures, and identities'), giving a precise scope. It clearly distinguishes this from sibling tools like browse or get_type, even if it doesn't explicitly contrast with iceberg_source_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives such as iceberg_api_browse, iceberg_api_get_type, or iceberg_source_search. The intended usage is only implied by the name and search semantics; no exclusions, conditions, or routing cues are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_api_browseA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the bar is lower. The description adds minor context by noting the index is 'complete published' and filters are exact, but it does not disclose additional behavioral details like pagination behavior or response structure beyond what annotations and schema already provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the primary action and resource, then immediately mentions the key filters. There is no wasted wording, and it is appropriately short for an API browse tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and annotations covering safety, the description is largely complete for a read-only browse operation. It clearly states the resource and filters, though it could have been more complete with an explicit usage note contrasting it against iceberg_api_search or iceberg_api_get_type.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 83%, and the schema already documents kind, prefix, package_name, version, and cursor. The description's mention of 'package and label-prefix filters' aligns with package_name and prefix parameters but does not add additional semantic detail beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Browse' with a clear resource: the complete published package, type, or member index. The mention of exact package and label-prefix filters distinguishes this from sibling tools like search or get by emphasizing it covers the full index rather than specific queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through 'complete published index' and 'filters', suggesting it is for browsing the full index rather than targeted search, but it never explicitly states when to choose this tool over alternatives. No exclusion criteria or direct comparison to siblings is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_api_compare_versionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive behavior. The description adds only the qualifiers 'exact' and 'published' but does not disclose operational details such as pagination behavior, rate limits, or output semantics. With annotations covering the safety profile, this is adequate but not richly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly written sentence with no filler. It front-loads the core action and scope, and every word contributes to distinguishing the tool from its siblings.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the annotations, output schema, and usable schema descriptions for the required version parameters, the overall tool definition is mostly complete. A small gap remains: package_name and the exact meaning of 'identities' in terms of required inputs are not fully explained, but this does not prevent a competent agent from invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50%, covering from_version, to_version, and cursor. The description helps with 'package, type, or member' which maps to the kind enum and package_name, but it does not clarify limit behavior, cursor handling, or how package_name interacts with kind. The description partially compensates for the schema gaps but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Compare'), names the exact resource ('published package, type, or member identities'), and defines the scope ('between two Iceberg Javadoc versions'). This clearly differentiates it from siblings like iceberg_api_search, iceberg_api_get_type, and iceberg_api_list_versions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the intended use clear: compare exact identities across two published versions. It does not explicitly name alternatives or exclusion cases, but the cross-version comparison purpose is strong enough that an agent can infer when to select it over single-version get/browse/search tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_api_get_typeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish readOnlyHint=true and idempotentHint=true, so the description builds on that rather than repeating it. It adds meaningful behavioral detail about the payload shape: declaration, description, deprecation, detailed members, links, and stability evidence. It does not mention pagination behavior for member_limit/member_cursor, but this is minor given the existing output schema and annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every element carries information: exactness, publication status, and the returned content fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only lookup tool with rich annotations, a 75%-covered schema, and an output schema, the description covers what the agent needs to know about the return scope and the one required input. It lacks explicit routing guidance between iceberg_api_get_type and the source/member siblings, but that does not make invocation incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75%, so the high baseline of 3 applies. The description does not explain parameters, but it reinforces that fully_qualified_name is the target and 'members' relates to member_limit/member_cursor. No parameter detail beyond the schema is provided, and none is strictly needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (Get) and resource ('one exact published Java type') and enumerates the returned content (declaration, description, deprecation, detailed members, links, stability evidence). 'Exact' and 'published' distinguish it from the search/browse siblings and from iceberg_source_get_type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'exact' implies this tool is for lookups when the fully-qualified name is already known, and 'published' implies released Javadoc rather than source. However, it never explicitly states when to prefer it over iceberg_api_search, iceberg_api_get_member, or iceberg_source_get_type, nor does it state any alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_source_find_implementationsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, non-destructive, and closed-world behavior. The description adds meaningful behavioral context beyond the annotations: results are 'bounded lexical' and 'evidence, not semantic compiler analysis', which warns the agent about the tool's precision and scope. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One tightly written sentence that front-loads the core operation and immediately states the key limitation. Every word earns its place; there is no filler or redundant restatement of the title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema, return-value documentation is not the description's burden. The description covers the tool's purpose, its lexical scope, and its 'evidence, not semantics' limitation, which are the most important contextual facts. It could be slightly stronger by naming a sibling alternative or clarifying parameter input format, but it is otherwise sufficient for a read-only lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, so the description needed to compensate for the undocumented fully_qualified_name and limit parameters. It adds only the hint that the type is a 'Java type', which helps with fully_qualified_name, but says nothing about the format, the meaning of limit, or how cursor pagination works. This is insufficient given the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('find') and resource ('bounded lexical extends/implements declarations for a Java type'), and clarifies that the result is lexical evidence rather than semantic compiler analysis. This clearly distinguishes the tool from siblings like iceberg_source_get_type and iceberg_source_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this when you need lexical extends/implements declarations, not when you need semantic compiler-accurate hierarchy analysis. It does not explicitly name alternative tools or provide when-not-to-use exclusions beyond the lexical/semantic distinction, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_api_get_memberA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is well covered. The description adds exact-match selection semantics but no further behavioral context such as not-found behavior, version-specific differences, or response characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action and resource, then packs the identifier forms and scoping constraint with zero filler. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Together with the input schema, output schema, and annotations, the description provides enough for an agent to invoke this exact-lookup tool correctly. It does not explicitly situate itself against sibling tools, but the exactness wording and 'member' resource make the intended use clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documents member and version, but fully_qualified_name has no description. The phrase 'within an exact Java type' helps clarify that fully_qualified_name is the exact type scope, and 'published label or Javadoc anchor' reinforces the member parameter. A concrete example would have made this stronger.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get'), a specific resource ('one exact member'), and a precise scoping constraint ('within an exact Java type'). It also differentiates this from search or browse by emphasizing exactness and the accepted identifiers ('published label or Javadoc anchor').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is the tool for exact single-member lookup when the label or anchor and exact Java type are already known. It does not explicitly name alternatives like iceberg_api_search or iceberg_api_browse, so it falls short of a full when-to-use/when-not-to-use statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_source_get_typeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, non-destructive behavior, so the description does not need to repeat safety guarantees. It adds useful behavioral context beyond annotations: the result is bounded by line window and line-numbered, and the lookup depends on an indexed Java type in the configured checkout.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. Every qualifier ('bounded', 'line-numbered', 'indexed', 'configured') carries meaning and contributes to accurate invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema and strong safety annotations reduces the need for the description to explain return values or side effects. The description supplies the essential context (source window, type-based lookup, fixed checkout), though it could optionally state the FQN form or default window behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The phrase 'line-numbered source window' loosely conveys start_line/line_count, and 'Java type' maps to fully_qualified_name, but the description does not specify the expected FQN format or explicitly define how the window parameters interact. It adds some semantic value but not complete compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Read'), a distinct resource ('bounded, line-numbered source window'), and a clear subject ('an indexed Java type from the configured Iceberg checkout'). This makes it immediately distinguishable from API metadata tools and source-search/find-implementation siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly communicates the intended use: retrieving a source code slice by Java type name from the configured checkout. It does not explicitly name alternatives or exclusion conditions, but the 'source window' framing and indexing qualifier provide enough context to know when this tool applies versus API/library search tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iceberg_source_searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only, idempotent, and non-destructive. The description adds that the search is bounded (limit applies), literal rather than regex, and scoped to indexed production Java files, giving useful operational context beyond what the annotations already provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tightly worded sentence that front-loads the verb and packs in the resource, scope, and search type. There is no redundancy or filler, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only search tool with an output schema, the description covers what is searched, how it is searched (literal), and the bounded nature of results. It would benefit from explicit routing to sibling API search tools, but nothing essential for invoking the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one of three schema properties (cursor) has a schema description, so the description must compensate for the other two. It does so by clarifying that 'literal' is a literal search string and that the search is 'bounded', which maps to the limit parameter; cursor is already documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('search'), a specific resource ('indexed production Java source files'), and a key constraint ('bounded literal (not regex)'). This clearly separates it from sibling iceberg_api_search (API docs) and iceberg_source_find_implementations (implementation discovery), making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys that the tool is for literal source-code search and explicitly notes it is not regex, implying a regex search would need a different tool. However, it does not name any sibling tool or state when to choose API search vs source search, leaving some routing guidance implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.