Server Quality Checklist

Tool Scores

• get_archiveB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It says it returns 'full metadata' and notes schema card existence, but doesn't disclose the return format despite an output schema existing, doesn't describe failure modes (nonexistent id), read-only nature, or access requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (two sentences plus an Args section). It's appropriately sized for a single-parameter retrieval tool. Minor inefficiency: 'Args' section slightly redundant given the one parameter, but the sourcing hint justifies it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-param tool, it's fairly complete. An output schema exists, so return format is covered there. Gaps: no guidance on alternatives among siblings, no mention of error handling for invalid ids, and no explicit read-only disclosure (though annotations are absent). Reasonable but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains the single parameter 'id' is 'Archive id from search_archives', which tells the agent exactly where to source the value — meaningful semantic context beyond the bare property name 'Id' in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb+resource ('Get full metadata for one archive by id') and distinguishes itself by specifying the scope (single archive by id, includes schema card existence). It's clear this is an individual-item retrieval tool, distinct from search_archives which lists/filters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only usage guidance is 'id: Archive id from search_archives' which indicates where ids come from. There's no guidance on when to use this vs. get_schema_card, list_archive_files, or get_data_access, no stated alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_archive_filesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It honestly discloses this is a 'sample' of files rather than a complete listing, which is valuable behavioral context. However, it doesn't mention potential read safety implications, permission needs, or rate limits—though it's clearly a read operation given 'list' semantics and preview behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences plus a compact docstring with Args. Efficient and front-loaded with the core purpose. The only minor waste is restating parameter info that also appears in the schema (limit default), but overall it's lean.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the has-output-schema flag is true (relieving the description of explaining return values) and only 2 params, the description is reasonably complete. The 'sample' disclosure for a listing tool is appropriate. But with no annotations and no explicit workflow guidance connecting it to siblings, there's room to add more about how it composes with get_archive or search_archives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does explain 'id' as 'Archive id from search_archives' and 'limit' with default/max values, which adds meaning beyond the bare schema. However, it doesn't clarify what the limit units are or format expectations for id beyond the pointer to search_archives.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource purpose: 'List a sample of the files/objects in an archive's storage.' This distinguishes it from siblings like search_archives and get_archive by specifying it lists files within a single archive rather than searching across archives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says id comes from search_archives, providing a useful prerequisite for sourcing the id. However, it doesn't explicitly state when to use this vs get_archive or get_schema_card, or any exclusions. The context implies it complements get_archive (which likely returns archive metadata) but doesn't say so.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_data_accessA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description mentions a fallback behavior ('or generated as a fallback') when no schema card snippet exists, which is useful. However, it doesn't disclose what the output structure looks like, whether network calls are made for snippet generation, potential failure modes for invalid archive ids, or rate limit behavior. For a read tool with no annotations, more transparency about the output/return value would help.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized—a clear opening sentence, a usage context sentence, and parameter explanations. No wasted words, everything earns its place. Minor deduction for not being front-loaded with the most critical behavior in the very first sentence, though the first sentence is quite informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (which covers return value explanation), 2 params, and moderate complexity. The description covers purpose, usage timing, and parameter semantics. The fallback behavior is disclosed. It's reasonably complete for a read-oriented access-tool. Slight deduction because the language parameter's accepted values aren't exhaustively listed and there's no mention of what happens if the archive id is invalid or if snippets are unavailable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does: 'id' is explained as 'Archive id from search_archives,' clarifying the data source, and 'language' is described as 'Optional filter for the snippet language' with examples. Since there is no output schema coverage and no enums constraining language values, the examples ('datasets', 's3fs', 'gcsfs') add meaningful value beyond the minimal schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: retrieves storage location and copy-paste code snippets for loading a dataset in one's own environment. The resource is clearly identified (dataset access), and the context—where to run analysis versus hosted notebook—adds specificity. It distinguishes from siblings like get_schema_card and list_archive_files by focusing on loading data locally.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this after finding a dataset when you want to run analysis where you are, rather than in a hosted notebook.' This clearly indicates when to use it. It doesn't explicitly name alternative tools or exclude other scenarios, which keeps it from a 5, but the context is strong and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schema_cardA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. The description does disclose what the tool returns (a compact card with specific components). However, it doesn't mention the id source (tells you to use search_archives id) or any caching/side effects. Since this is a read-only inspection tool, the risk profile is low, and the return content is well specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Compact and front-loaded. The first line states the core purpose and content list. The usage context is one sentence. Args section is minimal. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Has an output schema and only 1 parameter. The description fully covers the parameter sourcing (from search_archives), the content of the result, and the use case. For a simple inspection tool, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the single 'id' parameter. It does, by explaining that the id comes from search_archives, which gives meaningful provenance. Even though the schema has 100% of the single parameter documented in structure, the description adds sourcing semantics beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb+resource: 'Get the compact schema card for an archive' with detailed content enumeration (format, columns, sample rows/headers, sidecar docs, copy-paste starter cell). Clearly distinguishes from search_archives (searching) and get_archive/list_archive_files (retrieval/listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit use case: 'This is the fastest way to understand a dataset's structure without downloading it.' This provides clear context for when to prefer this tool. It doesn't explicitly name alternative tools for exclusion, but the context (fastest way to understand structure) signals when it applies and the siblings are understandable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_archivesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses what results contain (id, title, category, short summary), that there's a category filter, and the limit default (10) and max (50). This is reasonably transparent behavior for a read/search operation. It doesn't describe pagination or error behavior, but for a search tool this covers the essential behavioral surface.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a brief purpose statement, a clear return-value note, a follow-up recommendation, and compact Args documentation. Every sentence earns its place — no fluff. The format is scannable with clear section breaks.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description explains what a search returns and how to follow up. For a search tool with 3 parameters (1 required), plus an output schema, plus a sibling chain to drill into results, this is complete. The follow-up recommendation to get_schema_card completes the workflow story.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does explain each parameter in the Args section: query is a natural-language search with a concrete example, category is an optional filter with an example and comma-separate guidance, and limit is documented with default and max. This adds real value beyond the bare schema, though the category value format ('Genomics & Bioinformatics') is somewhat ambiguous as to whether it's an exact match or fuzzy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource+scope: 'Search the Scigantic catalog of public scientific data archives by natural-language query.' It states what is searched (catalog of public scientific data archives), the method (natural-language query), and explicitly notes returns 'across all domains,' distinguishing from siblings like get_archive and list_archive_files which target specific archives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States when to use it (to search the catalog by natural language) and gives a clear follow-up path (get_schema_card to understand a match's structure), which implicitly distinguishes it from the archive-specific sibling tools. It doesn't explicitly say when NOT to use it, but the follow-up recommendation and description of returning ranked matches across all domains provides clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.