Server Quality Checklist

Tool Scores

• get_featured_packagesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't describe traits like rate limits, authentication needs, pagination, or what 'featured/popular' means (e.g., curated vs. algorithmically determined). This leaves significant gaps for a tool that likely involves external data fetching.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff or redundancy. It's front-loaded and wastes no words, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema), the description is adequate as a basic overview. However, it lacks details on behavioral aspects (e.g., how 'featured/popular' is determined, response format) and usage context, which could help an agent use it more effectively. It's minimally viable but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100% (empty schema). The description doesn't need to explain parameters, so it meets the baseline expectation. No additional parameter information is required or provided, which is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get a list') and resource ('featured/popular packages from Typst Universe'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'search_packages' or 'list_categories' beyond the 'featured/popular' qualifier, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_packages' or 'list_categories'. It mentions 'featured/popular' packages but doesn't clarify if this is for discovery, recommendations, or other contexts, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_packagesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions search capabilities but lacks details on critical behaviors: it doesn't specify if this is a read-only operation (implied but not stated), how results are returned (e.g., pagination, sorting), rate limits, authentication needs, or error handling. For a search tool with zero annotation coverage, this leaves significant gaps in understanding its operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that efficiently convey the tool's purpose and key parameters. It's front-loaded with the main action ('Search for Typst packages'), and the second sentence adds useful context without redundancy. Every sentence earns its place, though it could be slightly more structured (e.g., bullet points for parameters).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with filters), 100% schema coverage, and no output schema, the description is partially complete. It covers the basic purpose and parameters but lacks behavioral details (e.g., result format, pagination) and explicit usage guidelines vs. siblings. Without annotations or output schema, more context on what to expect from the search results would improve completeness for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, providing clear details for all 4 parameters (e.g., 'query' for search text, 'kind' with enum and default, 'category' for filtering, 'limit' with default). The description adds minimal value beyond the schema by listing the parameters ('search by query text, filter by category, and specify the kind') but doesn't explain semantics like how 'query' interacts with 'category' or what 'limit' entails in practice. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search for Typst packages') and resource ('Typst Universe'), making the purpose immediately understandable. It distinguishes from siblings like 'get_featured_packages' (which likely shows curated items) and 'get_package_details' (which retrieves specific package info) by emphasizing search functionality. However, it doesn't explicitly mention how it differs from 'list_categories' (which might list categories without searching).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through phrases like 'You can search by query text, filter by category, and specify the kind,' suggesting when to use this tool for searching with filters. However, it doesn't provide explicit guidance on when to choose this over alternatives like 'get_featured_packages' (e.g., for curated vs. search-based results) or 'list_categories' (e.g., for browsing categories vs. searching within them). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_package_detailsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes what information is returned but does not address key behavioral aspects such as error handling (e.g., what happens if the package doesn't exist), rate limits, authentication needs, or whether it's a read-only operation. The description adds some value by listing the data fields, but misses critical operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently lists all key information points without redundancy. It is front-loaded with the main purpose and follows with specific details, making it easy to parse. Every part of the sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is adequate but incomplete. It covers the purpose and data returned, but lacks behavioral details (e.g., error cases) that would be important for an agent to use it correctly. Without annotations or output schema, the description should do more to compensate, but it only partially meets this need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'packageName' fully documented in the schema. The description does not add any parameter-specific details beyond what the schema provides (e.g., it doesn't clarify format constraints or provide examples beyond the schema's examples). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get detailed information') and resource ('a specific Typst package'), with explicit listing of the information included (description, authors, categories, etc.). It distinguishes from sibling tools like 'get_featured_packages' (which lists packages) and 'search_packages' (which searches) by focusing on detailed metadata for a single package.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when detailed package metadata is needed, but does not explicitly state when to use this tool versus alternatives like 'search_packages' or 'list_categories'. It provides context by specifying the type of information returned, but lacks explicit guidance on exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_categoriesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it indicates this is a read operation ('List'), it doesn't mention any behavioral traits like whether results are paginated, sorted, cached, or if there are rate limits. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's perfectly front-loaded and appropriately sized for a simple list operation with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description is complete enough to understand the basic purpose. However, without annotations or output schema, it lacks details about return format, pagination, or other behavioral aspects that would help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the input structure. The description appropriately doesn't add parameter information since none are needed, earning a baseline score of 4 for not introducing unnecessary complexity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List') and resource ('all available package categories in Typst Universe'), distinguishing it from sibling tools that focus on packages rather than categories. It provides a precise verb+resource combination that leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'all available package categories,' suggesting this is for browsing categories rather than searching packages. However, it doesn't explicitly state when to use this tool versus alternatives like get_featured_packages or search_packages, leaving some inference required for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.