Server Quality Checklist

Tool Scores

• create_translationB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only mentions file upload capability and does not disclose side effects (e.g., task creation, asynchronous processing), required permissions, or how to track the resulting task.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the purpose and immediately following with input constraints. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a create operation with no output schema and no annotations, the description lacks details about the response (e.g., task ID), follow-up steps (e.g., use get_translation to check status), or any workflow context. This leaves the agent uncertain about what happens after invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minimal meaning beyond the schema; 'file upload via path or URL' summarizes the file_path/file_url mutual exclusion, but this is already documented in the schema. No extra parameter context is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'create' and resource 'document translation task', making it distinct from sibling tools (get_translation, list_translations, search_documents). The added 'Supports file upload via path or URL' further specifies the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings. There is no mention of get/list for retrieving tasks or search_documents for finding documents. The usage is only implied by the tool's name/verb, not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It reveals that the search is 'AI-driven' and 'semantic,' which is useful, but it does not mention whether the operation is read-only, requires authentication, or what the response contains (e.g., list of papers, metadata). For a search tool, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that front-loads the core purpose. Every word contributes meaning without redundancy. It is appropriately sized for a straightforward search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple with four parameters all documented in the schema, and the description covers the primary purpose. However, without annotations or an output schema, the description omits behavioral details like return format, pagination, or limitations. It is minimally complete but leaves open questions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description itself does not add any parameter-specific meaning beyond the schema; it only says to input a natural language query. The schema already documents each parameter thoroughly, so the description's lack of parameter detail is acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs AI-driven semantic search for academic literature, using natural language queries to find relevant papers. It specifies the verb (search), resource (academic literature), and how to invoke it. However, it does not explicitly differentiate from sibling tools, though the siblings are translation-focused, so differentiation is apparent from context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: input a natural language query to find papers. It gives a basic how-to but does not outline when to use this tool versus alternatives or state any exclusions. The sibling tools are translation-related, implying search is for finding papers, but no explicit guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_translationsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It transparently indicates a read-only list operation with pagination, but adds no details on ordering, status scope, or return format. The term 'historical' is ambiguous and could mislead about whether in-progress tasks are included.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no fluff. The first sentence states the action and key feature (pagination); the second adds the 'historical' scope. Efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter list tool with no output schema or annotations, the description gives enough to understand the operation. However, it does not explain what data is returned, what 'historical' means, or how it relates to sibling search_documents, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with clear descriptions for 'limit' and 'offset'. The description's mention of pagination adds marginal value beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists translation tasks and supports pagination. 'List' sharply contrasts with sibling 'get_translation' and 'create_translation', making it easy for an agent to select it for multi-result retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It conveys the core use case: listing all historical tasks with pagination. While it does not explicitly name alternatives or exclusions, the context 'view all historical translation tasks' gives clear guidance for when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_translationA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It implies a read-only operation via the verb 'Get' and mentions the return content (details, status, result URLs), but it does not explicitly state the absence of side effects or address error cases. More explicit transparency would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the core purpose followed by a usage hint. Every word earns its place with no redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-parameter read tool with no output schema, the description covers the essential aspects: what it does, when to use it, and what it returns (details, status, result URLs). It does not enumerate possible statuses, but this is not critical for invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a description for task_id ('Translation task ID') with 100% coverage. The tool description adds no further meaning about how to obtain or format the task_id, so it does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get') and resource ('translation task details and status'), clearly distinguishing this from sibling tools like create_translation, list_translations, and search_documents. It focuses on a single task's details, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this to check progress and get result URLs.' This tells the agent when to invoke the tool, though it doesn't contrast with alternatives or state exclusions. Given the sibling tools, this is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.