Server Quality Checklist

Tool Scores

• list_by_sectionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for disclosing behavioral traits. It only states the basic action and gives parameter examples, but does not mention edge cases (e.g., invalid section names), potential side effects, pagination, or authorization requirements. The behavior is treated as a black box, which is insufficient for a tool without structured annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded, with the main statement in the first line and the parameter explanation following. The bilingual Chinese translation is redundant but not harmful. It earns a high score for efficiency, slightly reduced by the unnecessary repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description covers the basic action and parameter. However, it lacks context about how section names are defined, what happens when a section is empty, and how this tool relates to the sibling tools. These gaps make it only minimally complete for a tool in a family of similar operations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only a type for the 'section' parameter, but the description supplies critical context with examples ('获取股票数据', '获取融资融券标的列表') and clarifies it is a section name. This meaningfully compensates for the 0% schema description coverage, giving the agent a concrete sense of valid values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List all API functions in a specific documentation section.' It identifies the verb ('list'), resource ('API functions'), and scope ('specific documentation section'). However, it does not explicitly differentiate from sibling tools like 'list_functions' or 'search_in_section', though the section-scoping is implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'search_in_section' or 'lookup_function'. It only states what the tool does, with no exclusions or contextual triggers. This absence of usage direction leaves the agent without clear selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_functionA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that this is a non-mutating read operation (lookup) and explains what the returned documentation includes (parameters, return value, example code). It does not cover error behavior or exact-match handling, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise but includes redundant bilingual repetition (English and Chinese say the same thing). The Args section is well-structured and front-loaded. Minor redundancy prevents a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description provides sufficient context for correct invocation. It explains what the output includes and the exact-name constraint. Missing error-case details are a minor gap, but overall it's nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only specifies 'function_name' as a string with no description. The description's Args section adds critical meaning: 'Exact function name' and concrete examples ('get_price', 'get_fundamentals'). This fully compensates for the 0% schema coverage and clarifies the expected input format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it looks up complete documentation for a JoinQuant API function by exact name, which is specific and distinguishes it from search or listing by implying exact-name matching. However, it does not explicitly contrast with sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you know the exact function name, but it provides no explicit guidance on when not to use it or mentions alternatives. 'By exact name' is the only contextual clue differentiating it from search_docs or list_functions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_functionsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the core behavior (list names, grouped by section) but omits any note about read-only nature, potential result size, or output structure. For a simple list operation this is adequate but not extra informative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences in English and Chinese, plus a clear usage note. No extraneous information. Front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple (no params), and output schema exists, so description doesn't need to explain return values. It covers the main function and usage purpose. Could mention if it returns descriptions or just names, but for discovery it's sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema is empty with zero parameters, so baseline is 4. Description adds nothing about parameters (none exist), but this is fine since the schema trivially covers everything and no parameter semantics are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists all available JoinQuant API function names, grouped by section. This is a specific verb+resource and differentiates from siblings like list_by_section (which lists a specific section) and lookup_function (which finds details).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Use this to discover what functions are available before looking up specific ones.' This indicates when to use (for initial discovery) and implicitly contrasts with looking up specific functions. No formal alternatives named, but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the search covers function names, Chinese names, descriptions, and call signatures, and supports both languages. This adds meaningful behavioral context beyond a generic 'search docs', though it doesn't mention any limits or output details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose, followed by a short bilingual repetition and an Args section. Every sentence earns its place without unnecessary fluff, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description covers the essential purpose, search scope, and parameter semantics. It lacks explicit guidance on when to use sibling tools, but overall it is adequate for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides no description for the single 'keyword' parameter (0% coverage). The description compensates by explaining 'keyword' is a search term in English or Chinese and that it matches specific doc fields, adding value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Search JoinQuant API docs by keyword', with a specific verb and resource. It also specifies scope (English/Chinese, matching function names, descriptions, signatures), distinguishing it from sibling tools like lookup_function which likely target exact lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for keyword-based search across docs, and the mention of English/Chinese and searchable fields gives context. However, it does not explicitly state when to prefer this over alternatives like lookup_function or list_by_section, nor provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_in_sectionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry behavioral disclosure. It adds useful context: keyword can be English or Chinese, and section is a category name (with example). But it does not mention return format, edge cases, or authorization, though it's a simple search tool with no destructive side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences and an Args list. The purpose is front-loaded, and every sentence contributes meaning. The Chinese translation is an extra but not wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter search tool with an output schema, the description covers purpose, parameter semantics, and scope. It does not explicitly direct users to alternatives, but the sibling names and the 'only' qualifier cover the main contextual gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining 'keyword' as a search term with language flexibility and 'section' with a concrete example. This adds meaning beyond the bare schema names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description directly states action ('Search'), resource ('API docs'), and scope ('within a specific section only'), clearly distinguishing it from sibling tools like search_docs (global) and list_by_section (listing without keyword).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'within a specific section only' clearly sets the context for section-scoped searches, implying when to use this tool. However, it does not explicitly name alternatives or exclusions, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_table_columnsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool performs a read-only lookup of column definitions and, via the Chinese text, specifies the output fields. It does not mention any limitations or side effects, but for a simple metadata lookup, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: the core purpose is stated upfront, followed by a usage hint and a clearly formatted Args section. The bilingual line (English + Chinese) adds value by clarifying what 'definitions' means, and no extraneous content is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema, the description is complete. It explains the tool's purpose, provides a usage scenario, and documents the parameter with examples. There is no need to explain return values since an output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only a string parameter with no description (0% coverage). The description compensates by explicitly documenting the parameter: 'table_name: Table name (e.g. 'balance_sheet', 'income_statement')', including concrete examples, which is highly useful for an agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'looks up all column definitions for a JoinQuant data table', specifying both the action and resource. It also includes a Chinese translation that enumerates the returned fields (name, type, meaning), distinguishing it from sibling tools that handle functions or documentation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a concrete use case: 'Critical for get_fundamentals queries where exact column names are needed.' This tells the agent when to use the tool, but it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.