Server Quality Checklist

Tool Scores

• get_citationA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only describes the parameter and its values, without stating that this is a read-only operation, any permissions required, or side effects. The name 'get' implies read-only behavior, but the description does not confirm or elaborate on it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely brief and front-loaded with the main purpose. The Args block is clear and each element earns its place, with no unnecessary words or vague explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (single parameter, output schema present), the description covers the essential purpose and parameter semantics sufficiently. It lacks mention of error handling or relationship to sibling tools, but these are not critical for such a simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only describes the parameter as '引用类型' (citation type) with no allowed values. The description adds concrete enum values ('word' and 'bibtex') and their intended use cases, which is valuable semantic information that goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states '获取COPT的引用格式' (Get COPT citation format), which is a specific verb and resource, and lists the supported formats. This clearly distinguishes it from the sibling tools 'get_reference' and 'get_api_doc'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete guidance on the citation_type parameter values ('word' for Word documents, 'bibtex' for BibTeX files), which is helpful. However, it does not mention when to use this tool over get_reference or get_api_doc, so the usage context is only implied by the tool name and purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_referenceA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not disclose behavioral traits beyond the purpose itself. With no annotations provided, the description carries the full burden. It implies a read-only operation (get), but does not mention whether it returns code snippets, requires network access, or has any side effects. A basic 3 is given because the non-destructive nature is evident from the tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single lead sentence stating the purpose, followed by a clear Args list with bullet points for allowed values. Every sentence earns its place, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all necessary context for a two-parameter tool: purpose, when to use, and parameter values. An output schema exists, so return value details are not needed. It could be slightly more complete with an explicit mention that the example includes code, but overall it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the schema provides generic descriptions for both parameters, the description adds significant value by enumerating all supported values for problem_type (LP, MIP, SOCP, NLP) and language (Python). This goes well beyond the schema's '求解问题类型' and 'API接口语言', making parameter meaning fully explicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to retrieve reference examples for a specified problem type and language in COPT. The verb '获取' (get) combined with the resource '参考示例' (reference examples) and specific parameters (problem_type, language) makes it specific and distinct from sibling tools like get_citation and get_api_doc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by recommending calling this tool before solving a problem with COPT. This tells the agent when to use it, but it does not explicitly mention alternatives or when not to use it, which would warrant a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_api_docA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explains the retrieval and re-ranking behavior, the distinction between recall_num and return_num (with max values), and supported input types (natural language and code snippets). It doesn't state read-only behavior explicitly, but the search/ranking nature and absence of side effects is implied. It doesn't cover error cases or response format, but the output schema covers return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a front-loaded purpose sentence, a clear 'Args:' section with bullet-point examples, and a concise 'Hints:' section. Each part adds value; no filler or redundancy. The formatting aids readability and quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a search/retrieval operation with 5 parameters, all covered in the description. The description provides use cases, parameter details, and hints. Since an output schema is present, not explaining return values is acceptable. The only minor gap is the lack of explicit mention about read-only nature or potential limitations, but overall it is complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the schema. It provides defaults for recall_num (10) and return_num (3), enumerates valid values for language ('Python') and domain ('name'/'description'), and gives illustrative query examples for instructions ('Model.addConstr()' and a description query). Schema descriptions are generic, so the description compensates effectively with concrete usage details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to recall and rank the most similar API docs based on query instructions, specifically for COPT API usage. It distinguishes from sibling tools (get_citation, get_reference) by focusing on API documentation search, and uses a specific verb ('召回并排序') that conveys the retrieval and ranking behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('when you are unclear about COPT's relevant API usage') and provides hints on selecting 'name' vs 'description' fields for different query scenarios. It doesn't explicitly mention when not to use it or name alternatives, but the context and field-level recommendations offer clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.