Server Quality Checklist

Tool Scores

• create_outlineA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool can perform web searches (via the 'search' parameter) and requires environment variables for authentication, which are useful behavioral traits. However, it doesn't cover other aspects like rate limits, error handling, or what happens during the outline generation process, leaving gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for purpose, usage instructions, parameters, and returns, making it easy to scan. It's appropriately sized with no redundant sentences, though the Chinese language might add slight complexity for non-native agents, but the content is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no annotations, and no output schema, the description does a good job covering key aspects: purpose, usage, parameters, and return format. It mentions prerequisites and sibling tools, but could improve by detailing output structure or error cases more explicitly, given the lack of structured output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate fully. It adds significant meaning beyond the input schema by explaining each parameter: 'text' is for content description, 'language' specifies supported options (currently 'cn'), and 'search' controls web searching with boolean semantics. This provides clear semantics that the schema alone lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as '创建PPT大纲' (create PPT outline) and specifies it generates outlines from text content, which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'create_outline_by_doc' which likely serves a similar purpose with different input, leaving room for improvement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it indicates when to use this tool (for generating outlines from text) and mentions an alternative tool 'create_ppt_by_outline' for subsequent steps. It also notes prerequisites (setting environment variables). However, it lacks explicit guidance on when not to use it or comparisons with other siblings like 'create_outline_by_doc'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_theme_listA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It does reveal that this is a read operation (获取/获取), mentions pagination behavior (page_num, page_size with max 100), and notes authentication prerequisites. However, it doesn't disclose rate limits, error conditions, or whether the operation is idempotent/cacheable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage instructions, parameters, return). Most sentences earn their place, though the Chinese formatting with numbered lists and bullet points is slightly verbose. The core information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter read operation with no annotations and no output schema, the description does quite well. It covers purpose, prerequisites, all parameters, pagination behavior, and return format. The main gap is lack of output schema details (only mentions '包含模板列表的字典' - contains template list dictionary), but given the context, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must fully compensate. It provides excellent parameter semantics: it explains all 6 parameters with clear meanings, optional values for pay_type, examples for style/color/industry, and constraints for page_size. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as '获取PPT模板列表' (get PPT template list), which is a specific verb+resource combination. It distinguishes this tool from its siblings (create_outline, create_ppt_by_outline, etc.) by focusing on template retrieval rather than creation or task management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it explains that this tool should be called first to obtain template_id for subsequent PPT generation. It also mentions environment variable prerequisites (AIPPT_APP_ID and AIPPT_API_SECRET). However, it doesn't explicitly state when NOT to use this tool or name specific alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_ppt_by_outlineA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job disclosing behavioral traits. It explains the asynchronous nature (returns task ID for polling), workflow dependencies, authentication requirements, and output format (dictionary with sid). It also clarifies failure behavior (throws exceptions). The only minor gap is lack of rate limit or quota information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage instructions, parameters, returns) but could be more concise. Some sentences could be combined (e.g., the multiple tool references in usage guidelines). However, every sentence earns its place by providing essential information, and the structure helps with readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, asynchronous workflow, multiple dependencies) and absence of both annotations and output schema, the description provides complete contextual information. It covers purpose, prerequisites, workflow, parameter semantics, and return behavior. For a complex tool with no structured metadata, this description is comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for 8 parameters, the description fully compensates by providing detailed semantic explanations for every parameter. It clarifies data sources (outline from specific JSON path, template_id from get_theme_list), boolean parameter meanings, dependencies (ai_image only when is_figure is True), and enum values for ai_image. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '根据大纲创建PPT' (create PPT based on outline). It specifies the verb ('创建' - create) and resource ('PPT'), but doesn't explicitly differentiate from sibling 'create_ppt_task' which might have overlapping functionality. The purpose is clear but sibling differentiation is incomplete.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides excellent usage guidelines with explicit prerequisites and workflow context. It specifies when to use (after generating outline via create_outline or create_outline_by_doc), prerequisites (environment variables AIPPT_APP_ID and AIPPT_API_SECRET), and the complete workflow (returns task ID for polling with get_task_progress, then download from results). It also references template_id source from get_theme_list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_task_progressA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the polling behavior ('需定期轮询本工具直到任务完成'), the expected outcome ('任务完成后，可从返回结果中获取PPT下载地址'), and authentication requirements ('需先设置环境变量AIPPT_APP_ID和AIPPT_API_SECRET'). It doesn't mention rate limits or error handling, but covers the essential behavioral traits for this type of progress-checking tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage instructions, parameters, returns) and each sentence earns its place. It could be slightly more concise by combining some points, but the information density is high and the structure facilitates understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (polling progress checker), no annotations, and no output schema, the description provides good completeness. It covers purpose, usage flow, prerequisites, parameter meaning, and expected return content. The main gap is the lack of output schema details, but the description compensates by describing the return format ('包含任务状态和PPT下载地址的字典').

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It provides clear semantic meaning for the single parameter 'sid', explaining it's a task ID obtained from specific sibling tools (create_ppt_task or create_ppt_by_outline). This adds significant value beyond what the bare schema provides, though it doesn't specify format constraints or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('查询' meaning 'query') and resource ('PPT生成任务进度' meaning 'PPT generation task progress'), distinguishing it from sibling tools like create_ppt_task or get_theme_list. It explicitly identifies the tool's purpose as checking progress for PPT generation tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: it specifies when to use (for tasks created by create_ppt_task or create_ppt_by_outline), when to stop (poll until task completion), prerequisites (environment variables AIPPT_APP_ID and AIPPT_API_SECRET), and what to expect after completion (PPT download address). This covers both usage context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_outline_by_docA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well: it discloses document format support (pdf, doc, docx, txt, md with exclusions like scanned PDFs), size/word limits (10M, 8000 words), authentication requirements (environment variables), and integration with another tool (create_ppt_by_outline). It doesn't mention rate limits or error handling, but covers key behavioral aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: it starts with the core purpose, then lists usage instructions in numbered points, followed by parameters and return value sections. Every sentence adds value—no fluff or repetition. It's appropriately sized for a tool with 6 parameters and complex behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no annotations, no output schema), the description is largely complete: it covers purpose, usage, behavioral traits, and parameter semantics. The main gap is the lack of output details—it only states '包含生成的大纲内容的字典' without specifying structure or examples. However, it compensates well with other context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds significant meaning: explains file_url vs file_path as mutually exclusive options, clarifies file_name must include extension, describes text as supplementary guidance, specifies language support (cn), and explains search parameter for online content. It doesn't detail default values or all constraints, but provides substantial context beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '从文档创建PPT大纲' (create PPT outline from document). It specifies the action (create outline), the resource (document), and distinguishes it from sibling tools like create_outline (which likely doesn't use documents) and create_ppt_by_outline (which uses outlines to create PPTs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: it states when to use ('用于根据文档内容生成PPT大纲'), mentions a sibling tool alternative ('生成的大纲可用于create_ppt_by_outline工具'), and includes prerequisites ('需先设置环境变量AIPPT_APP_ID和AIPPT_API_SECRET'). It also specifies document format and size limits, which help determine when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_ppt_taskA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by explaining the asynchronous nature (returns task ID for polling), error behavior (throws exceptions on failure), and environmental requirements (AIPPT_APP_ID and AIPPT_API_SECRET). It doesn't mention rate limits or specific permission requirements, keeping it from a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (使用说明, 参数, 返回) and bullet points, but could be slightly more concise. Every sentence earns its place by providing essential information, though some redundancy exists in the parameter explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, asynchronous workflow, no annotations, no output schema), the description provides comprehensive coverage including prerequisites, parameter details, workflow steps, return value explanation, and error handling. It addresses all necessary context for proper tool usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for 7 parameters, the description fully compensates by providing detailed explanations for each parameter including purpose, constraints (template_id must come from get_theme_list), dependencies (ai_image only works when is_figure is True), and enum values for ai_image. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('创建PPT生成任务' - create PPT generation task) and resource (PPT task). It distinguishes from siblings like create_outline or create_ppt_by_outline by focusing on task creation rather than outline creation or direct PPT generation from outlines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions including prerequisites (must call get_theme_list first, must set environment variables), workflow dependencies (need to poll with get_task_progress), and clear alternatives (different tools for different purposes like create_outline).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.