Server Quality Checklist

Tool Scores

• get_project_summaryA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes what the tool does (summarize) and what the output includes, but does not explicitly disclose whether it is read-only, side effects, or auth requirements. The verb 'summarize' strongly implies read-only, but explicit disclosure would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus an args section. It is front-loaded with the main purpose, and every word contributes value. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is reasonably complete. It covers the tool's output nature (dates, counts, completion, outline) and provides guidance for the parameter. It does not detail the exact return format, but the description gives sufficient context for an AI agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It provides a clear explanation for the 'path' parameter: 'Absolute path to the .pod file (use list_projects to find it).' This adds meaning beyond the schema's type and title, including a prerequisite hint. For a single parameter, this is effective.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Summarize') and resource ('.pod project'), and specifies the summary contents (dates, counts, completion, top-level outline). It is easy to understand, though it does not explicitly differentiate it from sibling tools; however, the sibling names are distinct enough to imply different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a helpful prerequisite hint ('use list_projects to find it') but does not explicitly state when to use this tool vs alternatives or any when-not conditions. Usage context is implied by the tool's purpose, but explicit guidance is lacking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_scheduleA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description indicates a read-only reporting behavior by listing key parameters (path, as_of, horizon_days). However, since no annotations are provided, the description could more explicitly state that no modifications are made. It adequately discloses the temporal scope and required input.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: one sentence for purpose followed by a clear argument list. No redundant information; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully explains input parameters but omits details about the output format (e.g., whether it returns a list, summary, or structured data). Given no output schema, more context on return type would improve completeness. However, the description is adequate for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining each parameter's purpose: path (absolute file path), as_of (reference date), horizon_days (window). The schema provides types but no descriptions, so the description adds significant meaning. It could further clarify default behavior for as_of when null.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reports overdue, upcoming, and critical-path tasks for a .pod project. The verb 'Report' and specific task categories differentiate it from siblings like list_projects and list_tasks, which focus on listing rather than analytical reporting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for schedule analysis but does not explicitly state when to use this tool versus alternatives like list_tasks or get_project_summary. No exclusions or direct comparisons are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool finds files (read-only), but does not disclose any additional behavioral traits such as permissions, side effects, or return format. This is adequate for a simple listing tool but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no unnecessary words. It is front-loaded with the verb and resource. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, an output schema, and sibling tools listed, the description fully explains the tool's purpose. No additional information is needed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is complete. The description does not need to add parameter meaning. Baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds ProjectLibre .pod files under configured search roots. It uses a specific verb 'Find' and resource '.pod files', distinguishing it from sibling tools like analyze_schedule or list_tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for discovering project files, but it does not explicitly state when to use this tool versus siblings or provide any exclusions or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tasksA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It explains argument effects (e.g., incomplete_only drops rows) but does not disclose side effects, authentication needs, or read-only nature. Output schema compensates but description could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise: one-sentence purpose followed by clear argument list. No fluff or repetition, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema, description doesn't need to detail return values. It covers all parameters and hints at output content. Missing explicit mention of read-only or performance characteristics, but still adequately complete for a listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description provides detailed docstring for all 4 parameters (path, incomplete_only, start_after, finish_before), explaining their purpose and formats clearly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'List' and resource 'tasks in a .pod project', and specifies included data (hierarchy, dates, predecessors). Distinguished from siblings like list_projects and get_project_summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Purpose implies usage for listing tasks, but description lacks explicit when-to-use or when-not-to-use guidance relative to siblings. No exclusion criteria or alternative tool mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.