Server Quality Checklist

Tool Scores

• create_eventA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It states it appends an event and returns the persisted dict, but does not discuss side effects, idempotency, authorization needs, or what happens with the 'parent' parameter (e.g., replacing branch). Limited transparency for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description uses a docstring format with clear sections (Args, Returns). It is front-loaded with the main purpose and every sentence contributes meaningful information. While not extremely short, it is appropriately sized for the number of parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all parameters (two required) and explains the return value. However, it does not mention error conditions, file path conventions, or tag formatting rules. Given the complexity (6 params, no annotations), additional context would be beneficial but not strictly necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description adds crucial meaning to each parameter: task is imperative title, summary describes what was done, reason is strongly encouraged for history, files are project-relative paths, tags are free-form labels, and parent indicates linear chain via id or null for new branch. This compensates effectively for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with 'Append a new event to the project's timeline,' clearly stating the verb and resource. It distinguishes this tool from siblings (which are read/search operations) by focusing on creation of timeline events.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage is for adding events to a timeline, but does not explicitly state when to use this instead of siblings or any exclusions. Context is clear but lacks definitive guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_fileA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It shows output format via Returns and Example, implying it's a query tool. But it doesn't explicitly state it's read-only or disclose side effects/permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise: first sentence states purpose, then Args, Returns, Example. Every part is useful with no redundancy. Well-structured for easy scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with output schema, the description covers purpose, parameter, and return format. Lacks error handling (e.g., what if file doesn't exist) and edge cases, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description adds full meaning: 'Project-relative path to the file (e.g. ``activation.py``)'. This clarifies format and gives an example, going beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Explain why a file exists', providing a specific verb and resource. The sibling tools (create_event, get_project_story, search_history) are distinct, so no confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. No mention of when not to use or prerequisites. The agent must infer usage from the name and example alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_storyA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It explains selection criteria (up to max_milestones, all decision events) and mentions the compressed nature. However, it does not disclose error conditions, performance implications, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with bullet points and a returns section. Every sentence adds value; no fluff. It is appropriately front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one parameter and an output schema (not shown but indicated). The description covers the return structure comprehensively, including a human-readable text field. It is mostly complete but could include when to use it or what the 'compressed' means in practice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly explains the sole parameter 'max_milestones' by stating it limits the number of milestone events in the story. This adds value beyond the schema, which only provides a default. Schema coverage is 0%, so the description fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a 'compressed project story' and lists its contents (name, timestamps, milestones, spine file, decision events, human-readable text). It distinguishes from siblings like 'create_event' and 'explain_file' by focusing on retrieval of a summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide guidance on when to use this tool versus alternatives (e.g., search_history), nor does it mention prerequisites or exclusions. The context signals show no usage hints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_historyA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses case-insensitive substring matching, searched fields, ordering by id ascending, and return structure. Does not mention performance or error conditions, but provides solid behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, front-loaded with 'Search the timeline.' Structured Args and Returns sections with no wasted sentences. All information earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and low schema coverage, description covers search scope, parameters, and return format. Output schema exists, so return details are optional. Lacks details on pagination or performance, but adequate for typical search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explains each parameter: query as case-insensitive substring, limit as cap, search_files/search_tags as boolean inclusions, and defaults. Adds meaning beyond schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search the timeline' with specific verb and resource, listing matched fields (task, summary, etc.), and distinguishes from siblings that create events, explain files, or get project story.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives; however, siblings have distinct actions, so usage is implied. No exclusions or when-not advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.