Server Quality Checklist

Tool Scores

• search_project_contextB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'semantically search' without specifying side effects, permissions, or prerequisites like whether the context must be indexed first. Minimal behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, concise and front-loaded with the core purpose. No unnecessary words, though slightly more structured parameter hints could improve clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no output schema, and no annotations, the description is too sparse. It omits details about the required project_path format, result behavior, and any prerequisites, leaving significant gaps for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low (33%). The description adds context for the query parameter via examples, but does not explain the required 'project_path' parameter or 'n_results' (default but no description). The description does not compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (search) and resource (indexed project context), and gives concrete examples of what to find (past decisions, architecture notes, code summaries). It clearly distinguishes from siblings like index_project_context or list_repositories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool ('find relevant past decisions...') but does not explicitly state when not to use it or mention alternatives among sibling tools. Usage guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_project_contextB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only says 'Re-index...' without explaining whether the operation is destructive, whether it overwrites existing data, or what the impact is on the vector store. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single two-sentence paragraph that is concise and front-loaded. The first sentence states the action, the second gives usage triggers. No unnecessary words, and each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one param, no output schema, no annotations), the description covers the basic purpose and usage triggers. However, it lacks details about the parameter and the effect on the vector store, making it minimally viable but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter is project_path, and the schema description coverage is 0%. The description does not mention project_path or provide any additional meaning beyond its name. It relies on the parameter name being self-explanatory, but adds no context about format or restrictions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Re-index the .context/ directory into the vector store.' It also provides context on when to run it, which helps understand its purpose. However, it does not explicitly distinguish from siblings like search_project_context, but the unique action 're-index' sets it apart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear triggers for use: 'after updating project.md, adding ADRs, or refreshing BUNDLE.md.' This provides good guidance on when to use the tool. It lacks explicit when-not-to-use instructions or alternatives, but the given context is sufficient for most cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_repositoriesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only adds 'accessible via configured provider', but does not disclose pagination, rate limits, auth needs, or behavior when provider is unavailable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, compact and front-loaded. No unnecessary words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, and description omits return format, pagination, ordering, or error handling. For a simple list tool, it is borderline adequate but lacks completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and parameter description in schema is clear. Description repeats essentially the same info, adding no new semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action (list) and resource (repositories), and distinguishes from sibling tools which focus on project context and sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this tool for discovery in multi-tenant deployments before other tools, and mentions optional filtering. Lacks when-not-to-use or alternatives, but siblings are unrelated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_session_summaryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states the output location and naming convention but does not disclose overwrite behavior, directory creation, or any side effects. Basic but adequate for a simple write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the key information: purpose, destination, and usage timing. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple parameters and no output schema or annotations, the description covers the essential aspects. It could mention whether the file is created/overwritten, but the overall completeness is high for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%. The description adds meaning to 'summary' by listing what to include (work done, decisions, next steps). However, 'project_path' remains unexplained beyond its type, missing an opportunity to clarify its format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool saves a session summary to a specific file path, distinguishing it from sibling tools that index, load, or search project context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to call at end of session with a concise summary, providing clear when-to-use guidance. Does not discuss alternatives or when not to use, but the context with sibling tools implies differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load_project_contextA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It describes what is returned but omits behavioral traits such as read-only guarantee, error handling, or side effects. The added context of what files are loaded is useful but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states purpose, the second adds mandatory usage. Every word earns its place; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description adequately covers purpose and usage. Minor omissions like read-only or error handling are acceptable for a loader tool with strong usage guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter (project_path) already described in the schema. The description adds 'for the given project path' which does not provide additional meaning beyond the schema, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'Load' and the resource 'full project context' for a given project path. It lists the specific items returned (project.md, ADRs, session summary) and is distinct from sibling tools like index_project_context or search_project_context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'You MUST call this at the start of every session,' providing a clear directive on when to use the tool. No exclusion of alternatives is needed as this is the primary loading tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.