Server Quality Checklist

Tool Scores

• search_file_changesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It does not explicitly state that the tool is read-only or disclose any side effects, auth requirements, or rate limits. Basic behavior (finding modifications) is described, but safety profile is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: purpose first, then usage examples, then parameter list with clear labels, and finally returns. It is front-loaded and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers parameters well but lacks detail about the return structure beyond 'Dict with file change results'. No output schema is provided, and the description does not explain ordering, pagination, or format of results. For a search tool, this is a gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaning for all 7 parameters: explains partial match for file_path, semantic query, project/operation filters, date format (ISO-8601), and limit bounds. This compensates for the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds file modifications in conversation history, with example user queries. It distinguishes from siblings like search_conversations by focusing on file changes, but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides example queries ('What did we change in auth.dart?') and a parameter list, giving clear context for when to use the tool. However, it does not mention when not to use it or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_session_summaryA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It only states the return type ('Dict with session summaries') and parameter explanations, but omits whether the operation is read-only, destructive, or has any side effects, rate limits, or prerequisites. This is insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short (7 lines) and well-structured: purpose, usage examples, args, returns. Every sentence adds value, and the key information is front-loaded. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description only vaguely mentions 'Dict with session summaries' without detailing the structure or keys. It also does not cover pagination, error behavior, or performance implications. For a tool with 3 parameters and no additional schema, this is adequate but has notable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions for its 3 parameters, so the description takes on the full burden. It clearly explains each parameter's meaning and default behavior (e.g., 'session_id: Specific session ID, or None for recent'), adding essential semantics beyond the raw schema types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get summary of conversation session(s)' with a specific verb and resource. It provides concrete usage examples that distinguish it from siblings like search_conversations, which is for searching individual messages rather than summarizing entire sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases ('What did we work on in the last session?', 'Summarize our recent conversations') that help an agent determine when to invoke this tool. However, it does not explicitly state when not to use it or compare it to alternatives, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_statusA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not explicitly state that the tool is read-only or safe for repeated calls, but the return structure implies a non-destructive health check. This is adequate but could be more explicit about side effects or permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: purpose, usage, args, returns. It is somewhat lengthy but every sentence provides value. It could be slightly more concise without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema, the description provides a comprehensive list of return fields covering server, health, database, indexing, performance, cache, embedder, file_watcher, errors, and configuration. This is more than sufficient for an AI agent to understand the tool's output. The single parameter is fully covered. Sibling tools are distinct, so no missing context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The one parameter 'detail_level' is fully explained in the description with examples of values ('basic', 'full') and what each returns. The schema only provides name, type, and default, so the description adds essential semantic meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get comprehensive MCP server status and health information.' It uses a specific verb and resource, and is distinct from siblings like get_index_status, get_session_summary, etc. No confusion about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use when you need to check server health, performance metrics, indexing progress, or debug issues with the memory system.' This provides clear context for when to use the tool, though it does not explicitly mention when not to use it or list alternatives beyond the sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_conversationsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return value structure based on flags (analysis, synthesis, debug) and mentions default behaviors. With no annotations, this adequately discloses read-only search behavior and result shape.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with main purpose, Args, and Returns sections. Slightly long but each line adds value; front-loaded purpose sentence is effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers search functionality, parameter details, and return variations comprehensively. Lacks error handling or rate limits, but adequate given no output schema and 9 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 9 parameters are thoroughly described in the Args section, including types, defaults, and effects (e.g., date_from: ISO-8601, enable_analysis: adds 'analysis' and 'evaluation'). Schema coverage is 0%, so description fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Search conversation history for relevant context' and lists specific use cases (previous discussions, decisions, compacted context), effectively distinguishing from sibling tools like search_file_changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use this to find:' list of scenarios, guiding when to invoke. Does not explicitly mention when not to use or alternatives, but the context is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_index_statusA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided. The description implies a safe read-only operation but does not explicitly state it is non-destructive or disclose any behavioral traits beyond return values, such as authentication needs or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct, uses bullet points for return fields, and front-loads the core purpose. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description adequately describes what the tool returns. However, it lacks mention of error conditions or behavior when the index is not initialized, which would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, and schema description coverage is 100%. The description does not need to add parameter semantics, so it fully meets expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets the status of the RAG index, with a specific verb ('Get status') and resource ('RAG index'). It distinguishes from sibling tools like 'get_server_status' by focusing on memory system health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use when user asks about memory system health or why something isn't being found.' This clearly indicates when to invoke the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.