Server Quality Checklist

Tool Scores

• get_sessionA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully informs about storage location ('~/.codearchitect/sessions/') and behavior when no parameters are provided (lists all sessions). It does not cover authentication or side effects, but as a read-only tool, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear headings and sections. While somewhat long, all content is relevant. Minor redundancy (e.g., repeating 'main folder') but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks details on what the tool returns (e.g., session content format). There is no output schema, so the description should explain the response structure. The 'format' parameter suggests different outputs, but no description of them is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining each parameter's purpose in the 'USAGE' section, e.g., 'filename: Get specific session by topic folder name'. This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves previous conversations, using verbs like 'retrieve' and 'list'. However, it does not explicitly differentiate from the sibling tool 'search_session', which could also retrieve conversations but likely via content search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'WHEN TO USE' section with specific user phrases and scenarios. It does not explicitly state when not to use it or provide alternatives, but the guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_sessionA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers behavioral traits: storage location (~/.codearchitect/sessions/), search behavior (full-text, case-insensitive, relevance scoring, sorting), and response structure (relevance score, snippets). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (WHEN TO USE, STORAGE BEHAVIOR, etc.), but verbose with repeated examples (e.g., user phrases). Could be more concise while retaining key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description details return values (relevance score, snippets, match count, metadata). Covers parameters, search behavior, and storage. Contextually complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds some value like default for limit (not in schema) and format hints, but mostly repeats schema info. No major additional meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as searching across sessions, using specific verb 'search' and resource 'sessions'. It distinguishes from siblings like store_session and get_session by focusing on search functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit user action phrases and scenarios (e.g., 'User remembers discussing something but not the exact session name'), guiding when to use. Lacks explicit when-not-to-use and direct sibling comparison, but context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codearchitect_helpA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that the tool returns available features with examples and next steps. It doesn't mention side effects or auth, but as a help tool, this is adequate. Slightly improved by stating it's safe and read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no waste. First sentence defines purpose, second provides usage guidance. Front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one optional parameter, no output schema, no annotations, the description is complete: it explains purpose, when to use, and what is returned. No missing context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%. The description adds value by listing valid feature names (store_session, get_session, search_session) and explaining behavior when feature is not provided, which goes beyond the schema's description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for getting help about CodeArchitect MCP features, specifying when to use it (when user says 'use codearchitect' without specifying a feature or asks capabilities). It distinguishes from siblings (store_session, get_session, search_session) by being the help tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: when user says 'use codearchitect' without specifying a feature or asks 'what can codearchitect do?'. This provides clear context and implies alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• store_sessionA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: storage locations (main folder always, project folder optionally), no auto-detection, export file vs direct conversation methods, error handling, and response format. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured with clear sections (WHEN TO USE, STORAGE BEHAVIOR, PARAMETERS, WORKFLOW, ERROR HANDLING, RESPONSE). It is front-loaded with purpose. Some redundancy exists, but overall it is organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no output schema, and no annotations, the description is exhaustive: it covers purpose, usage, behavior, parameter guidance, workflow steps, error handling, and response format, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining when to use each parameter (e.g., 'only use if export file unavailable' for conversation) and providing workflow context, making semantics clearer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Save important conversations for future reference.' It differentiates from siblings (get_session, search_session) by focusing on saving, and provides explicit usage examples like 'save this conversation'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  A dedicated 'WHEN TO USE' section lists specific user phrases and contexts (e.g., after solving complex problems). It implicitly excludes retrieval use cases handled by sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.