Server Quality Checklist

Tool Scores

• read_codeA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses access restrictions (allow-listed project, allowed type, excluded directories) and implies truncation via max_chars, but does not detail error behavior for missing files or unauthorized paths. This is moderate coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a concise summary, usage guidance, constraints, and an Args list. Every sentence provides value without redundancy, making it easy to scan and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema or annotations, the description covers the core aspects: what it does, when to use, constraints, and parameter meanings. It could explicitly mention the return value type (file content) and error handling, but the purpose is clear enough for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates with an Args section explaining project and path with concrete examples, and defines max_chars as 'Maximum number of characters to return.' This adds meaningful semantics beyond the schema's basic parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Reads the current content of one source file in a project,' using a specific verb and resource. It distinguishes itself from siblings like list_files (listing files) and search_code (searching within code) by focusing on direct file content retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage context: 'Use this to see the up-to-date version of a file instead of relying on a pasted copy.' It also describes constraints (allow-listed project, allowed type, excluded directories), but does not explicitly mention alternatives or when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that excluded directories and patterns are already filtered out, and that the list is exactly what read_code can open. This adds meaningful behavioral context beyond a simple 'list files' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the main purpose. Each subsequent sentence adds useful detail (exclusion behavior, parameter guidance) without fluff. The structured 'Args' block is clean and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with one parameter and an output schema, the description covers purpose, key behavioral filtering, parameter sourcing, and relationship to sibling tools. No critical gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description (0% coverage), so the description compensates by explaining 'project' is a project name from list_projects, with an example. This gives the agent a clear origin and expected value format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Lists the readable source files inside one project' — a specific verb and resource. It also distinguishes itself from siblings by clarifying the list matches exactly what read_code is allowed to open, making its role clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context by explaining that the output corresponds to read_code's allowed files, implying when to use it (e.g., before read_code). It also references list_projects as the source of valid project names. However, it doesn't explicitly state when not to use it or compare to search_code.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and discloses useful behavioral details: it lists only visible projects, includes folder existence status, and counts readable source files. This goes beyond a bare 'list' and helps set expectations, though it omits edge cases like empty results or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose, followed by useful detail and usage guidance. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, has output schema) and the presence of an output schema, the description fully covers the essential aspects: what it lists, what state it reports, and when to use it. The sibling context further clarifies its role in the workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema is trivially complete. The description adds meaning by explaining what the returned entries contain, making it unnecessary to speculate about parameters. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource ('Lists the projects you have made visible to project-mcp') and differentiates from sibling tools like list_files and read_code by focusing on projects as a distinct entity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to 'Start here to see what is available before reading or searching,' which gives clear timing guidance and implicitly points to sibling tools (read_code, search_code) as later steps. Does not name alternatives explicitly, hence not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral details: case-insensitivity, return format ({project, path, line, text}), and scope ('readable project files'). It also mentions max_results as a cap. Since no annotations are provided, this transparency is essential and adequately covers the main behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose, followed by a clear Args block. Each sentence serves a purpose without redundancy, making it easy to scan and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, parameters, and behavior well, and the presence of an output schema reduces the need to explain return values. Minor omissions like handling of no matches or error cases are acceptable, but the description could be slightly more complete given no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning to each parameter: query gets examples, project is described as restricting to a single project, and max_results is described as a cap. The schema only provides titles and defaults (0% description coverage), so this parameter documentation is vital and well-executed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Searches for a plain-text substring across readable project files.' It also provides concrete examples ('SecurityLevel', 'def safe_resolve') and distinguishes itself from siblings by focusing on search rather than listing or reading files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use it 'to locate where something is defined or used without having to open files one by one,' which gives clear context. However, it does not explicitly name alternatives or provide when-not-to-use conditions, so it stops short of full exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.