Server Quality Checklist

Tool Scores

• write_fileC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says 'write a file' without specifying whether it creates new files or overwrites existing ones, how path resolution works, or any error conditions. The workspace constraint adds some context but falls short of disclosing the mutation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundant wording, making it concise and front-loaded. However, it is under-specified for a mutation tool, so it could be slightly expanded without becoming wordy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks critical information about overwrite behavior, path resolution, and what happens on success or failure. The low complexity of the tool does not excuse the absence of these details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter descriptions and 0% coverage, so the description must compensate. It provides no explanation of what 'path' is relative to, whether 'content' is written exactly as provided, or any other parameter semantics. The agent must infer everything from parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (write a file) and the resource (file inside the VS Code workspace). It distinguishes from sibling tools like read_file and list_files, which have clearly different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention when to use write_file instead of read_file or ask_ai, nor any prerequisites or restrictions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_aiC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that data is sent to OpenAI (an external service), but it does not disclose authentication requirements, rate limits, latency, costs, privacy implications, error behavior, or whether the operation is asynchronous. This lack of detail for an external API call is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 9 words, cleanly structured and front-loaded with the core action. Every word is purposeful, with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters, no output schema, and no annotations. The description only explains the basic prompt-and-answer flow, leaving the optional parameters, return type details, and operational context (e.g., network dependencies, API requirements) unexplained. For an external API tool, this is insufficient for an agent to invoke it reliably.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not compensate. The description mentions 'prompt' but provides no meaningful semantics for the required 'prompt' parameter or the optional 'model' and 'system' parameters. The schema itself lacks descriptions, and the tool description fails to clarify their roles, defaults, or possible values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Send a prompt to OpenAI and return the answer' uses a specific verb ('send'), names the resource ('prompt to OpenAI'), and states the outcome ('return the answer'). This clearly distinguishes the tool from its siblings, which are all file/workspace operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any exclusions or prerequisites. The description simply states what the tool does without explaining context for choosing it (e.g., for general AI queries) or how the optional model/system parameters affect usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. The verb 'list' implies a read-only operation, which is helpful but not explicit. It omits details like recursive listing, hidden file behavior, or what happens with a nonexistent path. For a simple read tool, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is appropriately brief for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is minimal and does not cover key context such as parameter behavior, return format, or usage nuances. While the tool is simple, the lack of output schema and annotations means the description should provide more detail to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain the 'path' parameter at all. The agent cannot determine what path does, whether it is optional, or how it affects the listing, making the parameter semantics practically undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('list') plus a resource ('files and directories in the workspace'), clearly differentiating it from sibling tools like read_file (which reads content) and write_file (which writes files). It unambiguously states the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives like read_file or workspace_info. No exclusions or scenarios are mentioned, so the agent must infer usage on its own.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It provides a scope constraint (inside the workspace) and implies a read-only operation. However, it does not disclose behavior such as what happens with large files, whether max_chars truncates output, or how errors are handled. The absence of these details for a safe read operation keeps it at a neutral 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no unnecessary words. It efficiently conveys the core purpose and scope without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, but with no output schema and no annotations, the description should clarify return value and parameter effects. It does not, leaving gaps about what the tool returns (e.g., file contents as a string) and how max_chars behaves. For a tool with only two parameters, more detail would make it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has two parameters (path, max_chars) with 0% description coverage, and the tool description provides no explanation of these parameters. The description does not compensate for the schema's lack of context, leaving the agent to infer that 'path' is the file path and 'max_chars' limits output length without clarification.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Read') on a specific resource ('a file') with an explicit scope ('inside the VS Code workspace'). It differentiates from sibling tools like write_file and list_files by its clear focus on reading content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied: use when you need to read a file's content. However, there is no explicit guidance on when not to use it or mention of alternatives like list_files for discovering files. The description does not state exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• workspace_infoA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'Show,' which implies read-only, but does not state what the returned data looks like, whether any side effects occur, or what happens if configuration is unavailable. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the verb and object. No filler or redundant information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no parameters and no output schema, the description adequately conveys the core function. However, it lacks details about the return format or any prerequisites, so it is not fully complete, but the low complexity keeps it at a reasonable level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. The baseline for 0 parameters is 4, and the description adds no parameter information, which is acceptable since there are no parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Show' and identifies the resource 'MCP workspace and model configuration.' It clearly distinguishes this tool from siblings like read_file, write_file, and list_files, which focus on file operations or AI interaction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when the user needs workspace or model configuration information, but it provides no explicit guidance about when not to use it or alternative tools. The context is clear but lacks exclusions or comparisons to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.