Server Quality Checklist

Tool Scores

• list_signaturesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it returns a catalog without raw match tokens and is for transparency, but it does not cover behavioral traits like rate limits, authentication, or potential side effects. The description adds moderate value beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and every word adds value. No extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters and an output schema (implied), the description fully covers the tool's purpose and output structure. It explains the return fields and the use case for transparency, making it complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100% by default. The description does not need to add parameter information, and a baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists everything watchcheck can identify, with a specific resource ('signature catalog') and fields returned (vendor, product, etc.). It also mentions the purpose of transparency and identifying gaps, distinguishing it from sibling tools that likely have different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to tell the user what the tool knows — and what gaps remain', providing clear usage context. However, it does not mention when not to use it or direct alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• overviewA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It explicitly states the tool is read-only, classifies processes, collapses duplicates, and flags monitoring software. It does not disclose potential performance impacts or authorization requirements, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences: the first defines the tool's purpose, the second details functionality, and the third gives output presentation advice. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, behavior, and output usage. An output schema exists, so details about return structure are not required. It could briefly mention the format of the overview, but completeness is good.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description need not add parameter info. The schema coverage is trivially 100%. The description adds no param details, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides a friendly overview of all running processes on the Mac, classifying them into types and collapsing duplicates. This distinguishes it from siblings like raw_inventory (unfiltered list) or explain_process (individual process).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description targets users who don't understand processes, giving clear context for when to use this tool. It also provides presentational guidance (type counts first, then flags). However, it does not explicitly exclude alternative tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_processA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the resolution order (monitoring signature → common benign catalog → returns unknown) and guarantees it always answers something useful. This gives insight into behavior without revealing internal details, which is sufficient for a simple read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the purpose, then usage context, then behavioral detail. Every sentence adds value without redundancy, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter), the presence of an output schema, and the clear explanation of input and behavior, the description is fully complete. It does not need to explain return values as the output schema covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single string parameter 'name' with 0% description coverage. The description adds significant value by explaining that the name comes from Activity Monitor, providing concrete examples (WindowServer, EasyConnect, GoogleSoftwareUpdate), and clarifying it can be a process, launchd label, or bundle ID.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'explain' and the resource 'one process / launchd label / bundle id' in plain language. It distinguishes itself from sibling tools like list_signatures and raw_inventory by specifying that it explains a single item, not lists or scans.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly targets non-technical users pasting a single name from Activity Monitor, providing usage context and resolution order. However, it does not explicitly state when not to use the tool or recommend alternatives among siblings, though it is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scanA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers behavioral traits: read-only, inspection of processes, launchd, extensions, MDM, certificates, matching against a signature database, and return of grouped results with evidence, capabilities, and plain-language explanations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at about 100 words across 6 sentences, front-loaded with the main purpose. Each sentence adds distinct value without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of scanning for monitoring software and the presence of an output schema, the description adequately covers input (none), the inspection process, and the output format (grouped findings with evidence and explanations).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100%. The description does not need to add parameter details, and the baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'scan', the resource 'this Mac', and the read-only nature, distinguishing it from sibling tools that focus on individual processes (explain_process), signatures (list_signatures), overview (overview), or raw data (raw_inventory).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the tool is read-only, implying safe usage. It implicitly indicates when to use (to detect monitoring software) but does not explicitly enumerate when not to use or contrast with alternatives beyond the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• raw_inventoryA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description fully discloses the tool's read-only nature and what data it returns ('as collected'). It also implies no side effects by calling it 'raw read-only inventory'. This is sufficient for an agent to understand the behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with minimal waste. It could be slightly tightened by removing 'power users' parenthetical, but it remains clear and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, an output schema exists, and low complexity, the description covers all necessary context: what the tool does, what it returns, and when to use it. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters and 100% schema coverage, the description doesn't need to add parameter details. The baseline is 4, and the description does not miss anything in this dimension.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Return the raw read-only inventory' with a clear verb and resource. It lists the specific contents (processes, launchd jobs, etc.) and distinguishes itself from sibling tools like 'watchcheck' by emphasizing 'NO matching' for power users.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: 'investigate something watchcheck didn't recognize' or 'gather artifacts for a new signature contribution.' This clearly guides the agent on when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.