Server Quality Checklist

Tool Scores

• get_skill_summaryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns a trust score, verdict, and recommendation, but does not explain the computation or range of these outputs. Lack of behavioral details like side effects or read-only nature is somewhat mitigated by the tool's obvious safety-check intent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences: the first states the purpose and output, the second provides usage guidance and a sibling reference. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description is fairly complete, covering purpose, usage, and sibling tool. The absence of an output schema is compensated by describing the return values (trust score, verdict, recommendation). Minor improvement could be adding detail on the trust score scale, but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all parameters with clear descriptions (owner and repo with examples). The description does not add extra parameter semantics beyond what the schema provides, so a baseline score of 3 is appropriate given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: checking if a GitHub repository is safe to install as an MCP server or AI agent skill, returning a trust score, verdict, and recommendation. It also distinguishes itself from the sibling tool get_audit_report by noting it's the quickest safety check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use the tool before installing any third-party tool and positions it as the quickest safety check. It also directs users to get_audit_report for the full report, providing clear guidance on when to use the sibling. However, it does not explicitly state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_audit_reportA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description describes return content (trust score, verdict, etc.) but does not disclose behavioral traits like authentication requirements, rate limits, or non-destructive nature beyond the implicit 'get'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with purpose first, no redundant words, and a clear alternative suggestion. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully enumerates the return values (trust score, verdict, findings, category scores, recommendation). Covers all needed context for a moderate-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for both parameters. The description adds no meaning beyond the schema, meeting baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Get', resource 'full behavioral security audit report', and context 'for a GitHub repository'. Distinct from sibling tool 'get_skill_summary'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'review all findings before installing a third-party MCP server, plugin, or tool'. Also suggests alternative 'get_skill_summary' for quick checks. Lacks explicit when-not conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_auditsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that only completed audits are returned and caps at 100 results. However, it lacks details on pagination for larger result sets, authentication requirements, or other side effects. While adequate, key behavioral specifics are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sentences with no redundancy. The first sentence states purpose, second details filtering, third specifies result limit, fourth gives usage tip. Well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers what is returned (up to 100 completed audits, skill information), but does not specify exact fields or structure of results. It adequately supports the search use case but leaves some gaps for a fully self-contained description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds context like 'filter by verdict or minimum trust score' and ties usage to checking prior audits, but does not elaborate beyond schema definitions for sort and order. Overall, it adds moderate contextual meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches completed audits, with filtering by verdict and min_score, and returns up to 100 results. It effectively differentiates from sibling tools (check_audit_status, get_audit_report, etc.) by focusing on listing/filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states: 'Use this to check if a skill has already been audited before submitting a new audit.' This provides clear when-to-use guidance and implicitly advises against using it for other purposes like submitting or viewing individual reports.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_auditA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description fully covers: URL types, return value, rate limit, deduplication, and force rescan option. All relevant behaviors disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five dense sentences, each adding distinct info. No wasted words. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers return value, dedup, rate limit, and sibling tool. Lacks error handling details but sufficient for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds value by explaining deduplication context for force_rescan and rate limit (not in schema).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'submit' and resource 'third-party skill for behavioral security audit'. Distinguishes from siblings like check_audit_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly indicates when to use ('before installing'), mentions rate limiting, deduplication, and directs to sibling tool for polling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_audit_statusA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Fully explains polling behavior, status progression, and that completion includes full report. No annotations, so description carries burden and does so completely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose, then essential details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all aspects for a polling status check: timing, status meanings, terminal states, and return content. No output schema but description explains what's returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'audit_id' is well-described as UUID from submit_audit. Schema coverage 100% and description adds necessary context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks the status of an Oathe security audit and references the submission tool. It differentiates from siblings like 'submit_audit' and 'get_audit_report'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit polling instructions: wait 90 seconds, poll every 10 seconds. Lists all statuses and terminal ones, making usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.