Server Quality Checklist

Tool Scores

• healthA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing side effects. The verb 'report' strongly implies a read-only, non-mutating operation, but the description does not explicitly state that the tool has no side effects, makes no external calls, or is safe to invoke at any time. It is not contradictory, but it lacks explicit behavioral guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that contains no redundant words or filler. It is front-loaded with the verb and immediately specifies what is reported. The structure is ideal for a no-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and the output schema is available (though not displayed), the description sufficiently covers the essentials. It states what the tool does without needing to elaborate on inputs. The only minor gap is the lack of detail about the output format or structure, but since an output schema exists and the purpose is simple, this is not a significant omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (empty properties object). The description does not need to explain parameter semantics because there are none. Per the baseline for 0 parameters, a score of 4 is appropriate; there is no additional parameter-related context to add.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Report embedding backend, model, and GitHub auth status.' It uses a specific verb ('report') and identifies the exact subjects, making the purpose unambiguous. It also naturally distinguishes itself from the sibling tools (find_repos, repo_intel) which focus on repository discovery and analysis, not system health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any explicit guidance on when to use this tool versus the siblings. While the purpose is clear, there is no direct statement like 'Use this when you need to check system health' or any comparison to find_repos/repo_intel. This leaves the agent to infer usage context from the tool's name and purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_reposA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It discloses that the tool returns a 'verifiable, adoption-grade verdict with evidence and quality signals (stars, activity, license, archived status)', which gives the agent a clear picture of the output characteristics. It does not mention side effects, rate limits, or read-only nature, but as a search/ranking tool, that is minor. The description adds meaningful behavioral context beyond a bare 'find repos'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero fluff. The first sentence states the core purpose, and the second provides the usage context and expected output. It is front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters) and the presence of an output schema, the description provides adequate context. It explains the intended use case, what the tool returns (ranked projects with quality signals), and the decision it supports. It does not go into detail on ranking methodology or edge cases, but for a search tool with a rich schema, this is sufficient. The output schema likely covers return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% because the tool description does not explain any of the seven parameters. The description does hint at some filters by naming 'license, archived status' and 'stars', which map to the license, include_archived, and min_stars parameters, but it omits query, top_k, intent, and language entirely. Since coverage is low, the description must compensate, but it only partially does. The schema itself has strong descriptions, so the agent can still understand the parameters, but the tool description adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Rank') and resource ('open-source GitHub projects'), and clarifies the outcome ('implement a described feature'). It distinguishes the tool's role as an adoption/study selection aid, which is distinct from the likely deeper-analysis siblings repo_intel and health, even though it does not name them explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states the trigger condition: 'Use when an agent needs to pick a library/project to adopt or study'. This gives clear when-to-use guidance. However, it does not mention alternatives or when not to use, and it does not differentiate from sibling tools, which would have strengthened the guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repo_intelA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It does indicate a read-only verification action and that it returns a status verdict plus alternatives, but it does not disclose potential side effects, network behavior, rate-limit implications, or error conditions. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short, focused, and front-loaded with the core purpose. Every sentence contributes value: what it verifies, what it returns, and when to use it. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the sibling tools (find_repos, health) and the explicit note that the repo is 'already found,' the description is contextually complete for typical use. It could provide more detail about the output shape, but it covers the essential need: verifying a repo before dependency commitment.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for owner_repo, query, and top_k, including an example for owner_repo. The description adds context by mentioning 'best-of-kind' and 'a few alternatives,' which aligns with query and top_k behavior, but it does not explicitly map each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Verify') and identifies the resource (an existing repo the agent already found) and the key checks (alive, licensed, best-of-kind). It clearly distinguishes this from the sibling tools by emphasizing it is not for discovery, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool: 'BEFORE committing to a dependency to catch archived/stale/no-license repos.' This gives direct, actionable guidance and implicitly contrasts with the find_repos and health sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.