Server Quality Checklist

Tool Scores

• list_starsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states that the tool lists starred repos with optional filters/sorting, but does not disclose pagination behavior via limit/offset, default sort order, read-only nature, authentication requirements, or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler: 'List your GitHub starred repos, with optional language filter and sorting.' Every word contributes to the core purpose, making it appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description is too sparse for a tool with four parameters, including pagination. It does not explain limit/offset, default behaviors, or what the returned data looks like, leaving significant gaps for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50% (sort and language have descriptions; limit and offset do not). The description mentions 'language filter and sorting' but does not clarify the pagination parameters (limit/offset), their defaults, or their interaction. It adds little meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'List your GitHub starred repos'. It also mentions optional language filter and sorting, which differentiates it from sibling tools like search_stars (searching), get_star (singular retrieval), refresh_stars (sync), and stars_status (status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use list_stars versus alternatives such as search_stars or get_star. There are no explicit when/when-not conditions or mentions of sibling tools, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stars_statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. The verb 'Show' implies a read-only operation, but the description does not explicitly state that it makes no changes or whether it accesses remote data. For a simple status tool, this is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, clear sentence that front-loads the purpose without any filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless status tool, the description fully conveys what information is returned (timestamps for last check and sync). No output schema exists, but the description covers the essential return value qualitatively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. Per the rubric, the baseline is 4. There is nothing for the description to explain about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Show' and names the resource 'local stars database' plus the exact information displayed (last checked and last synced). This clearly differentiates it from sibling tools like search_stars, list_stars, get_star, and refresh_stars, which focus on querying or modifying data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention any conditions, prerequisites, or relationships to sibling tools such as refresh_stars. Users are left to infer that this is for status checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_starA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure. It indicates a read operation ('Get') and that it returns 'full details', but it doesn't explain what happens if a repo isn't found, whether authentication is required, or the exact response structure. This is a basic getter, so transparency is acceptable but thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that front-loads the verb 'Get' and includes the key qualifier 'by its full name'. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with no output schema and no annotations, the description is largely sufficient. It tells the agent what the tool does and how to provide the argument. 'Full details' hints at the response, but it doesn't specify the response shape or error conditions. Still, for a simple read operation, this is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: the schema already documents full_name with an example ('facebook/react'). The description adds 'owner/repo' which essentially repeats the schema example. It does not introduce new meaning like validation rules or special formats beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get full details for one starred repo by its full name (owner/repo).' It specifies the action (get), resource (starred repo), and the identifier format (owner/repo). This distinguishes it from siblings like list_stars or search_stars, which operate over multiple repos.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: use this when you need details for a single known repo, identified by full_name. It doesn't explicitly mention alternatives or when not to use it, but the 'one' and 'by its full name' phrasing provides clear context for when it applies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_starsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the key behavior: forcing a full re-fetch and updating the local database. However, it does not mention rate limits, whether the update is destructive, or failure behavior, leaving some gaps for a mutation-like operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, with no wasted words. The first states the action, the second gives context, making it effective and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool, the description covers the main purpose and usage context. It could mention return values or potential side effects, but the refresh action is self-explanatory enough that the description feels complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters, and the empty schema fully covers the input contract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Force a full re-fetch of your starred repos from the GitHub API and update the local database.' It uses a specific verb and resource, and the mention that read tools check automatically distinguishes its purpose from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides useful context by stating 'Read tools already check for updates automatically on use,' implying this tool is for forcing a manual refresh. It does not explicitly list alternatives or exclusions, but the context is clear enough to guide usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_starsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It usefully discloses that the search matches name, description, and topics, and that language filtering is optional. However, it does not explicitly state that the operation is read-only, nor does it mention any pagination, sorting, or authentication requirements. This leaves some behavioral ambiguity, though the term 'search' implies a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the core action and key details without any unnecessary words. Every part (keyword scope, language filter) earns its place, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with three parameters fully described in the schema, the description covers the essential search semantics and optional filter. It does not explicitly state the return format or mention the 'limit' parameter, but these are implied by the tool's purpose and covered in the schema. Given the lack of an output schema and annotations, the description is reasonably complete but could add a brief note on return value or pagination behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has descriptions for all parameters (100% coverage), providing a baseline of 3. The description adds meaning beyond the schema by explaining that the 'query' keyword matches name, description, and topics, which the schema's 'Keyword to search for' does not convey. It also reinforces the language filter. This extra context justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a keyword search over GitHub starred repos, specifying that it matches name, description, and topics. This specific verb+resource+scope distinguishes it from sibling tools like list_stars, get_star, and refresh_stars, which imply different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the primary use case: finding starred repos that match a keyword, optionally filtered by language. It provides clear context for when to invoke the tool, though it does not explicitly mention alternatives or exclusion criteria. Sibling tool names (e.g., list_stars) help imply when a different tool would be more appropriate, so this is not misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.