Server Quality Checklist

Tool Scores

• index_statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It indicates a read-only viewing operation, but does not explicitly state it is non-destructive or mention any side effects. For a simple status query, this minimal information is sufficient, but it adds no extra context beyond the bare action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that fully communicates the tool's function with no extraneous detail. It is appropriately sized and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description is adequate. It explains what the tool does, and since an output schema is present, return values are already documented. However, it could benefit from a note that this is a read-only operation, but that is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to explain any. The baseline of 4 is appropriate because there is nothing to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to view which code files are currently indexed locally. The verb '查看' (view) is specific, and it differentiates from sibling tools like search_code or read_github_file by focusing on index status rather than file content or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or related tools. The usage context is implied (e.g., checking what is available before searching) but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_historyA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states the tool queries history (implying read-only) but lacks details on return format, pagination, or side effects. Adequate but not exhaustive for a query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is brief, front-loaded with the purpose, and includes usage context. Every sentence earns its place, though the Args section repeats schema name but adds example.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema, the description provides necessary context including trigger scenario and next steps. Missing some behavioral details but complete enough for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and description adds example for category ('电商web'), but does not specify allowed values or format beyond the example. With a single parameter, this is acceptable but not enriched.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states '查询某个类别之前读过什么项目' (query previously read projects by category), distinguishing it from sibling tools like search_code and search_github by specifying its role in the workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use: '用户说"做个XX"时，第一步调本工具查历史记录' and provides decision logic for alternatives (有记录 → search_code, 无记录 → search_github), which is excellent guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_search_githubA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the mechanism (parsing HTML) and the free/unlimited nature, but doesn't mention potential rate limits on the HTML scraping itself, pagination behavior, or what happens if the HTML structure changes. It's adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the key purpose and advantage. The Args section is clear. Minor redundancy: '搜索 GitHub 上的仓库' repeats the purpose already stated, but overall it's efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 params, no annotations, output schema exists), the description covers the essential purpose, usage context, and parameter semantics. It doesn't describe the output structure, but the output schema exists, so that's acceptable. It could mention error cases (e.g., if search page is blocked) but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains query as search keywords with an example, and limit as result count with default and max. This adds meaning beyond the bare schema, but doesn't specify the format of results or how limit interacts with pagination. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches GitHub open-source projects via GitHub's official search page, explicitly noting it's free, requires no API key, and has unlimited quota. It distinguishes itself from sibling tool search_github by being a fallback when rate-limited, and from search_code by targeting repositories rather than code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool when search_github is rate-limited, providing a clear alternative. It also clarifies it parses github.com/search HTML results and doesn't rely on third-party search engines, giving the agent context on when this is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• db_cleanupA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description carries the burden. It discloses destructive actions (purge_all with warning '慎重') and vacuum behavior, but does not mention reversibility, permissions, error handling, or return format. Adequate but not deep.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Structured with a brief overview followed by a bulleted list of parameters. Each line is purposeful, no redundancy, and front-loaded with the main purpose. Fits within a reasonable length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given multiple actions and dependencies, description covers the main functionality well. Has output schema so return details are not required. Missing edge cases like validation, partial failures, but overall complete enough for a cleanup tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage and only default values, but the description provides detailed explanations for all parameters including valid action values and their prerequisites (category for purge_category, repo for purge_repo). Fully compensates for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool cleans up database tables and lists specific actions (stats, purge_category, purge_repo, purge_all, vacuum). It uses a specific verb 'cleanup' targeting a resource, and differentiates from sibling db_inspect which inspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions when to use: 'when db_inspect finds data is too much or outdated, use this tool to clean'. It references an alternative tool (db_inspect) implicitly, but lacks explicit 'when not to use' or exclusions. Clear context but incomplete exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_github_filesA
  4.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only says 'browse file list' without explicitly stating it's read-only, what it returns, or any side effects. While the output schema may cover return format, the description lacks safety/behavioral details like rate limits or destructive potential. Compared to the calibration example where annotations existed, here the burden is on the description and it is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, structured in two paragraphs plus an args list, and front-loaded with the core purpose. Each sentence contributes to usage or parameter guidance with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature of the tool and the presence of an output schema, the description covers the essential workflow and parameter semantics. It doesn't elaborate on edge cases or return formats, but the output schema likely handles that. Minor missing: no mention of pagination or max results, but for a directory listing this is acceptable. Score 4 due to slight lack of behavioral detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, but the description explains both parameters clearly: 'url' is the repository URL from search_github results, and 'path' is the subdirectory path with empty meaning root. This adds significant meaning beyond the schema's generic titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists files in a GitHub repository ('浏览 GitHub 仓库的文件列表'), and explicitly says '看目录结构' (view directory structure). It distinguishes from read_github_file by noting it is to be used before reading content, and from search tools by implying it operates on a known repo.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow: after finding a repo, use this tool to see directory structure, then use read_github_file to read content. Also forbids using gh CLI or curl as alternatives. This clearly guides when to use and when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_github_fileA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses non-obvious behaviors: file content is automatically indexed locally for later search_code, and code reuse requires source attribution. It does not cover auth or rate limits, but the disclosed side effects are meaningful for a read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: a one-sentence purpose, two sentences of behavioral notes, and a concise Args list. Every sentence earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-file tool with an output schema, the description covers purpose, parameters, side effects, and usage context. It lacks auth/size caveats, but these are not critical given the tool's simplicity and available schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by explaining both parameters: url as 'repository URL (e.g., https://github.com/owner/repo)' and path as 'file path (e.g., "src/main.py")'. These examples add meaning beyond bare property names, though advanced options like branch/ref are not mentioned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states '读取 GitHub 仓库中某个文件的内容' (read a file's content from a GitHub repository), with a specific verb and resource. It distinguishes itself from sibling tools by noting that the content is auto-indexed locally and can later be searched with search_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: read code can be reused with source attribution, and content is auto-indexed for later search_code. However, it does not explicitly mention when not to use it or name alternatives like list_github_files, so it stops short of fully explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It clearly discloses the indexing dependency on read_github_file and implies this is a read-only search over local scope. It doesn't mention edge cases like missing index or result ordering, but the core behavioral boundary is clearly communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: one-sentence summary, clear usage instruction, search scope constraint, and a compact Args list. Every sentence serves a purpose and no unnecessary filler is included.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple three-parameter search tool with an output schema, the description covers scope, prerequisites, when to use, and limitations. It is sufficiently complete for an agent to decide when to invoke this tool and what to expect, although it could mention what happens when there are no previously read files.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the Args section in the description is essential. It explains all three parameters: query, repo, and top_k, with top_k's default and max limit. The descriptions are not very detailed but they do add practical meaning beyond the bare schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states '在已读过的代码中搜索关键词' which clearly identifies a search operation scoped to previously read code. It distinguishes itself from sibling tools like search_github or web_search_github by explicitly limiting results to files read with read_github_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: search before writing code and reuse existing implementations instead of writing from scratch. It also clarifies the limitation that only files read via read_get_file are searchable, but it does not explicitly name alternative tools for cases where code has not been read yet.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_githubA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly indicates a read-only search operation returning URLs, but it does not explicitly state that it has no side effects, nor does it mention any rate limits or potential errors. While the operation is obviously non-destructive, the description lacks explicit behavioral disclosures.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It uses two short sentences to convey purpose, exclusivity, and follow-up actions, followed by a compact list of parameters. There is no redundant or verbose wording; every sentence contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is contextually complete for a search tool. It explains the workflow with sibling tools (list_github_files, read_github_file), sets expectations for the return value (repository URL), and provides parameter definitions. It does not describe error handling or pagination, but for a simple search operation, this level of detail is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only titles and defaults, with no descriptions for 'query' or 'limit'. The description adds meaningful semantics: 'query' is search keywords (can be Chinese or English) and 'limit' is maximum results with a default and max value. This clarifies the purpose and constraints of both parameters, adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '在 GitHub 搜索仓库，返回仓库 URL' (search GitHub repositories and return the repository URL). It explicitly distinguishes itself from alternatives by declaring it is the only way to search GitHub and forbids using gh CLI or curl, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: it states when to use the tool (it is the only search method) and what to do after finding a repository (use list_github_files to browse files and read_github_file to read code). It also gives a clear directive not to use other methods, fully informing the agent about appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• db_inspectA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It goes beyond the name by detailing exactly what data is returned (all tables, fields, record counts, recent records), implying a read-only inspection. This is useful context, though it doesn't explicitly note side-effect freedom or any potential performance implications, so it falls short of a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose, then adds usage guidance and a comparison to a sibling tool. Every sentence earns its place, and there is no redundancy or unneeded detail. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists (so return values need not be explained), the description covers all essential aspects: what it does, what it shows, and when to use it relative to a sibling. It is fully complete for a zero-parameter inspection tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is an empty object, so the baseline for parameter semantics is 4. The description itself doesn't need to explain parameters since there are none, and it doesn't add anything beyond that, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('查看' / view) and resource ('数据库表结构和内容概况'), enumerating what it includes (all tables, fields, record counts, recent records). It also distinguishes itself from the sibling tool 'index_status' by noting it is more detailed and shows complete table structure, making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool: '想看当前索引了哪些数据时用这个' (use this when you want to see which data is currently indexed). It also provides an alternative comparison, stating it is more detailed than 'index_status', which helps an agent choose between the two. This satisfies the 'when' and 'alternatives' criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.