Server Quality Checklist

Tool Scores

• skill_getC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Because there are no annotations, the description carries the behavioral disclosure burden. It discloses the shape of the returned package and hints at a progressive-disclosure stage, which is useful. However, it does not explicitly state read-only behavior, possible side effects, output truncation, or failure modes; the verb implies retrieval but the behavioral story is incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the primary action and immediately enumerates the package components. Every item adds information, and there's no wasted prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two optional undocumented parameters and no output schema, the description should explain how to choose the skill, how the response is shaped, and what the 'progressive disclosure' stage means in practice. The description lists contents, but it leaves the invocation context and return semantics incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has two parameters, key and name, but zero description coverage for both, and the description says nothing about how these parameters identify the skill or whether one, both, or either is required. The description fails to compensate for the low schema coverage, leaving the agent without enough meaning to select parameters confidently.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('获取') and names the resource ('skill 的完整包') with a clear list of package contents: manifest, SKILL.md body, file list, and resolved dependencies. It is clear and useful, but it does not explicitly differentiate this tool from the sibling tools such as skill_inspect or skill_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use skill_get versus the other related tools (skill_inspect, skill_search, skill_run, etc.) and no mention of prerequisites or a step in a progressive disclosure flow beyond the parenthetical. There is no 'when to use this, when to use that' direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_inspectC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It does state that the content body is not included and that the operation is low-cost, which adds some context. However, it does not mention authorization, output format, side effects, or failure modes, leaving the agent to infer the tool's superficial behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler. It front-loads the verb and resource, and the parenthetical adds useful scope and cost information. A bit more structure would improve scannability, but the length is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and two optional-looking parameters, the description is not complete enough for confident invocation. An agent still needs to understand the required parameter combination, output shape, and fallback when both parameters are absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50% because the 'name' parameter lacks a description, and the tool description does not compensate. The 'key' parameter has example formats, but the relationship between 'key' and 'name', whether either is required, and what happens if both are provided remain unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies what the tool does: inspect a skill's manifest, dependency tree, and permission list. It explicitly excludes the body ('不含正文'), which helps differentiate it from likely content-focused siblings like skill_get, though it doesn't name the alternative directly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance about when to choose skill_inspect over sibling tools such as skill_search nor skill_get. '成本低' hints at lightweight use, but there are no stated conditions, exclusions, or alternative tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_planC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the burden. It does disclose meaningful behavior: the planning process follows deliberately decompose-retrieve-compose, and without an LLM it falls back to capability closure graph search. That is useful beyond the schema. It still omits information about side effects, whether the plan is stored or merely returned, and what happens in failure cases, so it is only partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence with no filler. It front-loads the core action ('把任务规划成 skill 组合的 workflow DAG') and then adds relevant pipeline details in parentheses. The phrasing is somewhat dense and technical, but every part carries meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, no parameter descriptions, and a nested object parameter, this definition leaves significant gaps. It tells the agent the shape of the output in the abstract (a workflow DAG), but not the concrete return format, how to populate context, or how planning interacts with the sibling tools. An agent would likely need additional tool calls or documentation to invoke it safely and correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for both parameters. It only glosses the 'task' concept through the phrase '把任务规划成', and says nothing about the 'context' object, its expected structure, or its role in planning. The nested object parameter is completely unaddressed, leaving the agent to guess what context is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly names the verb-action (plan), the object (task), and the produced artifact (workflow DAG of skill combinations). It also reveals the high-level pipeline (decompose-retrieve-compose), giving the agent a concrete sense of what the tool does. However, it does not explicitly contrast itself with siblings like skill_search or workflow_run, so it earns a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells what the tool does but not when the agent should pick it over alternatives. There is no mention of skill_search, skill_run, or workflow_run, and no exclusion criteria such as 'use workflow_run if the DAG already exists'. The mention of 'when no LLM is available' is an implementation condition, not usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• workflow_runB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses meaningful execution behavior: topological ordering, retries, checkpoints, and chain-wide authorization. However, it does not mention side effects, required permissions, failure modes, or what happens after execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short and front-loads the core purpose: executing a DAG. Every phrase—topological data flow, retries, checkpoints, authorization—adds distinct value without filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a moderately complex tool with 5 parameters, no output schema, and no annotations, but the description does not explain the optional parameters or return/result behavior. The core DAG-execution context is present, but the agent would still need to infer how 'approveAsks' and 'grantTokenId' work.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only clarifies 'dag'. The optional parameters 'task', 'inputs', 'approveAsks', and 'grantTokenId' are left unexplained; the '整链授权' phrase implicitly relates to authorization parameters, but not explicitly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb (执行/execute) and resource (the DAG produced by skill_plan), which clearly separates it from the skill_* siblings. It also names key behaviors of the operation, though it does not explicitly contrast it with skill_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly implies the usage context: after skill_plan has produced a DAG, this tool runs it. It does not explicitly list when-not-to-use or name alternatives, so it lacks the strongest routing guidance, but the context is unambiguous enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_runA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  由于没有任何 annotations，描述承担了行为透明度的主要责任。它明确披露了权限沙箱、requiresApproval 返回、两段式 HITL 以及 approveAsks: true 授权后重试的机制，这些是关键的运行时行为。但它没有具体说明成功时的输出结构、错误码、超时行为或副作用，所以未到满分。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  一句话完成了“产在哪里执行 + 权限异常路径提示”两部分信息，没有冗余。信息前置，后续补充授权重试条件，结构紧凑，每个短语都有实际价值。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  该工具没有 annotations 或 output_schema，描述必须承担完整的路由和调用指引责任。虽然核心用途和 HITL 流程是清晰的，但成功/失败返回格式、inputs 对象的约定、timeoutMs 语义、grantTokenId 的用途都没有覆盖，agent 在真实调用时仍面临多个空白。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  输入 schema 有 6 个参数，而 schema_description_coverage 为 0%；描述只解释了 approveAsks 的含义，帮助 agent 理解 HITL 重试，但 task、inputs、timeoutMs、grantTokenId 等其余参数均未获得说明。由于 schema 本身没有字段注释，描述必须登记大部分参数语义，当前明显不足。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述用明确的动词“执行”和资源“单个 skill”概括了工具用途，并用“权限沙箱”限定了执行环境。这与 skill_search、skill_inspect 等检视类兄弟工具形成清晰区分。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述强调了“单个 skill”的语义，从而暗示与 workflow_run 等多 skill 编排工具的差别，但没有显式说明何时应该用这个工具、何时不应使用。缺少对兄弟工具的明确排他指引。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_statsA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the burden of behavioral disclosure. It clearly describes the action as read-only ('查看') and lists what the tool returns, including an extracted workflow recipe. It would benefit from caveats about whether stats require existing run/feedback data, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one focused sentence and every phrase adds information about the tool's purpose or output. There is no filler, repetition, or unnecessary explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no annotations, no output schema, an undocumented parameter, and no usage caveats. The description tells the agent what the tool reports but not how to identify the skill or whether the key is required, making confident invocation difficult.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes a single 'key' parameter with no explanation, and the description never mentions the parameter, its format, its purpose, or how it should identify a skill. With 0% schema description coverage, the description does almost nothing to help an agent construct a valid invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('查看' / view), a specific resource (skill stats), and enumerates the exact metrics reported: Wilson success rate, Elo, quality score, latency cost, and workflow recipe. This clearly separates it from siblings like skill_get, skill_inspect, or skill_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit statement of when to prefer this tool over siblings such as skill_get or skill_inspect. The usage is only implied: use it when you need skill evaluation metrics rather than skill content or execution.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_searchA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  由于没有任何 annotations，描述承担了全部行为披露责任。它补充了检索算法、打分维度和返回带理由候选列表等实质性信息，让 agent 知道这是只读检索；但未说明无结果时的行为、topK 截断/排序规则、打分分数区间或是否需要权限等边界行为。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  两句话把核心动作、算法、输出形式和首要使用时机都讲清楚了，全部前置，没有任何冗余。每句话都有作用，是简洁高效的结构。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  对检索工具而言，描述了核心行为和返回内容，且没有输出 schema 也必须自己交代返回值，它做了这一点。但缺少 topK 默认值/含义、filters 各字段用法、候选列表数量及五维得分的具体解释，因此 agent 仍需试探或依靠反馈才能充分利用全部参数。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  schema 说明覆盖率为 50%，query 和 context 已有描述，topK 和嵌套的 filters（5 个数组字段）均无描述。工具描述本身没有为任何参数新增语义，仅“按任务检索”间接对应 query，没能补偿 schema 中缺失的那一半参数信息。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述以具体动词「检索」加资源「skill」开头，并明确混合检索算法（BM25 + 向量 + RRF）和五维任务匹配打分，还说明返回「带理由的候选列表」。它清楚地与 skill_inspect/skill_get/skill_run 等兄弟工具区分开，并给出「先做这个再动手」的使用定位。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  「先做这个再动手」明确告诉 agent 在执行任务前应该先调用此工具，给出了非常清晰的时机指引。但没有显式说明何时不该用、以及后续应改用哪个兄弟工具（如 skill_inspect 查看详情、skill_run 执行），故扣一分。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_feedbackA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  没有 annotations，描述承担了行为披露责任：它明确说明会覆盖同一 skill_run 的自动记账，且不重复计数，还说明会影响 bandit/Elo 排名与备选检索。这比“提交反馈”这类模糊表达透明得多，但未提权限、失败条件或响应行为。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  一句话就传达执行目的、学习影响、覆盖语义和不重复策略，信息密度高且没有冗余。括号内的副作用说明是对主要用途的必要补充，而不是堆砌。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  工具用途和最重要的副作用都已覆盖，但 9 个参数的实际取值范围和含义仍依赖 schema，而 schema 覆盖又很低；无输出 schema、无 annotations，描述也缺少对如 rating 刻度和 beatenBy 委托等字段的解释。整体可执行，但有关参数语义的缺口明显。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  schema 覆盖率只有 11%，描述也没有对 skill、outcome、rating、costCents、latencyMs 等参数做补充说明。描述中的“结果与评分”和“覆盖自动记账”只能间接关联到 outcomlights/rating/cost/latency，无法弥补 9 个参数中 8 个缺少描述的问题。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述以“报告执行结果与评分”这个具体动词+对象开头，并说明它“驱动 bandit/Elo 排名与备选检索”，清楚定义了工具的核心职责。它与 skill_run、skill_search 等兄弟工具区分明显，不是对工具名的同义重复。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述通过“学习闭环”和“覆盖同一次 skill_run 的自动记账”暗示了应在 skill_run 之后用于提交反馈，并明确说明不会重复计数。虽然没有逐个对照兄弟工具给出 when-not-to-use，但已给出足够的调用时机与副作用边界。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_registerA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses append-only behavior and v3 mode-specific requirements, which are meaningful and likely safety-relevant. It does not describe return values or failure modes, but the disclosed constraints are more than adequate for selection.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is one compact sentence that conveys the operation, the append-only constraint, and the mode-specific signing/test requirements. There is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is minimally viable for selecting the tool, but with no output schema and no detailed manifest/files schema, an agent cannot fully infer return behavior or package structure. The v3 conditions are helpful, but important surrounding context is left implicit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents signature and publisherKey, and the description adds that signature is required in trust mode and tests are needed in gate mode. However, the manifest and files objects remain undocumented in both the schema and the description, leaving an obvious semantic gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (register/upgrade) and resource (skill), plus the key append-only invariant. This makes it clearly distinguishable from read/execution-oriented siblings like skill_search and skill_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly limits the tool to registration/upgrade and provides mode-specific guidance for trust vs gate modes. It does not explicitly name alternative tools, but the scope is specific enough that an agent can select this tool appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.