Server Quality Checklist

Tool Scores

• kasko_archive_fileA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and openWorldHint, so the description's addition is minimal. It does not contradict annotations, but it adds little behavioral detail beyond stating it 'resolves' a file—no mention of network dependency, file size, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (~50 words) and front-loaded with the core purpose. Every sentence serves a function: stating purpose, explaining context, listing use cases. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, output schema present), the description covers purpose, usage context, and practical applications. It could mention what happens if the requested month doesn't exist, but openWorldHint reduces the need. Overall, it feels complete for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers both parameters with clear descriptions (year with example, month with min/max/example). The description adds no additional meaning or context for the parameters, meeting the baseline for 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool resolves a monthly spreadsheet and provides use cases like citing or comparing. It distinguishes from siblings by focusing on retrieving a specific month's file, though the verb 'resolve' is slightly vague compared to 'retrieve' or 'fetch'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool (to get a file for a given month or compare values) and provides background on TSB's publishing pattern. It does not explicitly state when not to use it, but the context is clear enough to avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_list_modelsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so no destructive behavior is expected. The description adds behavioral context about model name composition (carrying trim, engine, gearbox info) and that model_id is the key for kasko_get_value, enhancing transparency beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences. The first sentence is a clear purpose statement, and the additional sentences provide useful detail without being verbose. However, the first sentence could more explicitly front-load the essential action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 required parameters, 100% coverage, no enums), the description is adequate. It explains the meaning of returned data and how it connects to another tool (kasko_get_value). The presence of an output schema means return values don't require further explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already describes both parameters (brand and model_year) with examples. The description does not add further parameter semantics; the example values in the description are redundant with the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and the specific resource ('models of one brand in a model year'). It also distinguishes itself from siblings by noting that the returned model_id is used by kasko_get_value, which differentiates it from tools like kasko_list_brands or kasko_list_model_years.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool returns (model names with trim, engine, gearbox info) and the significance of model_id for downstream use, but does not explicitly state when to use this tool over alternatives or mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_get_valueA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and openWorldHint, so the description's claim of reading aligns. However, the description adds no extra behavioral context (e.g., latency, auth needs, or that results may vary per openWorldHint). It is adequate but does not add value beyond what annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundancy. The first sentence states the core purpose; the second provides usage guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (two required params, output schema exists), the description is sufficient. It covers what the tool does, when to use it, and how to differentiate from siblings. No gaps are evident.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already includes meaningful descriptions for both parameters. The description mentions 'by its identifier' but does not add new semantic detail beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Read the kasko value of one exact model by its identifier'), specifies the resource ('kasko value'), and distinguishes it from siblings by referencing kasko_list_models and kasko_lookup. This is specific and leaves no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool ('after the user picks a specific row from kasko_list_models') and when to use an alternative ('When the vehicle is only described in words, use kasko_lookup instead'). This provides clear decision context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_lookupA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  注释已提供readOnlyHint（只读）和openWorldHint（结果可能变化）。描述在此基础上增加了关键行为：解析描述后匹配模型并按车辆返回价值；每个结果携带vehicle_code（保单上使用的品牌型号对）；当多个配置匹配时，要求代理呈现给用户而非猜测。这些行为准则超出了注释的覆盖范围，但未解释openWorldHint的具体含义（如结果集是否随时间变化）。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  描述仅用三个句子便涵盖了核心功能、使用定位（主工具）、返回值关键字段以及多匹配处理原则。每句承载明确信息，无冗余或无关内容，信息密度高且前端加载了最重要信息。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  针对4个参数（含2个必需）且有输出模式的场景，描述给出了基本完整的介绍：工具做什么、参数使用建议、返回值关键含义、以及用户交互策略。但缺少对无匹配或查询不精确时的处理说明，也未涉及与其他兄弟工具（如kasko_list_models）协作的典型流程。考虑到复杂度，描述已覆盖核心用例，留有少量改进空间。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  模式描述覆盖率为100%，基线为3。描述通过示例（“Include brand, nameplate and any trim or engine detail the user gave”）解释了如何填充query参数，并强调了brand可选以加速搜索。同时描述了返回值中的vehicle_code字段及其业务意义（保险公司要求的值），这为参数的使用提供了语义上下文，超过了模式本身的描述。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述以“Look up the kasko value of a vehicle from a free text description”开篇，明确动词（look up）和资源（kasko value），并指出输入是自由文本。后续强调这是“primary tool”，无需预先进行标识符查找，直接区隔于需要编码的兄弟工具（如kasko_get_value），使目的高度清晰且区分于同类工具。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述明确说“This is the primary tool”并给出使用时的行为准则（当多个配置匹配时呈现给用户而非猜测），暗示了这是首选入口。但未明确说明何时应转向其他工具（如kasko_list_brands或kasko_get_value），也缺少当自由文本无法解析时的回退建议，从而遗漏了明确的替代路径。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_download_archiveA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate a write operation (readOnlyHint: false) with no destructiveness (destructiveHint: false). The description adds that it saves a spreadsheet to disk, which is consistent and provides useful context about output format. It does not go into details like file naming or overwrite behavior, but given the annotations, the transparency is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, front-loaded with the action. Every word serves a purpose. Perfectly concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (3 parameters, simple semantics), perfect schema coverage, existing output schema, and clear annotations, the description covers all necessary aspects. It includes usage guidelines and sibling differentiation, making it fully self-contained for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add extra meaning to parameters beyond what the schema provides (year, month, destination). It correctly implies monthly granularity, but this is already clear from the parameter names and tool title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Download a monthly kasko list spreadsheet to disk,' providing a clear verb and resource. It distinguishes itself from sibling kasko_search_archive by noting when to use each, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance ('when the user wants the whole list as a file rather than a few rows') and explicitly names an alternative tool for a different case ('Prefer kasko_search_archive when they only need specific vehicles'). This fully satisfies the criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_search_archiveA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint: true and openWorldHint: true, indicating a safe, read-only operation with potentially expanding data. The description adds value by explaining that the spreadsheet is downloaded and filtered server-side, which implies a delayed response and potential size limits. It also clarifies the historical scope limitation, which is not in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: two sentences covering purpose, use case, and a behavioral caveat. Every sentence provides essential information without redundancy. The layout is front-loaded with the core action, making it easy for an AI agent to quickly grasp the tool's role.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, 2 required, no enums) and the presence of a full input schema and output schema, the description is complete enough. It explains the historical query purpose, the server-side processing behavior, and advises on query specificity. The annotations fill in safety and scope, and the sibling tools provide alternatives for other use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter has a description. The description adds meaning by clarifying that 'query' is an optional free text filter for brand and model names, which is consistent with the schema. While the schema already provides details, the description's mention of filtering 'server side' and keeping queries specific adds pragmatic context. The example year '2025' in the year field description is also helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool searches inside a past monthly kasko list, addressing historical questions like vehicle valuations earlier in the year. The verb 'search' and resource 'archive' are clear, and the purpose is distinguished from siblings by focusing on historical data not available via live endpoints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states when to use this tool: for historical questions that live endpoints cannot answer. It provides an example ('what a vehicle was valued at earlier in the year') and advises to keep queries specific, implying efficient usage. However, it doesn't explicitly exclude other scenarios or mention when not to use it, but the context of eight sibling tools and the open-world hint suggests alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_list_brandsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, which the description reinforces. It adds behavioral nuance by clarifying that the returned brand_id is not the brand code printed on policies, preventing misuse. It could mention the exact return format but the output schema likely covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, with exactly two short paragraphs. The first sentence delivers the core purpose, and the second adds essential details. Every sentence earns its place; there is zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no nested objects), the description fully covers the purpose, requirements, and the key distinction about brand_id. The output schema exists, so return values are documented elsewhere, making this description complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes model_year with an example, and the description emphasizes its necessity and the fact that availability varies by year. This adds meaningful context beyond the schema, such as why the parameter is required and its impact on results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists vehicle brands for a model year, using a specific verb and resource. It distinguishes from siblings like kasko_list_models by mentioning it returns brand_id for that consumer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the year is required because availability differs per year, and notes the relationship to kasko_list_models. This provides clear when-to-use guidance and key context for the expected workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kasko_list_model_yearsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint and openWorldHint annotations already indicate this is a non-destructive, open-ended read operation. The description adds that coverage starts at 2012 and the result is a mapping in descending order, which are useful behavioral details beyond the annotations. However, it does not disclose whether the output always includes a fixed set of years or may vary (e.g., if coverage is updated). The annotations cover the safety profile well, so the description's incremental value is solid but not maximal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: three clear sentences covering purpose, usage scenario, and a key behavioral constraint (start year). It immediately states the tool's function, then gives a concrete use case, and wraps with a brief note on the output format. No words are wasted, and every sentence serves a distinct purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (zero parameters, a mapping output, and clear semantics), the description is complete enough. The output schema exists (though the content is not shown here, but the description mentions the return type), and the start year constraint is disclosed. For a list tool with straightforward behavior, no additional info is needed to enable correct invocation and interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100%. The description does not need to explain parameters since there are none, but it clarifies that no input is required and the call itself is a self-contained check. This achieves a high baseline (4) because the schema is trivially complete and the description confirms the stateless nature.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists vehicle model years from the TSB kasko value list, specifying it returns them in descending order. The verb 'list' and the resource 'vehicle model years' are explicit, and it distinguishes from siblings by being a parameterless year-coverage check, unlike tools like kasko_lookup or kasko_get_value which operate on specific entries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to call this tool: when the user provides no model year, or to verify coverage before querying a specific year. This contrasts with siblings like kasko_list_brands or kasko_list_models which list other entities, and kasko_get_value which retrieves a specific value once a year is known. The guidance is direct and actionable, leaving no ambiguity about prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.