Server Quality Checklist

Tool Scores

• get_faq_questionsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It mentions that the tool 'відображає інтерактивний список питань' (displays an interactive list of questions), which is a behavioral trait. However, it does not disclose error handling, whether it returns data to the agent or only displays to the user, or any side effects. The description adds some context but remains shallow.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with three short sentences covering purpose, behavior, and parameter guidance. It is front-loaded and each sentence provides distinct information. No waste, though the misleading example reduces overall quality slightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema. The description covers the purpose and the interactive display behavior, but it is ambiguous whether the agent receives structured data or if the interaction is user-facing. It also lacks mention of what happens with an invalid category. For a tool of this complexity, it is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description's examples are problematic: it suggests 'туризм' as a category, which is not in the enum (valid values are 'питання-туризм' or 'туристичне-страхування'). This could mislead the agent into selecting an invalid value. The description adds little beyond the schema and introduces a potential error.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving FAQ questions and answers from hotline.finance for a specified insurance category. The verb 'отримує' (gets) and resource 'питання та відповіді FAQ' are specific, and it distinguishes itself from siblings like list_faq_categories, get_glossary_item, find_faq, and faq_wizard by focusing on category-based retrieval of Q&A.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description instructs to pass a category name and gives examples, implying usage. However, it does not explicitly state when to use this tool over alternatives, nor does it provide exclusions or mention related tools like find_faq or faq_wizard. The context is clear but lacks comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• faq_wizardA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does well: it discloses the interactive flow, the list display, the user choice capture, the relevant FAQ return, and the fallback hint. This provides meaningful behavioral context beyond a simple tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, well-structured sentences front-load the core purpose and add the fallback behavior without redundancy. Every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description covers the main interaction loop and fallback adequately. Minor omissions like the exact format of the returned FAQ or hint prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema defines no parameters, so the baseline of 4 applies. The description confirms the tool works through elicitation rather than explicit params, which is appropriate for its design.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: interactive insurance category selection via elicitation, showing a list, receiving a choice, and returning FAQ. It distinguishes itself from siblings through the elicitation concept, though it doesn't explicitly name alternative tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use when an interactive category selection is needed to return FAQ. The description mentions a fallback condition (client doesn't support elicitation) but doesn't explicitly state when to prefer this tool over list_faq_categories or get_faq_questions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_faqA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses a key behavioral trait: reliance on LLM sampling and a fallback response when sampling is not supported. However, it omits details such as whether the operation is read-only, the shape of returned questions, or behavior when no category matches, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and includes a concise fallback clause. Every sentence contributes value, with no redundancy or filler, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no annotations), the description provides a reasonable overview: input, classification mechanism, fallback behavior, and output type (relevant questions or category hint). It lacks some return value detail, but for a straightforward lookup/classifier it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description's mention of 'arbitrary text query in Ukrainian' adds no new meaning beyond what the schema already provides with its examples. Thus, the parameter semantics are adequately covered by the schema without extra contribution from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool accepts a free-text Ukrainian query, classifies it via LLM into an FAQ category, and returns relevant questions from hotline.finance. This specific verb+resource+behavior distinguishes it from siblings like list_faq_categories and get_faq_questions, which are more direct browsing tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates this tool is for natural-language queries, contrasting with list_faq_categories or get_faq_questions that likely require structured category selection. It also mentions a fallback when sampling is unsupported, giving contextual guidance, but it does not explicitly exclude alternatives or name them as usage options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_glossary_itemA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, so the read-only nature is known. The description adds the return fields (title, explanation, related questions) and the data source (hotline.finance), but does not disclose behavior such as error handling when a slug is invalid or not found. This is acceptable given the tool's simplicity and annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences that front-load the core purpose and then list return values and the alternative tool. Every sentence adds useful information with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read-only tool, the description is largely complete: it names the source, the identifier, the return fields, and directs users to get_glossary_list for the full list. It lacks details about error responses or exact output structure, but given the low complexity and the absence of an output schema, this is sufficient. A score of 4 reflects minor gaps rather than major omissions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents the 'slug' parameter with an enum of all valid values and a description—100% schema coverage. The description only says 'за його slug' (by its slug), which adds no additional meaning beyond the schema. Per the baseline for high schema coverage, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving a detailed insurance glossary term by slug from hotline.finance. It specifies the verb (Отримує), resource (термін глосарію страхування), and key fields returned (назву, пояснення, пов'язані питання). It also distinguishes the tool from a list tool by referencing get_glossary_list for obtaining slugs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions the alternative get_glossary_list, indicating that this tool is for retrieving a single term's details while the list tool is for browsing available slugs. Although it does not state explicit exclusions, the guidance is clear for typical use cases. The sibling FAQ tools are unrelated, so no confusion exists.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_faq_categoriesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the tool is known to be a safe read. The description adds that the output contains slugs that are useful as category parameters, which is a behavioral detail beyond the annotations. It does not mention pagination or exhaustive list guarantees, but the zero-parameter design and openWorldHint=false mitigate this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, focused sentences. The first states the purpose, the second provides a practical usage hint. No redundant information or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature (0 parameters, no output schema), the description covers the essential information: what it returns and how to use it. The only minor gap is the lack of an explicit list of fields in the output, but the slug reference provides a key detail and the tool is straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty (0 parameters), so the baseline is 4. The description correctly does not explain any parameters; the mention of 'slug' refers to the output, not input, so no confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it returns the list of available FAQ categories for hotline.finance. It also distinguishes itself by explaining how the result (slug) is used in get_faq_questions, which differentiates it from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a direct usage context: when you need a list of categories, use this tool, and then use the slug as the category parameter in get_faq_questions. It does not explicitly say when NOT to use it compared to other siblings, but the use case is clear and practical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.