Server Quality Checklist

Tool Scores

• keiro_searchB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns 'titles, URLs, and content snippets' and that results are optimized for AI, which is useful. However, it does not mention whether the operation is read-only, if any rate limits apply, or how query complexity might affect results. The absence of any caveats or behavioral constraints keeps this at a 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that conveys the core purpose and output without any filler. It is appropriately sized for a simple one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description adequately covers the main purpose and return format. It gets a 4 because it loses one point for not clarifying how this is distinct from sibling tools, which is relevant context for an agent choosing between them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the single parameter 'query' is already documented as 'The search query.' The description adds no new meaning beyond that, but the baseline of 3 applies because the schema fully covers the parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Search the web using KEIRO's semantic search engine.' It specifies the resource (web) and the method (semantic search), and mentions the output (titles, URLs, snippets). However, it does not explicitly differentiate from sibling tools like keiro_search_pro or keiro_research, so it earns a 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus the more specialized siblings (pro, research, answer, crawl). The only implied usage is that results are 'optimized for AI consumption,' but there is no explicit statement of preferred scenarios, limitations, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keiro_researchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does disclose the return structure (summary, key points, timeline, cited sources), which is useful. However, it does not reveal potential limitations such as runtime, data source reliability, or any side effects, leaving the tool's operational behavior less transparent than it could be.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences that are front-loaded with the core purpose, followed by the return value and a use case. Every sentence contributes meaningful information without waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description covers the essential information: what it does and what it returns. It lacks explicit comparison to sibling tools or any caveats about usage, but is largely complete for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a description for the only parameter ('The research topic or question'), achieving 100% coverage. The description adds minimal extra meaning beyond that, only broadening it with 'any topic'. This does not significantly enhance the parameter semantics beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's primary function ('Conduct comprehensive research on any topic') and specifies the return components ('detailed summary, key points, timeline, and cited sources'). It differentiates from search-like siblings by emphasizing 'comprehensive research' and 'in-depth analysis', but does not explicitly distinguish it from the similarly named keiro_research_pro.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context with 'Perfect for in-depth analysis', but provides no explicit guidance on when to use this tool versus alternatives like keiro_search, keiro_search_pro, or keiro_research_pro. It lacks exclusions or when-not-to-use information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keiro_crawlA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses that the tool 'Handles JavaScript-rendered pages' and 'Returns clean, structured text,' providing useful behavioral insight. Yet it omits details such as rate limits, authentication requirements, or behavior on non-HTML content, leaving transparency incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three tightly written sentences, each adding essential information: the core function, the output nature, and a key capability. It is front-loaded and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool, the description covers the core purpose, return format, and JavaScript handling. The lack of an output schema is offset by the clear statement of 'clean, structured text.' Minor gaps remain regarding failure modes or limitations, but the description is sufficiently complete for typical crawling use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the only parameter (url) with 100% coverage. The description adds no new parameter-specific meaning beyond the generic 'any web page,' so it meets the baseline but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action and resource: 'Extract and parse content from any web page.' This distinguishes it from sibling tools like keiro_search and keiro_research, which likely handle broader search or research tasks. Mention of JavaScript rendering further clarifies its unique capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use when you need to extract content from a specific web page. However, it does not explicitly state when not to use it or mention alternatives from the sibling set, so the guidance is limited to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keiro_research_proA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the tool's output behavior (detailed analysis, multiple perspectives, data visualizations, source evaluation) but does not mention side effects, limitations, or any safety traits. This is partial but reasonable for a research tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the key capabilities and closes with a clear usage context, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter research tool, the description provides a solid understanding of expected outputs and target scenario. It could further differentiate from sibling tools like 'keiro_search_pro,' but the 'complex research tasks' context and feature set make it adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a full description for the single 'query' parameter ('The research topic or question'), achieving 100% coverage. The tool description adds no additional parameter semantic detail, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly indicates the tool performs research with detailed analysis, multiple perspectives, data visualizations, and source evaluation. It distinguishes itself from the simpler 'keiro_research' via 'Advanced' and 'For complex research tasks,' though the verb is implicit rather than explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'For complex research tasks' provides clear context on when to use this tool. It does not explicitly mention when not to use it or name alternatives, but the guidance is sufficient for general targeting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keiro_search_proA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool returns enhanced results, metadata, related queries, and deeper content extraction, which gives a sense of behavior. However, it does not clarify what 'deeper content extraction' entails, whether results are structured, or any operational characteristics like speed or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose ('Advanced search') and followed by concrete feature bullets. Every clause adds value: the first sentence enumerates capabilities, the second states the ideal use case. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description is fairly complete. It explains the tool's purpose, capabilities (including what kinds of data it returns), and when to use it. It does not detail output structure, but that may be acceptable given the simplicity and the partial behavioral disclosure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage for the 'query' parameter with a simple description. The tool description adds context that the search is advanced and comprehensive but gives no additional specifics about the query parameter beyond what the schema already states. This aligns with the baseline 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies this as an advanced search tool with a specific verb ('search') and resource (comprehensive information gathering). It lists distinguishing capabilities (enhanced results, metadata, related queries, deeper content extraction) that set it apart from basic search, though it does not explicitly name sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Best for comprehensive information gathering' provides clear context for when to use this tool, implying a preference for in-depth research over quick lookups. However, it lacks explicit exclusions or references to alternatives like keiro_answer or keiro_search, so it does not fully meet the 5-level benchmark.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keiro_answerA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the tool is AI-powered and returns citations, which are key behaviors. It doesn't explicitly state read-only safety, but that's implied by 'answer' and not critical for a Q&A tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys purpose, key features, and usage context without any fluff. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description fully covers what the tool does, what it returns (answer with citations), and when to use it. No significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter 'query' described as 'The question to answer'. The description's 'any question' adds a slight hint of open-endedness but doesn't add meaningful semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get') and resource ('answer to any question'), clearly distinguishing it from sibling tools like search, research, and crawl. It also mentions key differentiators (AI-powered, inline source citations) and best-use case (factual questions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear guidance on when to use: 'Best for factual questions requiring authoritative responses.' It implies alternatives for non-factual or broader tasks, though it doesn't explicitly name them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.