Server Quality Checklist

Tool Scores

• compare_over_timeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that it 'returns changes' without explaining the output format, whether it is a summary or raw data, computational costs, or any limitations. The description adds minimal context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the core purpose and output without any waste or redundancy. It is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool with 8 parameters and no output schema, requiring more context about how the comparison is structured and what the result looks like. The description merely states the high-level function without addressing output format, computational implications, or behavior details, making it insufficient for an agent to fully understand the tool's capabilities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter well-documented. The description does not add extra meaning to parameters beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares Diet minutes across 2-3 periods and returns changes in discussion topics (increase/decrease, new/continuing). It uses a specific verb and resource, but it does not differentiate from similar siblings like 'compare_by_party' or 'analyze_topic_changes,' which limits its distinctiveness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as compare_by_party or analyze_topic_changes. It does not mention exclusions, prerequisites, or typical use cases, leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_topic_changesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavioral traits. It mentions the output includes sources and change types, but does not explicitly state that the operation is read-only or describe any side effects, error behavior, or cost implications (though 'mode' parameter hints at cost in schema). It does not contradict annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core purpose, scope, and output, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, yet the description only provides a high-level output summary (change types with sources). It does not specify the response structure, how mode affects granularity, or potential limitations, leaving some ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with descriptions, so the baseline is 3. The description adds no extra parameter-specific meaning beyond what the schema already provides; it refers to the 'periods' concept but not specifically to each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes Diet minutes across multiple periods (2-5) and returns change types (increase, decrease, continuation, new) with sources. However, it does not explicitly differentiate from the sibling tool 'compare_over_time' which may also handle temporal comparisons.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for analyzing multi-period topic evolution but provides no explicit guidance on when to choose this tool over siblings like compare_over_time or summarize_meeting. No exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_meetingA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the composite behavior (get_meeting + summarize_speeches) but does not state whether the operation is read-only, what output format to expect, latency/cost implications, or any side effects. This is a significant gap for a combined tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that captures the core function and composition. Every word adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description should explain return values and usage context. It does not describe the output format, nor does it provide clear guidance on when to use this tool versus siblings. The composite nature is helpful but leaves the agent uncertain about what the summarized output looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no parameter-specific meaning beyond the schema; it only mentions issueID in the general text, which is already documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb+resource: 'specify issueID to retrieve and summarize the entire meeting record.' It also distinguishes the tool from siblings by explaining it internally combines get_meeting and summarize_speeches, making its composite nature explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when you want both retrieval and summarization in one call, as it says 'internally executes get_meeting + summarize_speeches.' However, it does not explicitly contrast with alternatives (e.g., when to use get_meeting alone), so it lacks full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_speechesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explains the basic behavior (generating a summary) and parameter effects (mode for detail, focus for narrowing), but it does not disclose potential costs, latency, or side effects. It also leaves the return format unstated, which is a notable gap for a generation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the core action. It wastes no words and flows logically from the main purpose to usage details. This is a model of efficient tool documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description plus schema covers the purpose and parameters well. However, with no output schema, the description does not clarify the format of the returned summary, which creates ambiguity. Also, it omits mention of the 200-item limit (covered in schema), but the missing output format is a real gap for a tool that generates a result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds valuable semantic context beyond the schema by explaining that the items parameter can be populated directly from search_speeches output (a key integration detail) and by giving a natural-language explanation of mode and focus that mirrors but enriches the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it generates a summary from a list of speeches ('発言一覧を入力として要約を生成します') and specifies the integration path with search_speeches. This is a specific verb+resource, but it does not explicitly contrast with the sibling 'summarize_meeting', so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context by stating that the output of search_speeches can be passed directly as input, effectively telling the agent when to use this tool (after a search). It also explains the role of mode and focus, but it does not mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_by_partyA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that output includes sources ('出典付きで返します'), which is useful. However, with no annotations provided, the description carries the full burden for behavioral traits. It doesn't mention read-only nature, potential cost implications of 'detailed' mode, or how time filters apply. It adds some value but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately states the tool's purpose and output. Every word contributes meaning: aggregate, compare by party, return differences/commonalities with sources. No wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters and no output schema, the description should clarify return structure and key behaviors. It does state the output includes stances, differences, commonalities, and sources, which is helpful. However, it omits mention of the 'mode' parameter (brief/standard/detailed), time filtering, or any limitations, leaving gaps for a complex comparison tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3 even without parameter details in the description. The description only refers to the specified theme (query), but the schema already documents each parameter and its purpose. No additional semantic value is added beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'aggregates and compares statements by political party on a specified theme.' It explicitly mentions returning stances, differences, commonalities with sources, which distinguishes it from sibling tools like compare_over_time that likely focus on temporal comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: for aggregating and comparing party-specific discourse on a given theme. It does not explicitly exclude alternatives or mention siblings, but the 'by party' focus is explicit, giving clear usage context without needing further exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_speechesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  As no annotations are provided, the description carries the burden. It states that the tool searches (implying read-only) and that conditions can be combined, but it does not mention output format, the AND logic for query, or the limit behavior, though the latter is in the schema. The behavioral disclosure is minimal beyond the core action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, dense sentence with no redundant content, opening directly with the action. Excellent conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The absence of an output schema and sample usage leaves some ambiguity about the exact return structure. However, the parameter schema and the simple search intent make the tool adequately understandable. It might benefit from pointing to siblings or describing the result type, but current context is partly sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All six parameters are described in the schema (100% coverage). The description mentions the semantic groups (keyword, speaker, meeting, period) but does not add details beyond what the schema provides; the only incremental value is noting that the conditions are combinable. This warrants the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a concrete action: searching speeches in the Japanese Diet records. It names the key filter dimensions (keyword, speaker, meeting name, date period), which clearly differentiates this from summarizing or analyzing tools in the sibling list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use the tool: when you need a list of speeches matching specific criteria. However, it does not explicitly state alternatives or exclusions, so while the context is clear, there is no direct 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_meetingA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool retrieves the 'entire' minutes, indicating a read operation and full-document return. However, it does not mention potential side effects, permissions, errors, or pagination, leaving some ambiguity for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose, followed by a practical usage note. Every sentence earns its place with no redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter get tool, the description is mostly complete: it states what it does and how to obtain the parameter. It does not describe return format, but the lack of an output schema and simplicity of the tool make this acceptable, though a hint about the response structure would improve it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description covers 100% of the parameter, explaining issueID as the identifier from 'search_speeches の items[].issueID'. The tool description repeats this same information, adding no additional meaning beyond the schema. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Specify the meeting minutes identifier (issueID) to retrieve the entire meeting minutes.' It uses a specific verb (取得/retrieve) and resource (会議録全体/entire minutes), and distinguishes from siblings by implying it returns the full document rather than summaries or analyses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to obtain the required issueID by instructing to use 'search_speeches の items[].issueID'. This implies a workflow (search first, then retrieve full minutes) and differentiates from siblings, though it does not explicitly exclude other tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_qa_pairsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It transparently states that it extracts and summarizes pairs and enumerates output fields, which gives a clear functional picture. However, it does not address cost, rate limits, or error behavior, which could affect usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately states the purpose, required input, and output format. It contains no filler and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 5-parameter tool with no output schema, the description covers the core functionality and output structure adequately. It lacks explicit error/cost context, but the schema's thorough parameter documentation and the clear output field listing make it sufficiently complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-level guidance beyond what the schema already provides, such as default values or parameter relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs '抽出・要約' and names the resource '質問と答弁のペア' with the required input issueID. It also lists the outputs, clearly differentiating it from sibling tools like summarize_meeting or summarize_speeches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying issueID and the task of QA pair summarization, but does not explicitly mention when not to use it or name alternative tools. The schema's issueID description references search_speeches, but that is outside the main description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.