Server Quality Checklist

Tool Scores

• fetch_weatherC
  2.6/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It fails to mention any traits like rate limits, authentication needs, data sources, or response format, making it inadequate for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that is front-loaded with the core purpose. There is no wasted text, making it highly concise and well-structured for its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not cover behavioral aspects, parameter details, or return values, leaving significant gaps for a tool that fetches data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 1 parameter, the description adds minimal meaning beyond the schema. It implies the 'city' parameter is used for fetching weather but does not specify format, constraints, or examples, failing to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('fetch') and resource ('current weather for a city'), making the purpose immediately understandable. However, it does not differentiate from sibling tools, which are unrelated to weather, so it misses the highest score for sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as for real-time data, specific weather metrics, or error handling. It lacks explicit context or exclusions, leaving usage implied at best.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ip_my_addressC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool gets an IP address but doesn't explain how it works (e.g., via an API call to outian.net), what the response format is, potential rate limits, or error handling. This leaves significant gaps in understanding the tool's behavior and reliability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single sentence, 'Get IP address from outian.net', which is front-loaded and wastes no words. It efficiently communicates the core function without unnecessary details, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple but with an undocumented parameter), lack of annotations, no output schema, and low schema coverage, the description is incomplete. It doesn't explain what the tool returns, how the parameter affects the result, or any behavioral aspects, making it inadequate for effective use without additional guesswork or documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter ('city') with 0% description coverage, and the tool description doesn't mention any parameters. This fails to compensate for the lack of schema documentation, leaving the parameter's purpose (e.g., whether it filters by city or something else) unexplained and potentially confusing for users.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose as 'Get IP address from outian.net', which includes a verb ('Get') and resource ('IP address'), but is vague about what specific IP address is retrieved (e.g., user's current IP, IP for a city). It doesn't distinguish from sibling tools like 'fetch_weather' or 'get_chat_spaces', which are unrelated, but the purpose is clear enough to understand the basic function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention any context, prerequisites, or exclusions, such as whether it's for geolocation, network diagnostics, or other use cases. Without this, users must infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• addC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. 'Add two numbers' implies a simple mathematical operation but reveals nothing about error handling, computational limits, side effects, or response format. For a tool with zero annotation coverage, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise at three words with zero wasted language. It's front-loaded with the core operation and contains no unnecessary elaboration. Every word earns its place in conveying the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 integer parameters, no annotations, no output schema), the description is incomplete. While the operation is clear, it lacks context about return values (sum format), error conditions, or any behavioral nuances. For even a simple tool, more completeness would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. 'Add two numbers' implies two numeric parameters but doesn't name them (a, b) or provide any semantic context beyond their role in addition. The description adds minimal value beyond what's inferable from the schema's parameter count and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Add two numbers' clearly states the tool's function with a specific verb ('Add') and resource ('two numbers'). It's unambiguous about what the tool does, though it doesn't differentiate from sibling tools (which appear unrelated). The purpose is immediately understandable without being tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions. While sibling tools seem unrelated (weather, chat, IP, messages), the description offers no usage context beyond the basic operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_chat_spacesA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the authentication requirement (OAuth), first-run behavior (browser window), and prerequisites (credentials.json file). However, it doesn't mention rate limits, pagination, or what specific data is returned about each space.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized but not optimally structured. The first sentence clearly states the purpose, but the following authentication details could be more efficiently integrated. While all information is relevant, the flow from purpose to authentication setup could be smoother.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides good coverage of authentication requirements but lacks information about return values, pagination, or error handling. For a list operation with no structured output documentation, more detail about what data is returned would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on authentication requirements which is the correct emphasis for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all Google Chat spaces') and resource ('Google Chat spaces the bot has access to'), distinguishing it from siblings like 'get_space_messages' which focuses on messages within spaces rather than listing spaces themselves. The verb+resource combination is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. While it implicitly suggests usage for listing accessible spaces, there's no mention of when NOT to use it or how it differs from potential sibling tools. The description focuses on authentication setup rather than usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_space_messagesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes authentication requirements ('requires OAuth authentication'), input format constraints (space_name format, date format), time range behavior (how start_date and end_date affect the query), and error conditions (raises ValueError). It doesn't mention rate limits, pagination, or what happens with large result sets, but covers core operational aspects well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured: it starts with the core purpose, then covers prerequisites and format details, explains behavioral logic, and concludes with parameter documentation and error handling. Every sentence adds value without redundancy, and information is logically organized for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% schema description coverage, the description does an excellent job covering authentication, input formats, temporal logic, and errors. However, it doesn't describe the structure of returned message objects (only mentions 'List of message objects'), which would be helpful since there's no output schema. For a read-only listing tool, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining all three parameters: space_name (format requirements), start_date (required, format, behavior when used alone), and end_date (optional, format, behavior when combined with start_date). It adds crucial meaning beyond the bare schema, including format specifications and temporal logic.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List messages') and resource ('from a specific Google Chat space') with additional scope ('with optional time filtering'). It distinguishes from sibling tools like 'get_chat_spaces' (which likely lists spaces rather than messages) and 'add' or 'fetch_weather' (completely different domains).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (for listing messages from a Google Chat space with time filtering) but doesn't explicitly state when NOT to use it or mention alternatives among sibling tools. It doesn't compare with other message-fetching tools if they exist, though the sibling list doesn't suggest direct alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.