Server Quality Checklist

Tool Scores

• list_commoditiesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full burden. It does not disclose authentication needs, rate limits, or the nature of the data (e.g., live vs cached). The search parameter's effect is not explained, and the output format is not hinted, even though an output schema exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no extraneous words. However, it sacrifices completeness for brevity, missing key details about parameters and behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one optional parameter and an output schema, the description is insufficient. It does not mention the search filter, output structure (e.g., list of commodity names/objects), or temporal scope (today's data). Sibling tools are not referenced, so the agent lacks context for correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, and the description does not explain the purpose of the 'search' parameter. While the schema shows it is an optional string that can be null, the description adds no semantic value, such as indicating it filters commodities by name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'available commodities in today's Agmarknet data', which distinguishes it from sibling tools like list_markets (which lists markets) and others focused on prices. However, it could be more precise about the temporal scope (e.g., 'current date's data') to avoid ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives like list_markets or get_commodity_price. There is no mention of prerequisites, limitations, or comparison to siblings, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_marketsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not disclose important behavioral traits such as filtering behavior (despite parameters), data freshness (today only), or any limitations. No annotations are present, so the description should compensate but fails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short and to the point, but it omits crucial details about parameters and usage context. It is concise but not sufficiently informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is too brief. It does not cover the filtering functionality hinted by the parameters, nor does it differentiate from sibling tools. The output schema exists but is not mentioned. Overall, it lacks completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not describe or even mention the parameters (state, commodity). Since schema coverage is 0%, the description should explain what these parameters do, but it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list), the resource (markets/mandis), and the data context (today's Agmarknet data). It is distinct from sibling tools like list_commodities and get_commodity_price.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage context or comparison to alternatives is provided. The description does not help the agent decide when to choose list_markets over its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_price_summaryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description bears full responsibility for behavioral disclosure. It mentions the return type (summary) but omits side effects, permissions, or behavior on missing data. The tool is likely read-only, but the description does not confirm.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and all words contribute. It is concise but could be slightly more structured (e.g., listing returned fields explicitly).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with 2 parameters and an output schema, so the description need not detail returns. However, it misses explaining that 'state' is optional and affects market filtering. Overall, it provides a minimal but adequate context for a straightforward tool, falling short on optional parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'commodity' implicitly and that results span 'all markets', but fails to clarify the 'state' parameter is optional or explain parameter formats. This adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a statistical summary (average, lowest, highest) of a commodity's price across all markets. It uses specific verbs and resources, and the purpose is distinguishable from sibling tools like get_commodity_price or compare_markets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. The description implies it's for aggregate statistics across markets but doesn't mention circumstances or prerequisites, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_commodity_priceB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but only states it returns min, max, and modal prices from today's data. It does not disclose behavior if data is missing, freshness guarantees, or any rate limits. This gap leaves the agent uncertain about edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, front-loaded sentences that efficiently communicate the tool's purpose and output. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite an output schema existence, the description lacks explanation of optional parameters and behavioral details. Given 4 parameters and no annotations, the description is insufficiently complete for agents to reliably invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description only refers to a 'specific commodity' without explaining the state, district, or market parameters. The required commodity parameter lacks format or example, failing to add value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves current daily wholesale prices for a specific commodity, specifying returns of min, max, and modal prices from Agmarknet data. It distinguishes itself from sibling tools like compare_markets or list_commodities by focusing on price retrieval for a single commodity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for getting commodity prices but does not provide explicit guidance on when to use this tool versus alternatives like compare_markets or get_price_summary. No exclusions or conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_marketsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description bears full responsibility. It discloses that results are sorted by modal price (cheapest first), which is useful. However, it does not mention whether the operation is read-only, any authentication needs, or potential side effects. The disclosure is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences plus a standalone line. Every sentence adds value, with no redundancy. It is front-loaded with the core action and then provides a key benefit.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (existing but not shown), so return details are not required. However, the description lacks explanation for 'modal price' and does not clarify the meaning of 'top_n' or 'state'. For a relatively simple tool, it covers the primary use but misses minor contextual details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description only indirectly references the 'commodity' parameter by mentioning 'a commodity.' The 'state' and 'top_n' parameters are not explained at all. With 0% schema description coverage, the description should compensate but does not, leaving the agent to infer semantics for two of three parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Compare prices of a commodity across different markets,' specifying the action (compare), resource (prices of a commodity), and scope (across markets). It distinguishes from siblings like get_commodity_price (single price) and list_markets (just listing) by focusing on comparative pricing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Great for finding the cheapest mandi' implies a specific use case, but there is no explicit guidance on when to use this tool versus alternatives (e.g., get_price_summary for aggregated stats). No exclusions or conditions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.