Server Quality Checklist

Tool Scores

• get_operation_detailsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a read operation ('Get detailed information'), which implies it's likely non-destructive, but doesn't specify permissions, rate limits, error handling, or what 'detailed information' entails (e.g., format, fields). This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences: a purpose statement and a parameter explanation. It's front-loaded with the main action, and there's no wasted text, though the parameter section could be more integrated for better flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a read tool with 1 parameter), no annotations, and no output schema, the description is incomplete. It doesn't explain what 'detailed information' includes, potential errors, or how this differs from sibling tools, leaving the agent with insufficient context to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds minimal semantics by explaining 'operation_number' as 'The operation number to query', but this is basic and doesn't clarify format (e.g., integer range, examples) or context (e.g., where to find this number). With 1 parameter and low coverage, it doesn't fully compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get detailed information') and resource ('about a specific IOL operation'), making the purpose understandable. However, it doesn't differentiate this tool from its siblings like 'get_operations' or 'get_historical_data', which might also retrieve operation-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose it over siblings like 'get_operations' (which might list multiple operations) or 'get_historical_data' (which could include operation details in a historical context), leaving the agent without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_past_week_performanceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action but lacks details on permissions, rate limits, response format, or potential side effects. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's appropriately sized and front-loaded, clearly stating the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a data retrieval tool with no annotations, no output schema, and 0% schema coverage), the description is incomplete. It doesn't explain what 'performance' entails, the return format, or how to handle errors, making it inadequate for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, and the description doesn't add any meaning beyond the schema. It mentions 'stock' but doesn't clarify the 'stock_symbol' parameter's format, constraints, or examples. With one undocumented parameter, the description fails to compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('past week performance of a stock'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'get_historical_data' or 'get_quote', which might also provide performance-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention siblings like 'get_historical_data' or 'get_quote', nor does it specify any prerequisites or exclusions, leaving the agent with minimal context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quoteC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool gets a 'current quote,' implying a read-only operation, but doesn't specify whether this requires authentication, has rate limits, returns real-time or delayed data, or what the output format looks like. For a financial data tool with zero annotation coverage, this leaves significant behavioral gaps unaddressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by a structured 'Args:' section with parameter details. There's no wasted text, and each sentence adds value. However, the formatting with quotes and line breaks could be slightly cleaner for direct parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, no output schema), the description is partially complete. It covers the purpose and parameters adequately but lacks behavioral details like authentication needs, rate limits, error handling, or output format. Without annotations or output schema, the description should do more to explain what the tool returns and how it behaves in edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for both parameters: it explains that 'symbol' is a stock symbol with an example ('GGAL'), and 'market' is a market identifier with a default value and example ('bCBA' for Argentina). However, with 0% schema description coverage, the schema provides only titles and types. The description compensates somewhat by giving examples and defaults, but doesn't fully document all possible values or constraints (e.g., valid symbol formats or market options).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get current quote for a stock' specifies the action (get) and resource (quote for a stock). It distinguishes from siblings like get_historical_data or get_portfolio by focusing on current quotes rather than historical data or portfolio information. However, it doesn't explicitly differentiate from all siblings (e.g., get_operation_details might also involve quotes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose get_quote over get_historical_data for historical context, or get_portfolio for aggregated holdings. There's no discussion of prerequisites, timing considerations, or explicit exclusions. The only implied usage is for current stock quotes, but no alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historical_dataB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action but doesn't cover critical aspects like whether this is a read-only operation, potential rate limits, authentication needs, error handling, or what type of historical data (e.g., prices, volumes) is returned. This leaves significant gaps for a tool with three parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence, followed by parameter details. It avoids unnecessary fluff, though the 'Args:' section could be slightly more integrated into the flow for better readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a historical data tool with three parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format (e.g., JSON array, CSV), data granularity (daily, hourly), or potential limitations (e.g., max date range), which are crucial for effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explains that 'symbol' is a stock symbol with an example ('GGAL'), and clarifies the date format ('YYYY-MM-DD') for both start_date and end_date. This compensates well for the schema's lack of documentation, though it could specify date ranges or validation rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'historical data for a stock', making the purpose specific and understandable. However, it doesn't distinguish this tool from potential sibling tools that might also retrieve historical data with different scopes or formats, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_past_week_performance' or 'get_quote' from the sibling list. It lacks context about use cases, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action is to 'Get' account status and balances, implying a read-only operation, but doesn't mention any traits like authentication requirements, rate limits, or what specific data is returned. This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose with no wasted words. It's front-loaded and appropriately sized for a simple tool, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'account status and balances' entails, such as the return format or any behavioral aspects like error handling. For a tool with no structured data to supplement it, this leaves too much ambiguity for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate, earning a baseline score of 4 for adequately handling the lack of parameters without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'IOL (invertironline) account status and balances', making the purpose specific and understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'get_portfolio' or 'get_profile_data', which might also relate to account information, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_portfolio' and 'get_profile_data' that might overlap in functionality, there's no indication of context, prerequisites, or exclusions to help an agent choose appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_operationsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies a read operation ('Get') but doesn't disclose behavioral traits like authentication requirements, rate limits, pagination, error handling, or response format. For a tool with no annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose followed by parameter details. It uses bullet points for parameters, making it scannable. However, the first sentence could be more polished, and some redundancy exists (e.g., 'optional filters' repeated in parameter list).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 3 parameters with 0% schema coverage, the description is moderately complete. It covers parameter semantics well but lacks behavioral context, usage guidelines, and output details. For a read operation with filtering, it's adequate but has clear gaps in guiding effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the input schema, which has 0% description coverage. It explains that parameters are optional filters, specifies date formats (YYYY-MM-DD), and lists possible status values ('pendientes', 'terminadas', 'canceladas'). This compensates well for the schema's lack of descriptions, though it doesn't cover all edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'IOL account operations', making the purpose understandable. It distinguishes this from siblings like get_account_status or get_portfolio by specifying it retrieves operations rather than status or holdings. However, it doesn't explicitly differentiate from get_operation_details or get_historical_data, which might overlap in scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_operation_details or get_historical_data. It mentions optional filters but doesn't specify use cases, prerequisites, or exclusions. Without context, an agent might struggle to choose between similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_portfolioB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'get', implying a read operation, but doesn't clarify aspects like authentication needs, rate limits, data freshness, or what specific portfolio data is returned. This leaves significant gaps for a tool that likely accesses sensitive financial information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the portfolio data includes, how it's structured, or any behavioral traits like error handling. For a financial data tool with no structured support, more context is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the input schema has 100% description coverage (though empty). The description doesn't need to add parameter details, so it meets the baseline for a parameterless tool. No additional semantic value is required or provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'get' and the resource 'IOL (invertironline) investment portfolio', making the purpose specific and understandable. However, it doesn't explicitly differentiate this tool from siblings like 'get_account_status' or 'get_profile_data', which might also retrieve financial data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone among similar-sounding siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_profile_dataB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states 'Get' which implies a read operation, but doesn't mention any behavioral traits such as authentication needs, rate limits, response format, or whether it's idempotent. This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'profile data' includes, how it's returned, or any behavioral context, which is inadequate for a tool that likely returns structured user data. More detail is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description doesn't add parameter details, but with no parameters, a baseline of 4 is appropriate as there's nothing to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('IOL profile data'), making the purpose unambiguous. However, it doesn't differentiate from siblings like 'get_account_status' or 'get_portfolio', which likely retrieve different subsets of profile-related data, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_account_status' or 'get_portfolio'. The description implies it retrieves general profile data but doesn't specify context, prerequisites, or exclusions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.