Server Quality Checklist

Tool Scores

• get_sentiment_balanceB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool retrieves data and describes the return format, but lacks details on permissions, rate limits, error handling, or whether it's a read-only operation. For a data retrieval tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, with clear sections for purpose, parameters, usage, and returns. Every sentence adds value without redundancy, and it's front-loaded with the core function. There's no wasted text, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with 0% schema coverage and no output schema, the description does a decent job by explaining parameters and the return format. However, it lacks details on behavioral aspects like authentication or limits, and doesn't fully address sibling tool differentiation. For a simple retrieval tool, it's adequate but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics: it explains that 'asset' is a 'cryptocurrency slug' with examples, and 'days' is for 'calculating the average sentiment balance' with a default. This clarifies beyond the bare schema, though it could provide more context on valid asset values or day ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve the sentiment balance (sentiment_balance_total) for a given asset.' It specifies the verb ('retrieve') and resource ('sentiment balance'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like get_social_dominance or get_social_volume, which likely retrieve different social metrics for assets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'Usage' section that says 'Use this tool to get the average sentiment balance (positive minus negative sentiment) over a period,' which implies when to use it. However, it doesn't provide explicit guidance on when to choose this tool over alternatives (e.g., get_social_dominance for dominance metrics) or any exclusions, leaving usage context somewhat implied rather than fully articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_social_dominanceA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool retrieves data (read-only behavior) and explains what social dominance represents. However, it doesn't mention potential limitations like rate limits, authentication requirements, data freshness, or error conditions. The description adds some behavioral context but leaves gaps for a data retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (description, parameters, usage, returns) and front-loaded the core purpose. Most sentences earn their place, though the 'Usage' section slightly repeats the purpose. It's appropriately sized for a 2-parameter tool without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with 0% schema coverage and no output schema, the description does a decent job explaining parameters and return format. However, for a data retrieval tool with no annotations, it lacks details on error handling, data sources, or example responses beyond a single string format. The completeness is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clearly explains both parameters: 'asset' as a cryptocurrency slug with examples, and 'days' as the number of days for calculating average social dominance with a default value. This adds meaningful semantics beyond the bare schema. However, it doesn't specify constraints like valid slug formats or day ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve the social dominance (social_dominance_total) for a given asset' and explains what social dominance measures. It uses specific verbs ('retrieve', 'shows') and identifies the resource ('asset'), but doesn't explicitly differentiate from sibling tools like get_social_volume or get_sentiment_balance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'Usage' section stating 'Call this tool to get the percentage of social media discussion dominated by the asset,' which provides implied context about when to use it. However, it doesn't explicitly mention when NOT to use it or suggest alternatives among the sibling tools (e.g., when to choose get_social_volume instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• alert_social_shiftA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool does (detects shifts based on percentage change) and the return format (a string indicating shift occurrence). However, it doesn't mention important behavioral aspects like rate limits, authentication requirements, data freshness, or error conditions that would help an agent use it correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, parameters, usage, returns) and every sentence earns its place. It's appropriately sized at 4 sentences plus parameter explanations, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema description coverage, and no output schema, the description does a decent job explaining purpose, parameters, and returns. However, for a tool that performs analysis and returns alerts, it lacks details about the algorithm (e.g., how baseline is calculated), time windows for comparison, and potential limitations that would help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It provides clear semantic explanations for all three parameters: 'asset' is explained as 'cryptocurrency slug', 'threshold' as 'minimum percentage change', and 'days' as 'number of days to analyze for baseline volume'. Default values are also documented. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('detect significant shifts'), resource ('social volume for a given asset'), and scope ('spikes or drops in social_volume_total'). It distinguishes from sibling tools like get_social_volume (which presumably returns raw volume) by focusing on change detection and alerts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Usage' section explicitly states when to use this tool ('to check if the latest social volume has significantly spiked or dropped compared to the previous average'). However, it doesn't provide guidance on when NOT to use it or mention alternatives among the sibling tools (e.g., when to use get_social_volume instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_trending_wordsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's read-only nature (implied by 'Retrieve'), the ranking method ('aggregated and ranked by score'), and the return format. However, it doesn't mention potential limitations like data sources, rate limits, or freshness of data, which would be helpful for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with distinct sections (purpose, parameters, usage, returns). Each sentence earns its place by providing essential information without redundancy. The front-loaded purpose statement immediately clarifies the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, no annotations, and no output schema, the description does a good job covering purpose, parameters, usage, and return format. However, it could be more complete by addressing potential behavioral aspects like data sources or limitations, given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It provides clear semantic meaning for both parameters: 'days' specifies the analysis period, and 'top_n' controls how many results to return. Default values are also explained. The description adds significant value beyond the bare schema, though it doesn't detail constraints like valid ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve'), resource ('top trending words in the crypto space'), and scope ('over a specified period, aggregated and ranked by score'). It distinguishes this tool from siblings like get_sentiment_balance or get_social_volume by focusing on word trends rather than sentiment, dominance, or volume metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Usage' section provides clear context: 'Call this tool to get a list of the most popular words trending in cryptocurrency discussions, ranked across the entire period.' This gives guidance on when to use it, but doesn't explicitly mention when not to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_social_volumeA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool performs a calculation based on social data documents (telegram messages, reddit posts) and mentions the default value for days. However, it doesn't cover important behavioral aspects like rate limits, authentication requirements, data freshness, or error conditions that would be crucial for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, parameters, usage, returns) and every sentence earns its place. The front-loaded purpose statement is specific, and subsequent sections efficiently provide necessary details without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, no output schema), the description is mostly complete. It covers purpose, parameters, usage, and return format. However, without annotations or output schema, it could benefit from more behavioral context about limitations or edge cases to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed parameter semantics. It explains that 'asset' is a cryptocurrency slug with examples ('bitcoin', 'ethereum'), clarifies it's required, and explains that 'days' specifies the time period to sum with a default of 7. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('retrieve the total social volume'), the resource ('for a given asset'), and the calculation method ('calculates the total number of social data text documents that contain the given search term at least once'). It distinguishes from siblings by focusing on total volume rather than sentiment, dominance, or trending words.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Usage' section explicitly states when to call this tool ('to get the total number of social media mentions for an asset over a period'), providing clear context. However, it doesn't mention when NOT to use it or explicitly name alternatives like get_sentiment_balance or get_social_dominance for different analytical needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.