Server Quality Checklist

Tool Scores

• search_repliesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the case-insensitive behavior and the return fields (username and text), but does not explicitly state that it is read-only, mention any side effects, or clarify scope (e.g., all threads or a specific thread).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with the primary purpose stated in the first sentence and a simple args section. No redundant information is present; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter, and the description provides the essential information: what it does, the parameter meaning, and return structure. Given the expected output schema, it does not need to elaborate further. Minor gaps like scope or sorting are not critical for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates by explaining the 'keyword' parameter as 'the search term to filter replies by'. This adds meaning beyond the raw schema field name, though it could be more detailed (e.g., accepted formats or behaviors).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches Threads replies by a keyword, with a specific verb and resource. It distinguishes itself from siblings like 'get_all_replies' by focusing on keyword filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'get_all_replies' or 'get_reply_stats'. The description simply describes the function without contextualizing it among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reply_statsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden of disclosing side effects and operational context. It mentions 'scraped replies,' implying a read-only operation on stored data, and lists return values. However, it does not explicitly state that the tool is read-only, requires prior scraping, or has no side effects, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded. The first sentence states the core purpose, and the second lists the output fields. No wasted words or redundant information, maintaining excellent structure for quick parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless tool with an output schema, the description is nearly complete. It lists the key return stats. However, it leaves minor context gaps, such as the prerequisite of prior scraping or the exact data scope, which could be clarified to make it fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter semantics to convey. As per the rubric, the baseline for 0-param tools is 4, and the description adds no unnecessary parameter detail since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states precisely what the tool does: 'Get statistics about the scraped replies.' It then enumerates the specific stats returned (reply count, unique users, most active commenters, time range), making it clear and distinct from sibling tools that scrape threads, fetch all replies, or search replies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied rather than explicit. The description indicates it operates on 'scraped replies,' suggesting it is meant for post-scrape analysis, but it does not explicitly say when to prefer this over alternatives like get_all_replies or search_replies. No exclusions or conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_repliesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return format (formatted list with username, timestamp, text) and the dependency on scrape_thread. However, it doesn't specify behavior on empty data or potential errors, which is a notable gap for a data retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the primary action. Every sentence earns its place, covering purpose, output format, and necessary prerequisite without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature (0 params, output schema exists), the description is fairly complete. It covers the return fields and the dependency on scrape_thread. The only gap is lack of explicit error/empty-list behavior, but overall it's sufficient for a straightforward getter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds context about the output and prerequisite, which is helpful even though there are no parameters to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets all scraped Threads replies, with a specific verb and resource. It distinguishes itself from sibling tools like search_replies by emphasizing 'all' replies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit prerequisite ('Call scrape_thread first if no data exists yet'), giving context on when to use the tool. It doesn't explicitly exclude alternatives or list when not to use it, but the 'all' wording implies a contrast with filtered searching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_threadA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool opens a browser, navigates, intercepts GraphQL responses, and saves data locally—behavioral traits beyond the schema. It also notes the 'public' constraint. However, it does not detail potential side effects like rate limits, file overwrites, or cleanup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: purpose, mechanism, prerequisite, then a simple argument list. Every sentence serves a function, and it front-loads the key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (browser automation, local save) and that an output schema exists, the description covers the prerequisite relationship and parameter details. It could be more complete by noting potential failure modes, but it is sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It does: url is described as a 'Full URL' with an example, and max_scrolls as 'Maximum scroll iterations for loading more replies (default: 20).' This adds meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states its function: 'Scrape replies from a public Threads post URL.' It also details the mechanism (opens a browser, intercepts GraphQL API responses) and differentiates itself from siblings by noting it must precede get_all_replies/search_replies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states a prerequisite condition: 'This must be called before using get_all_replies or search_replies,' which tells the agent when to use it relative to other tools. It also implies it is the initial scrape step and works only on public posts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.