Server Quality Checklist

Tool Scores

• get_top_integrationsB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It includes a critical caveat that the data is 'intelligence, not ground truth' and recommends verification via source_urls. However, it omits other traits like rate limits or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, 30 words, immediately stating purpose then a crucial behavioral note. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no parameter descriptions, the description does not explain what return values look like (beyond mentioning source_urls) or how to construct valid requests. Essential context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no information about valid values for 'ecosystem' or 'use_case'. No enums, examples, or formats are provided, leaving the agent to guess.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets 'ranked integrations and MCP servers' with optional filtering by use case. It is specific about the resource and action, and distinct from siblings like 'find_mcp_servers' which likely searches unranked.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide guidance on when to use this tool versus alternatives such as 'find_mcp_servers' or 'search_ecosystem'. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_best_practicesA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description discloses data freshness (content_age_hours, data_freshness), cautions about time-sensitive info, and notes to verify against source_urls. Good transparency about reliability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, front-loaded with purpose, each adds value. Could slightly reduce length but overall well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple two-param tool without output schema. Mentions return fields (freshness, source_urls) but lacks explicit output structure or relationship between parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. Description mentions ecosystem and category but provides no valid values, examples, or constraints beyond what the param names imply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Get' and resource 'best practices' with scoping to ecosystem and category. Distinguishes well from siblings like get_latest_news or list_ecosystems.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when needing best practices, but no explicit when-to-use or when-not-to-use, nor mention of alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_mcp_serversA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description details behavioral traits: semantic similarity search, returned trust signals, capability flags, hosted endpoint, and important warnings about 'not ground truth' and verifying via source_urls. This goes beyond typical scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that front-loads the main purpose. While informative, it could be more concise by grouping related details; nevertheless, no unnecessary sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Considering the complexity (7 params, no output schema), the description explains key return fields and provides usage warnings. However, it omits detailed explanations for some parameters, leaving the schema to carry burden, which is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description only explains exclude_capability_flags and require_hosted. Parameters like category, limit, min_security_score, min_runtime_score are not described, leaving gaps for an agent to understand their semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search for MCP servers by use case or keyword using semantic similarity,' specifying verb and resource. It differentiates from sibling tools like get_top_integrations or list_ecosystems which focus on different entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on using exclude_capability_flags and require_hosted filters, and states that quarantined/archived servers are excluded. However, it does not explicitly contrast with sibling tools or specify scenarios where this tool should be avoided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_latest_newsA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses tier-dependent latency, item fields (content_age_hours, data_freshness), and the nature of data as intelligence rather than ground truth. This is comprehensive for a read-only tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three well-structured sentences, front-loaded with purpose, followed by important behavioral details. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, tier behavior, item fields, and verification guidance. Lacks explicit mention of the limit parameter and response structure, but the tool is simple and the description is sufficient for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has two parameters (ecosystem, limit) with 0% description coverage. Description implies ecosystem through the tool's purpose but does not mention the limit parameter at all, leaving its role undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get the latest news and updates for an AI ecosystem', specifying a concrete action and resource. It differentiates from siblings which focus on servers, practices, integrations, and ecosystem listing/search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides tier-based latency guidance (Pro tier real-time, Free tier 24h delay) and advises checking content_age_hours for time-sensitive information. Also includes a caution about verifying critical decisions via source_urls. Does not explicitly compare to alternative tools, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_ecosystemA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses that results are 'ranked by relevance' and warns that 'Strata provides intelligence, not ground truth' with a recommendation to verify against source_urls. This is sufficient transparency for a search tool, though it does not mention pagination or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose, followed by a usage tip and a crucial caveat. Every sentence provides value with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description is complete: it explains the search domain, how to use the optional parameter, and includes a caveat about verification. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaning to both parameters: 'query' is implied by the search context, and 'ecosystem' is explicitly explained with the instruction to leave blank for cross-ecosystem search. This compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search across all verified AI ecosystem content' with a specific verb and resource, and distinguishes from sibling tools like list_ecosystems and get_latest_news by emphasizing broad search and relevance ranking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage hint ('Leave ecosystem blank to search across all ecosystems') but does not explicitly specify when to use this tool over alternatives like find_mcp_servers or get_best_practices. Usage is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ecosystemsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  In absence of annotations, the description adequately implies a read-only operation (listing ecosystems) without side effects. It does not contradict any annotations. A slightly higher score is not warranted as it does not explicitly state idempotency or safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. All information is front-loaded and directly useful. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description is fully complete. It explains the tool's output (list of ecosystems), its dependency by other tools, and the action needed (call first).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, and schema coverage is 100%. The description adds no parameter specifics, which is acceptable as none are needed. Baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and clearly defines the resource 'AI ecosystems' with a scope qualifier 'on your current tier'. It differentiates from sibling tools by positioning itself as a prerequisite for specific data-fetching tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs the user to call this tool first to discover valid ecosystem slugs before using get_best_practices, get_latest_news, or get_top_integrations. Provides clear context on when and why to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.