Server Quality Checklist

Tool Scores

• get_market_evidenceA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds that data is 'normalized' and from 'stored research', but does not disclose return format, pagination behavior, or the effect of active_only. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence. It front-loads the main action and resource, then lists filters without unnecessary detail. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters, no output schema, and no required parameters, the description is too brief for fully correct invocation. It does not explain limit behavior, active_only semantics, the advertiser/competitor distinction, or the structure of the returned evidence. More context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions run, market, category, and 'competitor' filters, but omits limit and active_only. The term 'competitor' does not directly match the schema parameter 'advertiser', creating potential ambiguity for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Retrieve' and clearly identifies the resource: 'normalized ads and advertisers from stored research'. It also mentions optional filters (run, market, category, competitor), which helps distinguish it from siblings like search_meta_ads (likely live search) and get_research_run (run metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving stored research data with filters, but it does not explicitly state when to use this tool instead of alternatives or any exclusions. It provides no direct guidance on choosing between siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_competitor_adsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already communicate the safety profile (non-readonly, non-destructive, open-world, non-idempotent). The description adds research scope but does not disclose further behaviors like latency, data freshness, or side effects, so it provides only modest added transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action verb, and generally concise. The list of content areas is somewhat long but not wasteful, making it effectively sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool appears complex with multiple parameters and no output schema. The description lists what the research includes, but omits parameter details and return structure, leaving partial gaps for an agent deciding whether to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the meanings of 'markets' or 'results_per_market'. Only 'one competitor' indirectly refers to the competitor parameter, so the description fails to compensate for the lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run') and resource ('deep Meta Ads research pass for one competitor'), and lists specific content areas (creative mix, CTAs, landing domains, localization, social metadata, longest-running ads), which distinguishes it from sibling tools like search_meta_ads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when deep, single-competitor research is needed, which provides clear context. However, it does not explicitly name alternative tools or include exclusion criteria, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_research_runA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so safety is covered. The description adds meaningful context by specifying 'durable' and listing the returned contents (metadata, inputs, metrics, summary), which gives the agent a clear idea of what to expect from the response. However, it does not mention error behavior (e.g., missing run_id) or response formatting, which would be additional value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 13 words, front-loaded with the verb 'Retrieve' and the resource. Every word adds specificity (durable, metadata, inputs, metrics, summary). There is no redundancy or filler, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter, no output schema, and rich annotations, the description is largely complete. It tells the agent exactly what data will be returned and that it pertains to a past run. The main missing piece is parameter semantics, which is covered separately, but overall the description combined with annotations gives a solid understanding of the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description bears full responsibility for explaining the parameter. It does not explicitly mention 'run_id' or explain that it identifies the research run to retrieve. The phrase 'previous research run' implies it, but the tool name and description do not directly connect the single parameter to an identifier format or how to obtain it. This is a significant gap given only one parameter exists.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Retrieve' and clearly identifies the resource as 'a previous research run', listing the exact contents (durable metadata, inputs, metrics, summary). This distinguishes it from siblings like get_market_evidence or list_sources, which serve different retrieval purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool vs alternatives. It does not mention get_market_evidence or other related tools, nor does it state prerequisites or situations where another tool would be more appropriate. The phrase 'previous research run' implies use after a run exists, but no comparative direction is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_meta_adsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint false, but the description adds meaningful context: it discloses the paid nature of the underlying Actor and the side effect of storing normalized evidence. This goes beyond annotations, though it does not cover rate limits or authentication, which are less critical given the external API context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core purpose and followed by a key operational detail. Every word contributes, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does not clarify what the tool returns, which is a notable gap. It does provide context about paid execution and evidence storage, which helps, but it lacks guidance on output shape and relationship to sibling tools, making it only moderately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 40%, and the description does little to compensate. It mentions 'by keywords and market,' which aligns with the queries and markets parameters, but it does not explain active_only, results_per_query, or the significance of deep beyond the schema's own description. The description adds marginal value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search public Meta Ad Library data by keywords and market.' This is a specific verb+resource+scope and distinguishes it from siblings like analyze_competitor_ads and get_market_evidence, which focus on analysis or retrieval of stored evidence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning 'runs a paid Apify source Actor and stores normalized evidence for reuse,' hinting that this fetches fresh data for later retrieval. However, it does not explicitly state when to use this tool versus alternatives like get_market_evidence or analyze_competitor_ads, leaving the usage context implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• research_ai_employee_marketA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint=false, openWorldHint=true) already communicate the non-read-only, open-world nature. The description adds behavioral detail about using both Arabic and English queries and producing a 'clearly caveated' score, but does not disclose runtime, dependencies beyond schema, or potential side effects beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with scope and outputs, no redundant wording. Every sentence earns its place, making it highly efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers core inputs and outputs at a high level, which is adequate for a research tool with annotations present. However, it omits results_per_query semantics and does not detail what 'market metrics' or 'evidence' include, leaving some ambiguity without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 25% (only category_id has a description), so the description compensates by explaining taxonomy category, markets (EG/SA/UAE), and languages (Arabic/English). However, results_per_query is not addressed in the description. This provides meaningful added meaning for three of four parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool researches one taxonomy category across EG/SA/UAE using Arabic and English queries, and specifies outputs (market metrics, evidence, opportunity-screening score). This verb+resource+scope structure distinguishes it from sibling tools like compare_ai_employee_opportunities and get_market_evidence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a single-category, multi-market research use case but does not explicitly state when to prefer it over alternatives (e.g., compare_ai_employee_opportunities) or when not to use it. No exclusions or alternative tool references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sourcesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds a valuable behavioral guarantee: 'Never exposes secrets.' This provides context beyond the structured annotations and reassures the agent about data sensitivity, though it doesn't mention return-format or size limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that lists all output categories and ends with a critical safety note. Every word earns its place; no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description enumerates the major content areas (sources, storage status, taxonomy, research runs), which sufficiently conveys what the agent can expect. It doesn't specify exact formats, but for a no-parameter listing tool, this coverage is strong enough. Missing a little detail on how 'recent' is defined, but acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides complete coverage at 100%. With no parameters, the baseline is 4 per the rubric; the description doesn't need to add parameter details, and it correctly focuses on what the tool returns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') and enumerates exactly what the tool lists: connected and planned sources, storage status, AI employee taxonomy, and recent research runs. This clearly distinguishes it from the more focused sibling tools (search_meta_ads, analyze_competitor_ads, etc.) which target single operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the overview tool for seeing all sources and related metadata, but it does not explicitly state when to use it instead of a sibling or provide alternative suggestions. It lacks explicit when/when-not guidance, so it falls at the 'implied usage' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_ai_employee_opportunitiesA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavioral context beyond this: it guarantees no paid source runs and explains that missing research will be reported explicitly. This enriches the agent's understanding without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two crisp, front-loaded sentences. The first sentence states the core function, and the second provides a crucial behavioral boundary. Every word adds value with zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, one required), the description covers the main purpose, the no-cost guarantee, and the missing-data behavior. It lacks any description of the return value format, but the explicit statement about missing research reporting partially compensates. The rich annotations also reduce the overall burden.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter explanation. It only hints at category_ids via 'two or more AI employee categories' but completely ignores the 'markets' parameter and does not explain its purpose, allowed values, or default behavior. The schema provides no description either, leaving the agent guessing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Compare two or more AI employee categories using stored evidence.' It uses a specific verb (compare) and specific resource (AI employee categories). The second sentence differentiates it from tools that start paid research, distinguishing it from siblings like research_ai_employee_market.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it compares categories using stored evidence and explicitly states that it 'Does not start paid source runs,' indicating when not to use it. However, it does not explicitly name sibling alternative tools or provide a full when-to-use vs. when-not-to-use matrix.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.