Server Quality Checklist

Tool Scores

• analyze_pageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It describes what the tool analyzes (SEO factors) but doesn't disclose behavioral traits like whether it makes network requests, requires authentication, has rate limits, or what format the analysis results take. The mention of 'JavaScript rendering analysis' and 'basic load time measurement' hints at some behavior but lacks specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear introductory sentence followed by a bulleted list of analysis categories and a usage context sentence. Every element serves a purpose, though the bulleted list is somewhat lengthy (8 items) which slightly reduces conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no annotations and no output schema, the description provides good purpose and usage context but lacks behavioral details (how it works, what it returns) and doesn't compensate for the missing output schema. The complexity of analyzing 8 SEO factors suggests more completeness would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema (e.g., it doesn't explain how 'waitForSelector' interacts with the analysis or why 'device' matters for SEO). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes a single web page for specific SEO factors, listing 8 distinct categories. It distinguishes from siblings like 'analyze_sitemap' (multiple pages) and 'crawl_site' (site-wide crawling) by focusing on detailed analysis of a single page.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('detailed analysis of specific pages like job detail pages, landing pages, or homepage'), which implicitly suggests it's not for bulk analysis or sitemap-level work. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_urlsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns (status code, redirect destination, response time) which is helpful, but doesn't mention important behavioral aspects like error handling, rate limits, authentication requirements, or what happens with invalid URLs. The description adds some value but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise - a clear purpose statement followed by bulleted use cases. Every sentence earns its place, with no wasted words or redundant information. The information is front-loaded with the core functionality stated first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, 100% schema coverage, but no annotations and no output schema, the description provides adequate context about what the tool does and when to use it. However, it doesn't describe the return format in detail (beyond mentioning what fields are included) or address potential edge cases, leaving room for improvement given the lack of structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters (urls and timeout) adequately. The description doesn't add any parameter-specific information beyond what's in the schema, such as URL format requirements or timeout implications. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check HTTP status codes') and resource ('for a list of URLs'), distinguishing it from sibling tools like analyze_page or crawl_site which focus on content analysis rather than status verification. It provides a precise verb+resource combination that immediately conveys the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit use cases ('Verify expired job pages are handled correctly', 'Check for broken links', 'Analyze redirect chains') that guide when to use this tool. However, it doesn't specify when NOT to use it or mention alternatives among the sibling tools, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawl_siteA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by describing what the tool returns (aggregated statistics, page classification, duplicate detection, etc.) and mentions respecting crawl limits and delays. However, it doesn't specify error handling, timeout behavior, or what happens when limits are exceeded.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core functionality. Every sentence adds value: the first explains what it does, the second lists returns, and the third provides usage context. There's no wasted text, and it's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-page crawling with analysis), no annotations, and no output schema, the description does a good job explaining the return values and use case. However, it could be more complete by detailing output structure or error scenarios, though the listed returns provide solid context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline of 3. It doesn't compensate for gaps because there are none in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('crawl multiple pages', 'discovers internal links', 'analyzes each page') and distinguishes it from siblings like analyze_page (single page) or analyze_sitemap (structured sitemap). It explicitly mentions comprehensive site audits as the primary use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for comprehensive site audits') and mentions respecting crawl limits and delays as operational constraints. However, it doesn't explicitly contrast when to use this versus alternatives like check_urls or sample_pages, though the comprehensive nature is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_auditA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: caching ('captures pages ONCE'), output generation ('saves everything to reports/[sitename]/ folder'), and workflow steps. It could improve by mentioning potential side effects like network usage or time requirements, but it covers most critical aspects well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a bolded summary upfront and a numbered list detailing steps, making it easy to scan. It could be slightly more concise by reducing repetition (e.g., 'cached' mentioned multiple times), but overall, it's efficient and front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no output schema, no annotations), the description provides substantial context: it explains the workflow, output storage, and return values ('Returns comprehensive findings and prioritized fix recommendations'). It could enhance completeness by detailing error handling or performance implications, but it's largely sufficient for understanding the tool's scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no specific parameter information beyond what the schema provides, such as explaining how parameters interact or affect the audit process. This meets the baseline for high schema coverage but doesn't add extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run a complete SEO audit with automatic sampling, caching, and report generation.' It specifies the verb ('Run'), resource ('SEO audit'), and scope ('complete'), distinguishing it from siblings like analyze_page or crawl_site by emphasizing its comprehensive, orchestrated workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool by describing it as 'the main audit tool that orchestrates the entire workflow,' suggesting it's for full audits rather than partial analyses. However, it lacks explicit guidance on when to use alternatives like plan_audit or sample_pages, which could help differentiate usage scenarios more clearly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_lighthouseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: it lists the comprehensive return data (scores, metrics, opportunities, diagnostics), mentions the prerequisite (Lighthouse CLI installation), and describes the mobile/desktop execution approach. However, it doesn't cover rate limits, error conditions, or execution time expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise: it starts with the core purpose, then details the returns in a bulleted format, provides usage guidance, and ends with a prerequisite note. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (performance audit with multiple metrics) and no output schema, the description does an excellent job explaining what will be returned. However, without annotations covering safety/behavioral aspects and no output schema, it could benefit from more detail about error handling or execution characteristics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for high schema coverage situations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Run a Lighthouse performance audit') on a specific resource ('on a URL'), distinguishing it from sibling tools like analyze_page or crawl_site by focusing on comprehensive performance analysis with specific metrics and categories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for performance analysis') and advises running separately for mobile/desktop if both matter, but doesn't explicitly state when not to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan_auditA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by describing the multi-step process (discover sitemaps, identify patterns, estimate counts, generate strategy). It mentions what the tool returns (sitemap validation, route classification, sampling strategy, issues/warnings) and its non-destructive nature (analysis only). However, it doesn't specify performance characteristics like rate limits or timeout behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, process steps, returns, usage guidance) and front-loads the most important information ('RECOMMENDED FIRST STEP'). While comprehensive, it could be slightly more concise by combining some of the bullet points about returns into a single sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 100% schema coverage, and no output schema, the description provides excellent context about what the tool does, when to use it, and what it returns. It explains the multi-step analysis process and connects to sibling tools. The main gap is the lack of output schema, but the description compensates by listing return categories.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters adequately. The description doesn't add any additional parameter semantics beyond what's in the schema - it focuses on the tool's purpose and outputs rather than explaining parameter usage. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('analyze sitemaps', 'create sampling strategy') and resources ('large sites', 'job boards'). It explicitly distinguishes from siblings by recommending use 'BEFORE crawl_site or sample_pages' and mentions alternative tools like Lighthouse analysis, showing clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('RECOMMENDED FIRST STEP', 'essential for job boards and large sites with 100k+ pages', 'BEFORE crawl_site or sample_pages') and when not to use it (for small sites or direct analysis). It names specific alternative tools (crawl_site, sample_pages, Lighthouse) for different scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sample_pagesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well: it explains the sampling methodology (random + oldest + newest, specific sample ranges), output structure (detailed analysis, aggregated issues, cross-cutting findings, ranked issues), and the strategic purpose (finding template-level issues). It doesn't mention performance aspects like rate limits or auth needs, but provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with purpose first, usage guidance second, then concrete examples and returns. The bullet points efficiently illustrate the sampling approach and output format. Slightly verbose in the example section, but every sentence adds value for understanding scope and results.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, 100% schema coverage, no annotations, and no output schema, the description provides excellent context: clear purpose, usage timing, behavioral methodology, and output format. It compensates well for the lack of output schema by detailing what the tool returns. Minor gap: doesn't mention error handling or performance constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description doesn't add specific parameter semantics beyond what's in the schema (e.g., it mentions 'route types' generally but doesn't elaborate on the routeTypes parameter). Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'intelligently sample[s] and analyze[s] pages based on an audit plan', specifying both the action (sample and analyze) and resource (pages). It distinguishes from siblings like analyze_page (single page) or crawl_site (full crawl) by emphasizing representative sampling across route types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use this AFTER plan_audit to analyze representative pages from each route type', providing clear when-to-use guidance. It also distinguishes from alternatives by explaining the sampling approach versus crawling all pages, helping the agent choose between this and tools like crawl_site or check_urls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_sitemapA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool returns (robots.txt rules, sitemap details, URL counts, freshness analysis, recommendations) and its role as a first-step analysis tool. However, it lacks details on potential limitations, such as rate limits, authentication needs, or error handling, which would be beneficial for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, starting with the core purpose, listing outputs in bullet-like format, and ending with usage guidance. Every sentence adds value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (analyzing sitemaps with multiple outputs) and lack of annotations or output schema, the description does a good job covering what the tool returns and its purpose. However, it could be more complete by mentioning potential outputs like error states or limitations, which would help the agent handle edge cases better.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters (baseUrl, includeSitemapUrls, maxUrls) with clear descriptions. The description does not add any parameter-specific semantics beyond what the schema provides, such as explaining how maxUrls affects processing or default behaviors. Baseline 3 is appropriate when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes robots.txt and XML sitemaps, listing specific outputs like robots.txt rules, sitemap discovery, URL counts, freshness analysis, and job board recommendations. It distinguishes itself from siblings like analyze_page, check_urls, and crawl_site by focusing on sitemap analysis rather than page-level or broader crawling tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use this as a first step to understand site structure before crawling,' providing clear when-to-use guidance. It positions this tool as a precursor to crawling or auditing tools like crawl_site, plan_audit, or run_audit, helping the agent choose this over alternatives for initial site assessment.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.