Server Quality Checklist

Tool Scores

• list_pagesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states 'This is a read-only operation on local files — no network requests are made,' which is a valuable behavioral trait beyond the schema. It also explains the utility of word counts. Since annotations are absent, the description carries the burden and meets it well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is about 6 sentences, front-loading the main purpose. Each sentence adds value, though the sentence about word counts could be seen as slightly extraneous. Overall, it is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, read-only, with output schema) and good annotations coverage (even though annotations are missing, the description covers the essential), the description is complete. It covers purpose, parameters, and behavior adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by explaining the parameter: 'jsonl_path: Full path to the pages.jsonl file. If empty, defaults to <MARKCRAWL_OUTPUT_DIR>/pages.jsonl.' This adds meaning beyond the schema's title and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all pages from a previous crawl with their URLs, titles, and word counts.' It identifies the verb (list) and resource (pages from a crawl), and distinguishes from sibling tools like crawl_site, read_page, and search_pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this to get an overview of available content before searching or reading specific pages,' providing clear context. However, it does not explicitly mention when not to use or name alternative tools, though the context implies such distinctions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_dataA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It transparently discloses that the tool makes external API calls to OpenAI (requiring OPENAI_API_KEY), saves results to extracted.jsonl, and includes LLM attribution metadata. It also mentions token cost implications for sample_size. Missing details on error handling or rate limits, but overall provides good behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with a concise summary sentence, followed by a paragraph on how it works, a phrase on use cases, and a clear bullet-like list for arguments. Every sentence adds value, with no redundant or irrelevant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity (4 parameters, all optional, an output schema exists), the description is complete. It explains auto-discovery when fields are empty, the role of context, and environment requirements. Despite not detailing the output schema content, the presence of an output schema reduces the need for that detail. The description covers all essential behavioral and usage aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description thoroughly documents all four parameters in a dedicated 'Args' section: jsonl_path (full path, default location), fields (comma-separated, example, auto-discovery behavior), context (purpose, ignored when fields specified), and sample_size (default, effect on quality and cost). This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Extract structured fields from crawled pages using an LLM.' It uses a specific verb ('extract') and identifies the resource ('crawled pages') and method. It also lists use cases (competitive research, API documentation analysis, building structured datasets), effectively distinguishing it from sibling tools like crawl_site, list_pages, read_page, and search_pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use this for competitive research, API documentation analysis, or building structured datasets from unstructured web content.' It implies when to use the tool but does not explicitly state when not to use it or suggest alternatives among sibling tools. Thus, while helpful, it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawl_siteA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and excels: it discloses that the tool respects robots.txt, uses sitemap-first discovery, handles subdomains, and offers JS rendering. It also notes that only public, non-authenticated pages are fetched, which is critical behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a concise overview followed by clear parameter explanations in a bullet-like format. It is informative without being verbose, though some default values are repeated from the schema, which could be trimmed slightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the existence of an output schema, the description adequately explains the output (Markdown files and pages.jsonl) and how the output integrates with sibling tools. It covers input, behavior, and integration, making it complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% coverage, so the description must compensate, and it does thoroughly. Each parameter (url, output_dir, format, max_pages, include_subdomains, render_js) has a detailed explanation including meaning, defaults, and usage tips (e.g., 'Use lower values (10-20) for quick previews'). This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: crawl a website and save extracted content as clean Markdown or plain text. It specifies the verb 'crawl' and resource 'site', and distinguishes from siblings like search_pages and read_page by mentioning that the output_dir is used by those tools, forming a typical workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this tool when asked to research, read, analyze, or archive a website' and provides a typical workflow (crawl_site → list_pages or search_pages → read_page). It does not explicitly state when not to use it or mention alternatives beyond the workflow, but the guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_pagesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description fully discloses it is read-only, operates on local files, uses OR logic, ranks by match count, and returns snippets. No side effects or auth requirements left unstated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with summary then Args section. Slightly verbose but every sentence adds value. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, parameters, usage context, behavior, and output (URL, title, snippet). Output schema exists, so return details are sufficient. Complete for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, description thoroughly explains each parameter: query as OR keywords with example, jsonl_path default path, max_results with usage guidance. Greatly adds meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it searches crawled pages by keyword, specifies case-insensitive search across titles and text, and ranks results by matching words. It distinguishes from sibling tools like crawl_site, read_page, list_pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states it requires prior crawl_site call and is read-only with no network requests. However, does not explicitly compare to alternatives or state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pageA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses read-only operation on local files, no network requests, case-insensitive matching, and trailing slash tolerance. With no annotations, description carries full burden and does so thoroughly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections, concise yet comprehensive. Every sentence adds value, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers return format (Markdown/text, title, source URL), usage order, and behavioral traits. Complements the output schema effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Provides detailed meaning for both parameters: url (type, case-insensitivity, example) and jsonl_path (optional, default path). Adds significant value over the bare schema (0% coverage, only title/type).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it reads the full extracted content of a crawled page by URL, distinguishing from sibling tools like search_pages (snippets) and list_pages (listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using after search_pages to read full content, providing clear context. Lacks explicit when-not-to-use, but context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.