AIMLPM/markcrawl

Tool Scores

• extract_dataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full disclosure burden and succeeds well: warns about external OpenAI API dependency, required OPENAI_API_KEY environment variable, output file location ('extracted.jsonl'), and cost implications ('cost more tokens'). Could improve by mentioning error handling or rate limiting behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with logical flow: purpose → mechanism → behavioral warnings → use cases → detailed args. The Args section is necessary given zero schema coverage. Slightly verbose in the opening but every section adds distinct value not present in structured fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0% schema coverage, no annotations, and presence of output schema, the description successfully covers: input file handling, output file destination (extracted.jsonl), authentication requirements, and the dual-mode operation (manual fields vs. auto-discovery). Adequately complete for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage (only titles provided). The Args section compensates comprehensively: documents all 4 parameters with types, defaults, format examples (comma-separated fields), and behavioral notes (context is ignored when fields specified, sample_size tradeoffs). Fully bridges the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with specific verb ('Extract'), clear resource ('structured fields from crawled pages'), and mechanism ('using an LLM'). The LLM-based extraction clearly distinguishes this from siblings like read_page or list_pages which presumably handle raw content rather than structured extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit use cases ('competitive research, API documentation analysis, or building structured datasets') that clarify when to use extraction over simple reading. Documents the conditional logic for auto-discovery vs. specified fields. Lacks explicit comparison to sibling alternatives (e.g., 'use this instead of read_page when...').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pageA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses critical safety info ('read-only operation on local files — no network requests'), return format ('Markdown or text'), and matching behavior ('case-insensitive and tolerates trailing slashes'). Could add error conditions for missing URLs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with purpose front-loaded, followed by return value, usage context, behavioral notes, and Args section. Each sentence earns its place. Slightly verbose in Args section but necessary given zero schema coverage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, description appropriately focuses on invocation context rather than return values. Covers purpose, prerequisites, parameter semantics, and safety properties. Missing only edge case handling (e.g., what happens if URL not found in index).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. Description fully compensates: url includes constraints ('exact URL', 'must match previous crawl', 'case-insensitive') and example; jsonl_path explains default behavior ('defaults to <WEBCRAWLER_OUTPUT_DIR>/pages.jsonl'). Adds essential meaning missing from schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb 'Read' + resource 'extracted content of a specific crawled page' + scope 'full content of a single page'. Clearly distinguishes from sibling search_pages (finds results vs reads them) and crawl_site (crawls vs reads existing data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use this after search_pages to read the full content of a relevant result', establishing clear workflow order. Also notes prerequisite 'Must match a URL from a previous crawl'. Lacks explicit 'when not to use' alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawl_siteA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: robots.txt compliance, sitemap-first discovery, headless Chromium option for SPAs (with performance warning 'Slower but necessary'), and authentication constraints ('Only public, non-authenticated pages will be fetched'). Minor gap: does not mention rate limiting or error handling behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with logical flow: purpose → usage context → dependencies → workflow → parameters. Information is front-loaded with the core action. Despite length, every sentence serves a necessary function given the lack of schema documentation and annotations. Minor deduction for slight verbosity in the Args section, though justified by the 0% schema coverage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive coverage for a 6-parameter tool with zero annotations and zero schema descriptions. Addresses the tool's role in the ecosystem (workflow integration), output artifacts (.md files and pages.jsonl), and behavioral constraints. Since an output schema exists (per context signals), the brief mention of output files is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, requiring the description to compensate fully. The Args section provides rich semantic detail for all 6 parameters: examples ('https://docs.example.com/'), value guidance ('Use lower values (10-20) for quick previews'), format implications ('preserves headings, code blocks'), and technology context ('Required for React/Vue/Angular sites').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Crawl a website and save extracted content as clean Markdown or plain text'), clearly distinguishing it from siblings like 'read_page' (single page) or 'extract_data' (data extraction). It further clarifies the scope by mentioning boilerplate stripping and robots.txt respect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('when asked to research, read, analyze, or archive a website') and provides a clear workflow sequence ('crawl_site → list_pages or search_pages → read_page'). Critically, it notes the dependency relationship: 'The output_dir from this tool is required by search_pages, read_page, list_pages, and extract_data', which is essential for correct orchestration.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_pagesA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden of behavioral disclosure. It explicitly states 'read-only operation on local files' and 'no network requests are made,' addressing safety and execution context. It also clarifies that it returns a summary of 'every page in the crawl index.' Minor gap: doesn't mention error handling (e.g., file not found) or performance with large files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with front-loaded key details (URLs, titles, word counts in first sentence). Zero waste: each sentence addresses functionality, return value, usage timing, data interpretation (word counts), safety characteristics, and parameter details. The Args section efficiently documents the single parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (so return values needn't be detailed) and only one optional parameter, the description is comprehensive. It covers purpose, workflow context, safety constraints, and parameter semantics—sufficient for an agent to invoke correctly without external documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (parameter lacks description field), but the description fully compensates by documenting: 'Full path to the pages.jsonl file' and the default behavior 'If empty, defaults to <WEBCRAWLER_OUTPUT_DIR>/pages.jsonl.' This provides complete semantic meaning where the schema fails to do so.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states the verb (List), resource (pages from a previous crawl), and return details (URLs, titles, word counts). It clearly distinguishes from siblings like crawl_site (emphasizes 'previous crawl' vs. new crawl) and read_page/search_pages (emphasizes 'overview' vs. specific content retrieval).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow guidance: 'Use this to get an overview of available content before searching or reading specific pages.' This establishes when to use the tool (before search/read operations) and implicitly references sibling alternatives (search_pages, read_page).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_pagesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, description carries full burden and excels: discloses read-only/local-file nature ('no network requests'), ranking algorithm ('ranked by the number of matching query words'), search logic ('case-insensitive', 'OR logic'), and result structure ('URL, title, and a text snippet').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with logical flow from general function to specific behavior to parameters. Length is appropriate given zero schema coverage necessitating detailed Args section. No redundant sentences, though slightly verbose compared to ultra-concise ideal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive for a search tool: covers search mechanics, ranking, result format, safety characteristics, file dependencies, and all three parameters. Output schema exists (per context signals), so brief mention of result contents is sufficient without detailing return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage (only titles), but description fully compensates: explains query syntax ('one or more keywords separated by spaces', example provided), jsonl_path context ('from a previous crawl', default location), and max_results usage guidance ('lower values for focused searches').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb+resource ('Search through previously crawled pages by keyword') clearly distinguishes from siblings: contrasts with crawl_site (which populates data) and implies difference from list_pages/read_page by specifying keyword search across content rather than enumeration or retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states prerequisite dependency: 'Requires a prior crawl_site call to have populated the pages.jsonl file.' This clearly defines when to use (after crawling) and implies when not to use (no crawl data available), providing essential workflow context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.