Skip to main content
Glama

search_and_crawl

Read-only

Search the web and crawl top results to extract full content, with optional Markdown and index persistence for later analysis.

Instructions

Search the web (engine as in search_web) and crawl top results. Combines search with full content extraction. Supply output_path (directory) in the request to persist per-page markdown (unsliced) + index.json and receive a slim response.

Input Schema

NameRequiredDescriptionDefault
requestYesDict with: search_query (required), crawl_top_results, search_genre (optional, one of: pdf, documents, presentations, spreadsheets, japanese, english; adds a filetype/language operator to the query, other values return an error), engine (optional, one of: auto, google_custom, brave, exa, searxng; default from SEARCH_ENGINE, auto tries the configured engines in SEARCH_ENGINE_ORDER and moves to the next one on errors other than zero results), result_type (optional; default web; one of web, discussions, faq, videos, news, or several as a list or a comma-separated string; everything except web runs on brave only; news is a separate Brave News Search request; results holds web when requested, else the first type, and the other types go to extra_results; discussions are forum posts with the forum name, answer count, post score, question and top comment; faq are question and answer excerpts; Brave returns these sections for some queries only, and num_results only shortens them), recent_days, generate_markdown, max_content_per_page. Optional persistence keys: output_path (absolute directory — per-page .md files + index.json, the full page bodies are written BEFORE max_content_per_page truncation; dot-containing dir names are fine), include_content_in_response (bool, default False — when True keeps crawled_pages in the response too, still subject to max_content_per_page truncation), overwrite (bool, default False). Failed pages appear in index.json with file=null.

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.4.2
    • changedInput schema / properties / request / description
      Previous value: -"Dict with: search_query (required), crawl_top_results, search_genre, recent_days, generate_markdown, max_content_per_page. Optional persistence keys: output_path (absolute directory — per-page .md files + index.json, the full page bodies are written BEFORE max_content_per_page truncation; dot-containing dir names are fine), include_content_in_response (bool, default False — when True keeps crawled_pages in the response too, still subject to max_content_per_page truncation), overwrite (bool, default False). Failed pages appear in index.json with file=null."New value: +"Dict with: search_query (required), crawl_top_results, search_genre (optional, one of: pdf, documents, presentations, spreadsheets, japanese, english; adds a filetype/language operator to the query, other values return an error), engine (optional, one of: auto, google_custom, brave, exa, searxng; default from SEARCH_ENGINE, auto tries the configured engines in SEARCH_ENGINE_ORDER and moves to the next one on errors other than zero results), result_type (optional; default web; one of web, discussions, faq, videos, news, or several as a list or a comma-separated string; everything except web runs on brave only; news is a separate Brave News Search request; results holds web when requested, else the first type, and the other types go to extra_results; discussions are forum posts with the forum name, answer count, post score, question and top comment; faq are question and answer excerpts; Brave returns these sections for some queries only, and num_results only shortens them), recent_days, generate_markdown, max_content_per_page. Optional persistence keys: output_path (absolute directory — per-page .md files + index.json, the full page bodies are written BEFORE max_content_per_page truncation; dot-containing dir names are fine), include_content_in_response (bool, default False — when True keeps crawled_pages in the response too, still subject to max_content_per_page truncation), overwrite (bool, default False). Failed pages appear in index.json with file=null."
  2. Addedv0.3.2
  3. Removedv0.1.6
  4. First observedv0.1.7

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond readOnlyHint=true by disclosing the filesystem side effect: per-page markdown plus index.json written to output_path, and that the response becomes 'slim' in that mode. It even notes persisted bodies are 'unsliced', i.e. written before max_content_per_page truncation, which is a non-obvious behavioral detail. Missing only permission/overwrite semantics and failure behavior, which the schema covers instead.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action and ending with the optional persistence mode, so an agent skimming gets the essential behavior first. The final sentence is dense but every clause carries a distinct fact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is moderately complex (nested request object, dual search/crawl behavior), the schema is fully documented, and an output schema exists, so return values need not be explained. The description covers the compound behavior and the two response modes, which is what an agent needs to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the nested request description already enumerates every key, including output_path, include_content_in_response, overwrite and failure handling. The description adds modest value by clarifying that output_path is a directory with a specific artifact layout and that persisted content is pre-truncation. That is useful but largely restates schema content, so baseline 3 fits.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific compound action: search the web and crawl the top results, with full content extraction. It anchors the engine parameter to the sibling search_web, so an agent can see this is search_web plus page fetching. It does not explicitly contrast itself with the pure crawl siblings (crawl_url, deep_crawl_site, batch_crawl), leaving the boundary to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Combines search with full content extraction' implies the condition for use: when snippets alone are insufficient and page bodies are needed. The third sentence describes the persistence mode. However, there is no explicit when-not or named alternative (search_web for listings only, batch_crawl for known URLs), so the routing decision is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.