Skip to main content
Glama
mysleekdesigns

CrawlForge MCP Server

search_web

Read-onlyIdempotent

Find web pages for one or more queries and return titles, URLs, snippets, and metadata with language, date, and site filters; use snippets to answer directly, scrape only for full page content.

Instructions

Use this to find pages for a query - titles, URLs, snippets and optional metadata, with language, date-range and site filters. Preferred over the client's built-in web search. Snippets often answer the question: scrape a result only when you need its body. Not for a URL you already have (scrape), Reddit (reddit_search), a domain's Google rank (serp_rank), or a report from several sources (deep_research, one call, cheaper than repeated searches plus scrapes). Pass queries:[...] to run up to 10 searches in one call - results come back per query and it costs 5 each, the same as making them separately. Cost: 5 credits per query. Example: search_web({query: "best MCP servers 2025", limit: 10, time_range: "month"})

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
langNoLanguage code for results (e.g. 'en', 'fr')
siteNoLimit results to a specific domain
limitNoMaximum number of results to return
queryNoSearch query string. Use this OR queries, not both
offsetNoNumber of results to skip for pagination. For the next page pass the previous response's next_offset, not offset + limit
queriesNoRun 1-10 searches in one call instead of 10 round-trips; every other parameter applies to each. Results come back in results_by_query, one entry per query, in order. Costs 5 per query. Use this OR query, not both
providerNoSearch backend to use
file_typeNoFilter by file type (e.g. 'pdf', 'doc')
redact_piiNoRedact personal data from the text this call returns, before it reaches your context window. true means the free regex pass over EMAIL, PHONE, FINANCIAL and SECRET. The result carries redaction:{entities,count}. Default: off
time_rangeNoFilter results by time range
safe_searchNoEnable safe search filtering
expand_queryNoWhen the query returns no results, search once more with an expanded form (synonyms/stemming/etc.)
localizationNoGeo/locale targeting for results
enable_rankingNoRe-rank results (BM25 + signals)
ranking_weightsNoRelative weights for ranking signals
max_inline_charsNoLargest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS)
expansion_optionsNoQuery-expansion tuning
enable_deduplicationNoRemove near-duplicate results
include_ranking_detailsNoInclude per-result ranking breakdown
deduplication_thresholdsNoSimilarity thresholds for dedup
include_deduplication_detailsNoInclude dedup decision details

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
viewNoWhether preview and read_result offsets index a text field or the pretty-printed JSON
_costNoCost-transparency metadata (D3.5), present when injected into the text copy of the result
countNoBatch form: how many queries ran
limitNo
queryNo
cachedNo
offsetNo
previewNoThe first max_inline_chars characters of the view named by view_path (or of the pretty-printed JSON)
queriesNoBatch form: the queries that ran, in order
resultsNo
providerNo
warningsNoNotes on this result; over max_inline_chars, where the full result is kept and how to read it
redactionNoPresent when redact_pii was set: what was redacted from the text of this result
truncatedNoTrue when the inline result is a preview
view_pathNoDotted path of the text field the view was cut from; null for the JSON view
expires_atNoWhen the stored result is dropped (ISO 8601)
processingNo
next_offsetNoThe offset to pass for the next page. Duplicates removed from this page are replaced from further down the provider's results, so it can be larger than offset + limit
search_timeNo
total_charsNoLength of the full view in characters
localizationNo
result_handleNoHandle for read_result; the full result is kept 1 hour
total_resultsNo
effective_queryNoPresent when query expansion changed the query actually used
expanded_queriesNoPresent only when the original query returned nothing: the queries searched, in order - the original, then its expanded form
results_by_queryNoBatch form: one entry per query, in order

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed15 schema fields changedv6.19.2
    • changedInput schema / properties / expand_query / description
      Previous value: -"Expand the query with synonyms/stemming/etc."New value: +"When the query returns no results, search once more with an expanded form (synonyms/stemming/etc.)"
    • addedInput schema / properties / max_inline_chars
      Added value: +{
      +  "description": "Largest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS)",
      +  "maximum": 10000000,
      +  "minimum": 1000,
      +  "type": "integer"
      +}
    • changedInput schema / properties / offset / description
      Previous value: -"Number of results to skip for pagination"New value: +"Number of results to skip for pagination. For the next page pass the previous response's next_offset, not offset + limit"
    • addedOutput schema / properties / expanded_queries / description
      Added value: +"Present only when the original query returned nothing: the queries searched, in order - the original, then its expanded form"
    • addedOutput schema / properties / expires_at
      Added value: +{
      +  "description": "When the stored result is dropped (ISO 8601)",
      +  "type": "string"
      +}
    • addedOutput schema / properties / next_offset
      Added value: +{
      +  "description": "The offset to pass for the next page. Duplicates removed from this page are replaced from further down the provider's results, so it can be larger than offset + limit",
      +  "type": "number"
      +}
    • addedOutput schema / properties / preview
      Added value: +{
      +  "description": "The first max_inline_chars characters of the view named by view_path (or of the pretty-printed JSON)",
      +  "type": "string"
      +}
    • addedOutput schema / properties / processing / properties / query_expansion / description
      Added value: +"{original_query, used_query, search_attempts} when the original query returned nothing and its expanded form was searched too; null otherwise"
    • removedOutput schema / properties / provider / properties / capabilities
      Removed value: -{
      -  "additionalProperties": {},
      -  "propertyNames": {
      -    "type": "string"
      -  },
      -  "type": "object"
      -}
    • addedOutput schema / properties / result_handle
      Added value: +{
      +  "description": "Handle for read_result; the full result is kept 1 hour",
      +  "type": "string"
      +}
    • addedOutput schema / properties / total_chars
      Added value: +{
      +  "description": "Length of the full view in characters",
      +  "type": "number"
      +}
    • addedOutput schema / properties / truncated
      Added value: +{
      +  "description": "True when the inline result is a preview",
      +  "type": "boolean"
      +}
    • addedOutput schema / properties / view
      Added value: +{
      +  "description": "Whether preview and read_result offsets index a text field or the pretty-printed JSON",
      +  "enum": [
      +    "text",
      +    "json"
      +  ],
      +  "type": "string"
      +}
    • addedOutput schema / properties / view_path
      Added value: +{
      +  "description": "Dotted path of the text field the view was cut from; null for the JSON view",
      +  "type": [
      +    "string",
      +    "null"
      +  ]
      +}
    • addedOutput schema / properties / warnings
      Added value: +{
      +  "description": "Notes on this result; over max_inline_chars, where the full result is kept and how to read it",
      +  "items": {
      +    "type": "string"
      +  },
      +  "type": "array"
      +}
  2. Changed25 schema fields changedv6.0.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • removedInput schema / properties / deduplication_thresholds / additionalProperties
      Removed value: -false
    • removedInput schema / properties / expansion_options / additionalProperties
      Removed value: -false
    • removedInput schema / properties / localization / additionalProperties
      Removed value: -false
    • removedInput schema / properties / localization / properties / customLocation / additionalProperties
      Removed value: -false
    • addedInput schema / properties / queries
      Added value: +{
      +  "description": "Run 1-10 searches in one call instead of 10 round-trips; every other parameter applies to each. Results come back in results_by_query, one entry per query, in order. Costs 5 per query. Use this OR query, not both",
      +  "items": {
      +    "minLength": 1,
      +    "type": "string"
      +  },
      +  "maxItems": 10,
      +  "minItems": 1,
      +  "type": "array"
      +}
    • changedInput schema / properties / query / description
      Previous value: -"Search query string"New value: +"Search query string. Use this OR queries, not both"
    • removedInput schema / properties / ranking_weights / additionalProperties
      Removed value: -false
    • addedInput schema / properties / redact_pii
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "boolean"
      +    },
      +    {
      +      "properties": {
      +        "entities": {
      +          "description": "Which classes to redact, case-insensitive: EMAIL, PHONE, FINANCIAL, SECRET, plus PERSON and LOCATION when mode is \"model\". Omitted or empty means all four regex classes (and both model classes in \"model\" mode). An unknown name, or a model-only name without mode:\"model\", is rejected",
      +          "items": {
      +            "type": "string"
      +          },
      +          "type": "array"
      +        },
      +        "mode": {
      +          "description": "\"fast\" (default) is regex only and free; \"model\" adds an Ollama NER pass for PERSON and LOCATION (+3 credits once per call)",
      +          "enum": [
      +            "fast",
      +            "model"
      +          ],
      +          "type": "string"
      +        },
      +        "replace_style": {
      +          "description": "\"tag\" (default) writes <EMAIL>, \"mask\" writes [REDACTED], \"remove\" deletes the value",
      +          "enum": [
      +            "tag",
      +            "mask",
      +            "remove"
      +          ],
      +          "type": "string"
      +        }
      +      },
      +      "type": "object"
      +    }
      +  ],
      +  "description": "Redact personal data from the text this call returns, before it reaches your context window. true means the free regex pass over EMAIL, PHONE, FINANCIAL and SECRET. The result carries redaction:{entities,count}. Default: off"
      +}
    • removedInput schema / required
      Removed value: -[
      -  "query"
      -]
    • changedOutput schema / properties / _cost / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / count
      Added value: +{
      +  "description": "Batch form: how many queries ran",
      +  "type": "number"
      +}
    • changedOutput schema / properties / localization / anyOf
      Previous value: -[
      -  {
      -    "additionalProperties": true,
      -    "properties": {
      -      "applied": {
      -        "type": "boolean"
      -      },
      -      "countryCode": {
      -        "type": "string"
      -      },
      -      "geoTargeting": {
      -        "type": "boolean"
      -      },
      -      "language": {
      -        "type": "string"
      -      },
      -      "searchDomain": {
      -        "type": "string"
      -      }
      -    },
      -    "type": "object"
      -  },
      -  {
      -    "type": "null"
      -  }
      -]New value: +[
      +  {
      +    "additionalProperties": {},
      +    "properties": {
      +      "applied": {
      +        "type": "boolean"
      +      },
      +      "countryCode": {
      +        "type": "string"
      +      },
      +      "geoTargeting": {
      +        "type": "boolean"
      +      },
      +      "language": {
      +        "type": "string"
      +      },
      +      "searchDomain": {
      +        "type": "string"
      +      }
      +    },
      +    "type": "object"
      +  },
      +  {
      +    "type": "null"
      +  }
      +]
    • changedOutput schema / properties / processing / additionalProperties
      Previous value: -trueNew value: +{}
    • changedOutput schema / properties / processing / properties / deduplication / anyOf
      Previous value: -[
      -  {
      -    "additionalProperties": {},
      -    "type": "object"
      -  },
      -  {
      -    "type": "null"
      -  }
      -]New value: +[
      +  {
      +    "additionalProperties": {},
      +    "propertyNames": {
      +      "type": "string"
      +    },
      +    "type": "object"
      +  },
      +  {
      +    "type": "null"
      +  }
      +]
    • changedOutput schema / properties / processing / properties / query_expansion / anyOf
      Previous value: -[
      -  {
      -    "additionalProperties": {},
      -    "type": "object"
      -  },
      -  {
      -    "type": "null"
      -  }
      -]New value: +[
      +  {
      +    "additionalProperties": {},
      +    "propertyNames": {
      +      "type": "string"
      +    },
      +    "type": "object"
      +  },
      +  {
      +    "type": "null"
      +  }
      +]
    • changedOutput schema / properties / processing / properties / ranking / anyOf
      Previous value: -[
      -  {
      -    "additionalProperties": {},
      -    "type": "object"
      -  },
      -  {
      -    "type": "null"
      -  }
      -]New value: +[
      +  {
      +    "additionalProperties": {},
      +    "propertyNames": {
      +      "type": "string"
      +    },
      +    "type": "object"
      +  },
      +  {
      +    "type": "null"
      +  }
      +]
    • changedOutput schema / properties / provider / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / provider / properties / capabilities / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • addedOutput schema / properties / queries
      Added value: +{
      +  "description": "Batch form: the queries that ran, in order",
      +  "items": {
      +    "type": "string"
      +  },
      +  "type": "array"
      +}
    • addedOutput schema / properties / redaction
      Added value: +{
      +  "additionalProperties": {},
      +  "description": "Present when redact_pii was set: what was redacted from the text of this result",
      +  "properties": {
      +    "count": {
      +      "description": "Total spans replaced",
      +      "type": "number"
      +    },
      +    "entities": {
      +      "additionalProperties": {
      +        "type": "number"
      +      },
      +      "description": "How many spans were replaced, by entity class; a class with no hits is omitted",
      +      "propertyNames": {
      +        "type": "string"
      +      },
      +      "type": "object"
      +    },
      +    "mode": {
      +      "description": "\"fast\" is the free regex pass; \"model\" added an Ollama NER pass for PERSON and LOCATION",
      +      "enum": [
      +        "fast",
      +        "model"
      +      ],
      +      "type": "string"
      +    },
      +    "model_ran": {
      +      "description": "mode \"model\" only: whether a model actually answered. False means no LLM route existed and the model surcharge was not charged",
      +      "type": "boolean"
      +    }
      +  },
      +  "type": "object"
      +}
    • changedOutput schema / properties / results / items / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / results / items / properties / metadata / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • addedOutput schema / properties / results / items / properties / pagemap / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • addedOutput schema / properties / results_by_query
      Added value: +{
      +  "description": "Batch form: one entry per query, in order",
      +  "items": {
      +    "additionalProperties": {},
      +    "properties": {
      +      "error": {
      +        "description": "Present when this query failed; the other queries in the batch are unaffected",
      +        "type": "string"
      +      },
      +      "query": {
      +        "type": "string"
      +      },
      +      "results": {
      +        "items": {
      +          "additionalProperties": {},
      +          "properties": {
      +            "displayLink": {
      +              "type": "string"
      +            },
      +            "formattedUrl": {
      +              "type": "string"
      +            },
      +            "htmlSnippet": {
      +              "type": "string"
      +            },
      +            "link": {
      +              "type": "string"
      +            },
      +            "metadata": {
      +              "additionalProperties": {},
      +              "propertyNames": {
      +                "type": "string"
      +              },
      +              "type": "object"
      +            },
      +            "pagemap": {
      +              "additionalProperties": {},
      +              "propertyNames": {
      +                "type": "string"
      +              },
      +              "type": "object"
      +            },
      +            "snippet": {
      +              "type": "string"
      +            },
      +            "title": {
      +              "type": "string"
      +            }
      +          },
      +          "type": "object"
      +        },
      +        "type": "array"
      +      }
      +    },
      +    "type": "object"
      +  },
      +  "type": "array"
      +}
  3. Changed2 schema fields changedv5.0.4
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "$schema": "https://json-schema.org/draft/2020-12/schema",
      +  "additionalProperties": false,
      +  "properties": {
      +    "_cost": {
      +      "additionalProperties": true,
      +      "description": "Cost-transparency metadata (D3.5), present when injected into the text copy of the result",
      +      "properties": {
      +        "actual": {
      +          "description": "Credits actually charged (0 in creator mode, half-rate on error)",
      +          "type": "number"
      +        },
      +        "projected": {
      +          "description": "Credits projected for this call before execution",
      +          "type": "number"
      +        },
      +        "projection_note": {
      +          "description": "Human-readable note about how the cost was projected",
      +          "type": "string"
      +        },
      +        "remaining_credits": {
      +          "description": "Credits remaining on the account after this call, if known",
      +          "type": [
      +            "number",
      +            "null"
      +          ]
      +        }
      +      },
      +      "type": "object"
      +    },
      +    "cached": {
      +      "type": "boolean"
      +    },
      +    "effective_query": {
      +      "description": "Present when query expansion changed the query actually used",
      +      "type": "string"
      +    },
      +    "expanded_queries": {
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": "array"
      +    },
      +    "limit": {
      +      "type": "number"
      +    },
      +    "localization": {
      +      "anyOf": [
      +        {
      +          "additionalProperties": true,
      +          "properties": {
      +            "applied": {
      +              "type": "boolean"
      +            },
      +            "countryCode": {
      +              "type": "string"
      +            },
      +            "geoTargeting": {
      +              "type": "boolean"
      +            },
      +            "language": {
      +              "type": "string"
      +            },
      +            "searchDomain": {
      +              "type": "string"
      +            }
      +          },
      +          "type": "object"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ]
      +    },
      +    "offset": {
      +      "type": "number"
      +    },
      +    "processing": {
      +      "additionalProperties": true,
      +      "properties": {
      +        "deduplication": {
      +          "anyOf": [
      +            {
      +              "additionalProperties": {},
      +              "type": "object"
      +            },
      +            {
      +              "type": "null"
      +            }
      +          ]
      +        },
      +        "localization_applied": {
      +          "type": "boolean"
      +        },
      +        "query_expansion": {
      +          "anyOf": [
      +            {
      +              "additionalProperties": {},
      +              "type": "object"
      +            },
      +            {
      +              "type": "null"
      +            }
      +          ]
      +        },
      +        "ranking": {
      +          "anyOf": [
      +            {
      +              "additionalProperties": {},
      +              "type": "object"
      +            },
      +            {
      +              "type": "null"
      +            }
      +          ]
      +        }
      +      },
      +      "type": "object"
      +    },
      +    "provider": {
      +      "additionalProperties": true,
      +      "properties": {
      +        "backend": {
      +          "type": "string"
      +        },
      +        "capabilities": {
      +          "additionalProperties": {},
      +          "type": "object"
      +        },
      +        "instanceUrl": {
      +          "type": [
      +            "string",
      +            "null"
      +          ]
      +        },
      +        "name": {
      +          "type": "string"
      +        },
      +        "note": {
      +          "type": "string"
      +        }
      +      },
      +      "type": "object"
      +    },
      +    "query": {
      +      "type": "string"
      +    },
      +    "results": {
      +      "items": {
      +        "additionalProperties": true,
      +        "properties": {
      +          "displayLink": {
      +            "type": "string"
      +          },
      +          "formattedUrl": {
      +            "type": "string"
      +          },
      +          "htmlSnippet": {
      +            "type": "string"
      +          },
      +          "link": {
      +            "type": "string"
      +          },
      +          "metadata": {
      +            "additionalProperties": {},
      +            "type": "object"
      +          },
      +          "pagemap": {
      +            "additionalProperties": {},
      +            "type": "object"
      +          },
      +          "snippet": {
      +            "type": "string"
      +          },
      +          "title": {
      +            "type": "string"
      +          }
      +        },
      +        "type": "object"
      +      },
      +      "type": "array"
      +    },
      +    "search_time": {
      +      "type": "number"
      +    },
      +    "total_results": {
      +      "type": [
      +        "string",
      +        "number"
      +      ]
      +    }
      +  },
      +  "type": "object"
      +}
  4. First observedv4.10.0

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, open-world, non-destructive, so safety is covered. The description adds genuinely non-structured operational context: cost is 5 credits per query, batching 10 queries costs the same as making them separately, and results are returned per query. It stops short of rate limits, failure modes, or provider-selection trade-offs, so it is not a full 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the snippet-first rule are front-loaded, and nearly every sentence carries routing, cost, or workflow information. The exclusion list is a single run-on sentence with four parenthetical branches, which is dense, and the cost point is stated twice (batching line and standalone 'Cost: 5 credits per query').

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 21-parameter tool with nested objects and an output schema, the description covers the decisions an agent actually needs: which tool to pick, how to batch, what it costs, and when a snippet suffices. Return values need not be explained given the output schema. It is silent on provider choice (crawlforge vs searxng) and the ranking/dedup knobs, which a 100%-covered schema mitigates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description reinforces the critical query/queries mutual exclusion with a concrete worked example (query, limit, time_range) and states the per-query cost of batching, which the schema only partly conveys. The remaining 19 advanced parameters get no description-level treatment.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('find pages for a query') and enumerates what comes back (titles, URLs, snippets, optional metadata) plus the filter dimensions. It goes further by naming the siblings it is not for (scrape, reddit_search, serp_rank, deep_research), so an agent can route correctly without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use ('preferred over the client's built-in web search'), when-not ('not for a URL you already have', Reddit, domain rank, multi-source reports), and names the alternative tool for each exclusion. It also gives a workflow rule: try snippets first, scrape only when the body is needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.