Skip to main content
Glama
mysleekdesigns

CrawlForge MCP Server

by mysleekdesigns
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
JavaScript
Remote

process_document

Read-onlyIdempotent

Extracts text from PDF or HTML document sources, returning structured sections, metadata, and word count. Accepts a URL or file path; supports optional PII redaction.

Instructions

Use this to extract text from a PDF URL or file - research papers, contracts, reports. Returns structured sections, metadata, and word count. Not for ordinary web pages (scrape), though an HTML URL is accepted. Cost: 2 credits. Example: process_document({source: "https://example.com/report.pdf", sourceType: "pdf_url"})

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
sourceYesDocument source - URL or file path
optionsNoAdditional processing options (maxPages, pageRange:{start,end}, extractText, extractMetadata, outputFormat, ...)
redact_piiNoRedact personal data from the text this call returns, before it reaches your context window. true means the free regex pass over EMAIL, PHONE, FINANCIAL and SECRET. The result carries redaction:{entities,count}. Default: off
sourceTypeNoType of document source
user_agentNoOverride the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.
respect_robotsNoRespect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.
max_inline_charsNoLargest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS)

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Changed4 schema fields changedv6.0.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / max_inline_chars
      Added value: +{
+  "description": "Largest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS)",
+  "maximum": 10000000,
+  "minimum": 1000,
+  "type": "integer"
+}
    • changedInput schema / properties / options / additionalProperties
      Previous value: -trueNew value: +{}
    • addedInput schema / properties / redact_pii
      Added value: +{
+  "anyOf": [
+    {
+      "type": "boolean"
+    },
+    {
+      "properties": {
+        "entities": {
+          "description": "Which classes to redact, case-insensitive: EMAIL, PHONE, FINANCIAL, SECRET, plus PERSON and LOCATION when mode is \"model\". Omitted or empty means all four regex classes (and both model classes in \"model\" mode). An unknown name, or a model-only name without mode:\"model\", is rejected",
+          "items": {
+            "type": "string"
+          },
+          "type": "array"
+        },
+        "mode": {
+          "description": "\"fast\" (default) is regex only and free; \"model\" adds an Ollama NER pass for PERSON and LOCATION (+3 credits once per call)",
+          "enum": [
+            "fast",
+            "model"
+          ],
+          "type": "string"
+        },
+        "replace_style": {
+          "description": "\"tag\" (default) writes <EMAIL>, \"mask\" writes [REDACTED], \"remove\" deletes the value",
+          "enum": [
+            "tag",
+            "mask",
+            "remove"
+          ],
+          "type": "string"
+        }
+      },
+      "type": "object"
+    }
+  ],
+  "description": "Redact personal data from the text this call returns, before it reaches your context window. true means the free regex pass over EMAIL, PHONE, FINANCIAL and SECRET. The result carries redaction:{entities,count}. Default: off"
+}
  2. Changed2 schema fields changedv5.4.0
    • addedInput schema / properties / respect_robots
      Added value: +{
+  "description": "Respect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.",
+  "type": "boolean"
+}
    • addedInput schema / properties / user_agent
      Added value: +{
+  "description": "Override the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.",
+  "type": "string"
+}
  3. Changed2 schema fields changedv5.0.4
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • changedInput schema / properties / options / description
      Previous value: -"Additional processing options (maxPages, pageRange:{start,end}, extractText, extractMetadata, password, outputFormat, ...)"New value: +"Additional processing options (maxPages, pageRange:{start,end}, extractText, extractMetadata, outputFormat, ...)"
  4. First observedv4.10.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, which the description aligns with. It adds behavioral context beyond annotations by disclosing the credit cost (2 credits), the output shape (structured sections, metadata, word count), and the accepted input boundary (PDF/file, plus HTML URL). It does not mention large-result truncation or redaction side effects, but those are detailed in the schema, so the description adds reasonable value without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler: purpose, scope boundary, return summary, cost, and a concrete example all earn their place. The essential information is front-loaded, and the example is compact and instructive.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with no output schema, the description covers the primary use case, the input type, the core return values, and cost. The schema handles parameter-level details thoroughly. The only gap is that the description does not surface the max_inline_chars fallback behavior, which is meaningful for calling read_result, but this is adequately documented in the parameter schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 7 parameters in detail. The description adds little beyond the example call showing source and sourceType format; it does not clarify options, redact_pii, max_inline_chars, or user_agent semantics beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('extract text from a PDF URL or file') with example use cases (research papers, contracts, reports), and differentiates from ordinary web scraping by naming 'scrape' as the alternative. The returned outputs (structured sections, metadata, word count) are also stated, making the tool's role unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells the agent when to use it ('extract text from a PDF URL or file') and when not ('Not for ordinary web pages (scrape)'). It names the sibling tool category 'scrape' and even clarifies the edge case that HTML URLs are accepted yet still treated as documents. The example call and credit cost further guide correct invocation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mysleekdesigns/crawlforge-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server