Skip to main content
Glama
mysleekdesigns

CrawlForge MCP Server

by mysleekdesigns
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
JavaScript
Remote

extract_structured

Read-onlyIdempotent

Extract structured data from any webpage using a JSON schema when you know fields but not selectors. Get clean JSON for product details, job listings, and event data.

Instructions

Use this to get a specific data shape from a page using a JSON schema when you can describe the fields but not their selectors - product details, job listings, event data. Uses an LLM by default; falls back to CSS selectors when no LLM is configured. Not for pages with stable markup whose selectors you know (scrape_structured, no LLM, cheaper). Cost: 3 credits. Example: extract_structured({url: "https://jobs.example.com/post/123", schema: {properties: {title: {type:"string"}, salary: {type:"string"}}, required:["title"]}})

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to extract structured data from
promptNoNatural language instructions for extraction
schemaYesJSON schema defining the data structure to extract
llmConfigNoLLM provider configuration for AI-powered extraction
user_agentNoOverride the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.
selectorHintsNoCSS selector hints to guide extraction
respect_robotsNoRespect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.
verify_numbersNoNumeric provenance guard (default: true): every price or numeric value the LLM returns must appear literally in the page source, else it is returned as null with a reason in `provenance.unverified`. Set false to get the model's raw numbers back, including ones it derived (a count, a sum, a total) rather than read off the page.
fallbackToSelectorsNoFall back to CSS selector extraction if LLM is unavailable

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlNo
dataNoExtracted fields matching the requested schema
_costNoCost-transparency metadata (D3.5), present when injected into the text copy of the result
errorNo
successNoFalse when the extraction errored or a required field came back missing or empty
confidenceNo
provenanceNo
validationNo
schema_usedNo
processingTimeNo
extractionNotesNo
extraction_methodNo"llm" | "css_fallback" | "keyword_fallback" | "none"

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Changed11 schema fields changedv6.0.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • removedInput schema / properties / llmConfig / additionalProperties
      Removed value: -false
    • removedInput schema / properties / schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / schema / properties / properties / propertyNames
      Added value: +{
+  "type": "string"
+}
    • addedInput schema / properties / selectorHints / propertyNames
      Added value: +{
+  "type": "string"
+}
    • changedOutput schema / properties / _cost / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / data / propertyNames
      Added value: +{
+  "type": "string"
+}
    • changedOutput schema / properties / provenance / additionalProperties
      Previous value: -trueNew value: +{}
    • changedOutput schema / properties / provenance / properties / unverified / items / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / schema_used / propertyNames
      Added value: +{
+  "type": "string"
+}
    • changedOutput schema / properties / validation / additionalProperties
      Previous value: -trueNew value: +{}
  2. Changed5 schema fields changedv5.4.0
    • addedInput schema / properties / respect_robots
      Added value: +{
+  "description": "Respect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.",
+  "type": "boolean"
+}
    • addedInput schema / properties / user_agent
      Added value: +{
+  "description": "Override the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.",
+  "type": "string"
+}
    • addedInput schema / properties / verify_numbers
      Added value: +{
+  "default": true,
+  "description": "Numeric provenance guard (default: true): every price or numeric value the LLM returns must appear literally in the page source, else it is returned as null with a reason in `provenance.unverified`. Set false to get the model's raw numbers back, including ones it derived (a count, a sum, a total) rather than read off the page.",
+  "type": "boolean"
+}
    • addedOutput schema / properties / provenance
      Added value: +{
+  "additionalProperties": true,
+  "properties": {
+    "enabled": {
+      "description": "Whether the numeric provenance guard ran",
+      "type": "boolean"
+    },
+    "nulled": {
+      "description": "Numeric values replaced with null because the source does not contain them",
+      "type": "number"
+    },
+    "skipped": {
+      "description": "\"empty_source\" when there was nothing to check against",
+      "type": "string"
+    },
+    "unverified": {
+      "items": {
+        "additionalProperties": true,
+        "properties": {
+          "path": {
+            "description": "Path to the field, e.g. configurations[2].price",
+            "type": "string"
+          },
+          "reason": {
+            "description": "\"not_found_in_source\"",
+            "type": "string"
+          },
+          "value": {
+            "description": "The value that was removed"
+          }
+        },
+        "type": "object"
+      },
+      "type": "array"
+    },
+    "verified": {
+      "description": "Numeric values found literally in the page source",
+      "type": "number"
+    }
+  },
+  "type": "object"
+}
    • addedOutput schema / properties / success
      Added value: +{
+  "description": "False when the extraction errored or a required field came back missing or empty",
+  "type": "boolean"
+}
  3. Changed1 schema field changedv5.1.0
    • changedOutput schema / properties / extraction_method / description
      Previous value: -"\"llm\" | \"css_fallback\" | \"none\""New value: +"\"llm\" | \"css_fallback\" | \"keyword_fallback\" | \"none\""
  4. Changed2 schema fields changedv5.0.4
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "additionalProperties": false,
+  "properties": {
+    "_cost": {
+      "additionalProperties": true,
+      "description": "Cost-transparency metadata (D3.5), present when injected into the text copy of the result",
+      "properties": {
+        "actual": {
+          "description": "Credits actually charged (0 in creator mode, half-rate on error)",
+          "type": "number"
+        },
+        "projected": {
+          "description": "Credits projected for this call before execution",
+          "type": "number"
+        },
+        "projection_note": {
+          "description": "Human-readable note about how the cost was projected",
+          "type": "string"
+        },
+        "remaining_credits": {
+          "description": "Credits remaining on the account after this call, if known",
+          "type": [
+            "number",
+            "null"
+          ]
+        }
+      },
+      "type": "object"
+    },
+    "confidence": {
+      "type": "number"
+    },
+    "data": {
+      "additionalProperties": {},
+      "description": "Extracted fields matching the requested schema",
+      "type": "object"
+    },
+    "error": {
+      "type": "string"
+    },
+    "extractionNotes": {
+      "items": {
+        "type": "string"
+      },
+      "type": "array"
+    },
+    "extraction_method": {
+      "description": "\"llm\" | \"css_fallback\" | \"none\"",
+      "type": "string"
+    },
+    "processingTime": {
+      "type": "number"
+    },
+    "schema_used": {
+      "additionalProperties": {},
+      "type": "object"
+    },
+    "url": {
+      "type": "string"
+    },
+    "validation": {
+      "additionalProperties": true,
+      "properties": {
+        "errors": {
+          "items": {
+            "type": "string"
+          },
+          "type": "array"
+        },
+        "valid": {
+          "type": "boolean"
+        }
+      },
+      "type": "object"
+    }
+  },
+  "type": "object"
+}
  5. First observedv4.10.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint, idempotentHint, and destructiveHint. The description adds meaningful behavioral context beyond annotations by stating that it uses an LLM by default, falls back to CSS selectors when no LLM is configured, and costs 3 credits. It does not contradict the annotations. The description could also mention response/provenance behavior, but the parameter descriptions and output schema already cover some of that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured: it front-loads the core use case, gives examples, names the alternative, states the cost, and provides a minimal illustrative example. Every sentence earns its place, and nothing is redundant with the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity—9 parameters, nested objects, and an output schema—the description covers the essential decision context: when to use it, when not to, LLM fallback behavior, and cost. The parameter schema supplies the remaining details. A minor gap is that high-level behavioral knobs like verify_numbers and respect_robots are not mentioned in the description, but they are adequately explained in the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameter descriptions carry the baseline. The description adds a concrete example showing how to pass the schema object and required fields, which helps clarify the schema parameter. However, it adds no semantic detail for most other parameters like prompt, llmConfig, selectorHints, or user_agent, so it stays at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'get a specific data shape from a page using a JSON schema.' It clarifies the exact use case—describing fields but not their selectors—and gives concrete examples (product details, job listings, event data). It also explicitly distinguishes the tool from scrape_structured, removing any ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance: use when 'you can describe the fields but not their selectors.' It names the alternative tool and the condition to choose it instead: 'Not for pages with stable markup whose selectors you know (scrape_structured, no LLM, cheaper).' This gives both positive and negative selection criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mysleekdesigns/crawlforge-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server