Skip to main content
Glama
mysleekdesigns

CrawlForge MCP Server

by mysleekdesigns
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

extract_structured

Read-onlyIdempotent

Extract structured data from a page by defining a JSON schema when you know the fields but not the CSS selectors. Perfect for product details, job listings, or event data.

Instructions

Use this to get a specific data shape from a page using a JSON schema when you can describe the fields but not their selectors - product details, job listings, event data. Uses an LLM by default; falls back to CSS selectors when no LLM is configured. Not for pages with stable markup whose selectors you know (scrape_structured, no LLM, cheaper). Cost: 3 credits. Example: extract_structured({url: "https://jobs.example.com/post/123", schema: {properties: {title: {type:"string"}, salary: {type:"string"}}, required:["title"]}})

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to extract structured data from
promptNoNatural language instructions for extraction
schemaYesJSON schema defining the data structure to extract
llmConfigNoLLM provider configuration for AI-powered extraction
user_agentNoOverride the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.
selectorHintsNoCSS selector hints to guide extraction
respect_robotsNoRespect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.
verify_numbersNoNumeric provenance guard (default: true): every price or numeric value the LLM returns must appear literally in the page source, else it is returned as null with a reason in `provenance.unverified`. Set false to get the model's raw numbers back, including ones it derived (a count, a sum, a total) rather than read off the page.
fallbackToSelectorsNoFall back to CSS selector extraction if LLM is unavailable

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlNo
dataNoExtracted fields matching the requested schema
_costNoCost-transparency metadata (D3.5), present when injected into the text copy of the result
errorNo
modelNoModel that produced the data, e.g. "gemma3:12b"; present when extraction_method is "llm"
successNoFalse when the extraction errored, or a required field came back missing, empty, or in the wrong shape
providerNoLLM provider that produced the data ("ollama" | "openai" | "anthropic"); present when extraction_method is "llm"
confidenceNo0-1: method and validation, scaled by the share of requested fields filled and, for "llm", the share of short string values found in the page
provenanceNo
validationNo
schema_usedNo
processingTimeNo
extractionNotesNo
extraction_methodNo"llm" | "css_fallback" | "keyword_fallback" | "none"

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed5 schema fields changedv6.19.2
    • addedOutput schema / properties / confidence / description
      Added value: +"0-1: method and validation, scaled by the share of requested fields filled and, for \"llm\", the share of short string values found in the page"
    • addedOutput schema / properties / model
      Added value: +{
+  "description": "Model that produced the data, e.g. \"gemma3:12b\"; present when extraction_method is \"llm\"",
+  "type": "string"
+}
    • changedOutput schema / properties / provenance / properties / nulled / description
      Previous value: -"Numeric values replaced with null because the source does not contain them"New value: +"Values replaced with null: numbers the source does not contain, and string fields whose value cannot be what the field names"
    • changedOutput schema / properties / provenance / properties / unverified / items / properties / reason / description
      Previous value: -"\"not_found_in_source\""New value: +"\"not_found_in_source\" | \"not_a_version\" (a version field with no number, e.g. \"latest\") | \"not_an_identifier\" (a sku/isbn/gtin/upc/ean/mpn field holding a phrase)"
    • addedOutput schema / properties / provider
      Added value: +{
+  "description": "LLM provider that produced the data (\"ollama\" | \"openai\" | \"anthropic\"); present when extraction_method is \"llm\"",
+  "type": "string"
+}
  2. Changed1 schema field changedv6.5.0
    • changedOutput schema / properties / success / description
      Previous value: -"False when the extraction errored or a required field came back missing or empty"New value: +"False when the extraction errored, or a required field came back missing, empty, or in the wrong shape"
  3. Changed11 schema fields changedv6.0.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • removedInput schema / properties / llmConfig / additionalProperties
      Removed value: -false
    • removedInput schema / properties / schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / schema / properties / properties / propertyNames
      Added value: +{
+  "type": "string"
+}
    • addedInput schema / properties / selectorHints / propertyNames
      Added value: +{
+  "type": "string"
+}
    • changedOutput schema / properties / _cost / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / data / propertyNames
      Added value: +{
+  "type": "string"
+}
    • changedOutput schema / properties / provenance / additionalProperties
      Previous value: -trueNew value: +{}
    • changedOutput schema / properties / provenance / properties / unverified / items / additionalProperties
      Previous value: -trueNew value: +{}
    • addedOutput schema / properties / schema_used / propertyNames
      Added value: +{
+  "type": "string"
+}
    • changedOutput schema / properties / validation / additionalProperties
      Previous value: -trueNew value: +{}
  4. Changed5 schema fields changedv5.4.0
    • addedInput schema / properties / respect_robots
      Added value: +{
+  "description": "Respect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.",
+  "type": "boolean"
+}
    • addedInput schema / properties / user_agent
      Added value: +{
+  "description": "Override the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.",
+  "type": "string"
+}
    • addedInput schema / properties / verify_numbers
      Added value: +{
+  "default": true,
+  "description": "Numeric provenance guard (default: true): every price or numeric value the LLM returns must appear literally in the page source, else it is returned as null with a reason in `provenance.unverified`. Set false to get the model's raw numbers back, including ones it derived (a count, a sum, a total) rather than read off the page.",
+  "type": "boolean"
+}
    • addedOutput schema / properties / provenance
      Added value: +{
+  "additionalProperties": true,
+  "properties": {
+    "enabled": {
+      "description": "Whether the numeric provenance guard ran",
+      "type": "boolean"
+    },
+    "nulled": {
+      "description": "Numeric values replaced with null because the source does not contain them",
+      "type": "number"
+    },
+    "skipped": {
+      "description": "\"empty_source\" when there was nothing to check against",
+      "type": "string"
+    },
+    "unverified": {
+      "items": {
+        "additionalProperties": true,
+        "properties": {
+          "path": {
+            "description": "Path to the field, e.g. configurations[2].price",
+            "type": "string"
+          },
+          "reason": {
+            "description": "\"not_found_in_source\"",
+            "type": "string"
+          },
+          "value": {
+            "description": "The value that was removed"
+          }
+        },
+        "type": "object"
+      },
+      "type": "array"
+    },
+    "verified": {
+      "description": "Numeric values found literally in the page source",
+      "type": "number"
+    }
+  },
+  "type": "object"
+}
    • addedOutput schema / properties / success
      Added value: +{
+  "description": "False when the extraction errored or a required field came back missing or empty",
+  "type": "boolean"
+}
  5. Changed1 schema field changedv5.1.0
    • changedOutput schema / properties / extraction_method / description
      Previous value: -"\"llm\" | \"css_fallback\" | \"none\""New value: +"\"llm\" | \"css_fallback\" | \"keyword_fallback\" | \"none\""
  6. Changed2 schema fields changedv5.0.4
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "additionalProperties": false,
+  "properties": {
+    "_cost": {
+      "additionalProperties": true,
+      "description": "Cost-transparency metadata (D3.5), present when injected into the text copy of the result",
+      "properties": {
+        "actual": {
+          "description": "Credits actually charged (0 in creator mode, half-rate on error)",
+          "type": "number"
+        },
+        "projected": {
+          "description": "Credits projected for this call before execution",
+          "type": "number"
+        },
+        "projection_note": {
+          "description": "Human-readable note about how the cost was projected",
+          "type": "string"
+        },
+        "remaining_credits": {
+          "description": "Credits remaining on the account after this call, if known",
+          "type": [
+            "number",
+            "null"
+          ]
+        }
+      },
+      "type": "object"
+    },
+    "confidence": {
+      "type": "number"
+    },
+    "data": {
+      "additionalProperties": {},
+      "description": "Extracted fields matching the requested schema",
+      "type": "object"
+    },
+    "error": {
+      "type": "string"
+    },
+    "extractionNotes": {
+      "items": {
+        "type": "string"
+      },
+      "type": "array"
+    },
+    "extraction_method": {
+      "description": "\"llm\" | \"css_fallback\" | \"none\"",
+      "type": "string"
+    },
+    "processingTime": {
+      "type": "number"
+    },
+    "schema_used": {
+      "additionalProperties": {},
+      "type": "object"
+    },
+    "url": {
+      "type": "string"
+    },
+    "validation": {
+      "additionalProperties": true,
+      "properties": {
+        "errors": {
+          "items": {
+            "type": "string"
+          },
+          "type": "array"
+        },
+        "valid": {
+          "type": "boolean"
+        }
+      },
+      "type": "object"
+    }
+  },
+  "type": "object"
+}
  7. First observedv4.10.0

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, open-world, non-destructive semantics, so the bar is lower. The description still adds genuinely non-structured context: LLM-by-default with CSS selector fallback and a 3-credit cost. It leaves the verify_numbers/respect_robots behaviors to the schema, which is acceptable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then routing, then cost, then a concrete example. The packed single paragraph is dense but every clause earns its place; only the example pushes it toward the long side.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values needn't be explained, and cost, fallback behavior and the key alternative are covered. Minor gaps remain on prompt vs selectorHints interaction, but nothing that would cause a mis-call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema descriptions are rich (respect_robots, verify_numbers, user_agent all explain themselves), so the schema does the heavy lifting. The inline example illustrates url+schema usage but adds no syntax beyond what the schema already documents. Baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (extract a specific data shape from a page via JSON schema) and adds the precise condition that makes it distinct: you can describe fields but not their selectors. The sibling contrast with scrape_structured makes the boundary unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives both sides explicitly: use when you know the fields but not the selectors; do not use for stable markup where selectors are known, naming scrape_structured as the cheaper no-LLM alternative. Cost is stated too, which supports routing decisions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools