Skip to main content
Glama
mysleekdesigns

CrawlForge MCP Server

by mysleekdesigns
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

extract_metadata

Read-onlyIdempotent

Extract a page's SEO metadata—title, meta description, Open Graph tags, canonical URL, and schema.org JSON-LD—from a given URL for search and social previews.

Instructions

Use this for a page's SEO metadata only: title, meta description, Open Graph tags, canonical URL, schema.org data. url is the final URL; redirected:true (with requested_url) says a redirect moved the request to another page. Not alongside a scrape of the same URL: scrape formats:["markdown","metadata"] returns both in one fetch. Cost: 1 credit. Example: extract_metadata({url: "https://example.com"})

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to extract metadata from
user_agentNoOverride the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.
json_ld_typesNoFilter the returned JSON-LD to nodes of these schema.org types, e.g. ["Product","Offer"]. Subtypes match their parent: "Event" returns MusicEvent, "Offer" returns AggregateOffer, "ItemList" returns BreadcrumbList. Nodes are found at any depth, including inside @graph and nested inside a parent node; a match nested inside another returned node comes back inside it, not again on its own. When set, json_ld carries only the matching nodes instead of the raw dump, and json_ld_type_counts reports how many matched per requested type, nested ones included. Documented types: ItemList, Product, Offer, Event, JobPosting, RealEstateListing — any other schema.org type is matched exactly.
respect_robotsNoRespect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.
max_inline_charsNoLargest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv6.19.2
    • changedInput schema / properties / json_ld_types / description
      Previous value: -"Filter the returned JSON-LD to nodes of these schema.org types, e.g. [\"Product\",\"Offer\"]. Subtypes match their parent: \"Event\" returns MusicEvent, \"Offer\" returns AggregateOffer, \"ItemList\" returns BreadcrumbList. Nodes are found at any depth, including inside @graph and nested inside a parent node. When set, json_ld carries only the matching nodes instead of the raw dump, and json_ld_type_counts reports how many matched per requested type. Documented types: ItemList, Product, Offer, Event, JobPosting, RealEstateListing — any other schema.org type is matched exactly."New value: +"Filter the returned JSON-LD to nodes of these schema.org types, e.g. [\"Product\",\"Offer\"]. Subtypes match their parent: \"Event\" returns MusicEvent, \"Offer\" returns AggregateOffer, \"ItemList\" returns BreadcrumbList. Nodes are found at any depth, including inside @graph and nested inside a parent node; a match nested inside another returned node comes back inside it, not again on its own. When set, json_ld carries only the matching nodes instead of the raw dump, and json_ld_type_counts reports how many matched per requested type, nested ones included. Documented types: ItemList, Product, Offer, Event, JobPosting, RealEstateListing — any other schema.org type is matched exactly."
    • addedInput schema / properties / max_inline_chars
      Added value: +{
+  "description": "Largest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS)",
+  "maximum": 10000000,
+  "minimum": 1000,
+  "type": "integer"
+}
  2. Changed1 schema field changedv6.0.0
    • removedInput schema / additionalProperties
      Removed value: -false
  3. Changed3 schema fields changedv5.4.0
    • addedInput schema / properties / json_ld_types
      Added value: +{
+  "description": "Filter the returned JSON-LD to nodes of these schema.org types, e.g. [\"Product\",\"Offer\"]. Subtypes match their parent: \"Event\" returns MusicEvent, \"Offer\" returns AggregateOffer, \"ItemList\" returns BreadcrumbList. Nodes are found at any depth, including inside @graph and nested inside a parent node. When set, json_ld carries only the matching nodes instead of the raw dump, and json_ld_type_counts reports how many matched per requested type. Documented types: ItemList, Product, Offer, Event, JobPosting, RealEstateListing — any other schema.org type is matched exactly.",
+  "items": {
+    "type": "string"
+  },
+  "type": "array"
+}
    • addedInput schema / properties / respect_robots
      Added value: +{
+  "description": "Respect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default.",
+  "type": "boolean"
+}
    • addedInput schema / properties / user_agent
      Added value: +{
+  "description": "Override the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with.",
+  "type": "string"
+}
  4. Changed1 schema field changedv5.0.4
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
  5. First observedv4.10.0

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry readOnlyHint/openWorldHint/idempotentHint/destructiveHint=false, so safety is covered. The description still adds non-obvious behavior: the redirected:true + requested_url signal and the 1-credit cost. It does not cover failure modes or throttling, but the added context is genuinely beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then scope, then the sibling exclusion, then cost and a call example. Every sentence carries a distinct piece of information with no filler; the example is short and directly usable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by naming the returned fields and the redirect signal. Combined with the schema's own documentation of truncation (max_inline_chars/result_handle) and robots behavior, an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning for the required url parameter that the schema does not: 'url is the final URL' and how a redirect surfaces via redirected/requested_url. It says nothing extra about user_agent, json_ld_types, respect_robots, or max_inline_chars, which the schema already documents at length.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('extract a page's SEO metadata') and enumerates exactly what comes back: title, meta description, Open Graph tags, canonical URL, schema.org data. It also names the boundary against the sibling scrape path, so an agent can separate it from extract_structured, extract_text, and scrape without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit scope ('SEO metadata only') plus an explicit when-not: 'Not alongside a scrape of the same URL: scrape formats:["markdown","metadata"] returns both in one fetch.' This tells the agent both when to use this tool and when to prefer the alternative in a single fetch.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools