Skip to main content
Glama
Sign In

URLpipe

by dev.urlpipe
OverviewDiscussionsAdmin

Extract page metadata

extract_metadata
Idempotent

Exposes POST /meta. Reads everything a page declares about itself — Open Graph, Twitter cards, meta tags, JSON-LD, microdata and links — into one object with the same keys on every page: title, description, site name, kind of page, language, authors, publication and update dates, share image and video, favicon and icons, logo, feeds, keywords, canonical URL, robots directives, hreflang alternates, oEmbed endpoint and structured-data types, plus the raw og: and twitter: tags. A value the page does not declare is null or an empty list. Values injected by JavaScript are included, because the page is rendered first. 1 credit.

Input Schema

NameRequiredDescriptionDefault
urlYesThe page to fetch. Must be a public http(s) URL.
syncNoWait for the result and return it (default false). When false the call returns a token immediately and you collect the result with get_result.
labelsNoYour own keys to find this request by later, e.g. {"client": "acme"}: they come back with the result, in the webhook and in list_requests, which can filter by them. Up to 16 keys of up to 40 letters, digits, _ - or .; string values up to 256 characters.
max_ageNoHow fresh a stored result must be to be reused, e.g. "2 hours" or "3 days". Default 7 days, maximum 30. A reused result is free. Pass "0" to force a fresh fetch.
report_toNoAsync only: a webhook URL to deliver the result to. Defaults to the project's configured endpoint, if it has one.
project_idYesWhich project this request belongs to. From list_projects.
residentialNoFetch the page from a residential (home ISP) exit instead of a datacentre one. Costs a surcharge per page visit — see get_usage. Use it for sites that block datacentre traffic.
auth_profileNoLoad the page signed in, as this auth profile's visitor: its id, ap_…. A profile is a browser's signed-in state uploaded once over the HTTP API (POST /auth_profiles); only URLs on its domains are accepted. Costs nothing extra. https://urlpipe.dev/docs/auth-profiles
page_optionsNoWhat to do to the page before anything is read off it. Applies to the result itself: removed ads and banners are gone from html, markdown and summaries too, and what scrolling loaded is in all of them.
idempotency_keyNoMakes the call safe to retry: sending it again with the same key within 24 hours returns the first call's token and result instead of starting (and charging for) new work, even with max_age "0". Up to 255 printable ASCII characters; a UUID is ideal.

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedInput schema / properties / auth_profile
      Added value: +{
+  "description": "Load the page signed in, as this auth profile's visitor: its id, ap_…. A profile is a browser's signed-in state uploaded once over the HTTP API (POST /auth_profiles); only URLs on its domains are accepted. Costs nothing extra. https://urlpipe.dev/docs/auth-profiles",
+  "type": "string"
+}
  2. Changed2 schema fields changed
    • changedInput schema / properties / page_options / description
      Previous value: -"What to do to the page before anything is read off it. Applies to the result itself: removed ads and banners are gone from html, markdown and summaries too."New value: +"What to do to the page before anything is read off it. Applies to the result itself: removed ads and banners are gone from html, markdown and summaries too, and what scrolling loaded is in all of them."
    • addedInput schema / properties / page_options / properties / scroll_to_bottom
      Added value: +{
+  "description": "Scroll down the page before reading it, for as long as scrolling loads more (up to 25s). For listings and feeds that load their content as they are scrolled.",
+  "type": "boolean"
+}
  3. Changed1 schema field changed
    • addedInput schema / properties / page_options / properties / block_popups
      Added value: +{
+  "description": "Remove newsletter, app-install and promotional popups laid over the page.",
+  "type": "boolean"
+}
  4. Changed1 schema field changed
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "properties": {
-      "additional_author_information": {
-        "type": [
-          "object",
-          "null"
-        ]
-      },
-      "author_name": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "description": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "favicon_url": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "feed_url": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "language": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "main_image_url": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "publication_date": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "title": {
-        "type": [
-          "string",
-          "null"
-        ]
-      }
-    },
-    "type": "object"
-  },
-  {
-    "properties": {
-      "labels": {
-        "type": "object"
-      },
-      "status": {
-        "const": "accepted",
-        "type": "string"
-      },
-      "token": {
-        "type": "string"
-      }
-    },
-    "required": [
-      "token",
-      "status"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "additionalProperties": false,
+    "properties": {
+      "alternates": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "hreflang": {
+              "type": "string"
+            },
+            "url": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "hreflang",
+            "url"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "authors": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "name": {
+              "type": "string"
+            },
+            "profiles": {
+              "items": {
+                "type": "string"
+              },
+              "type": "array"
+            },
+            "url": {
+              "type": [
+                "string",
+                "null"
+              ]
+            }
+          },
+          "required": [
+            "name",
+            "url",
+            "profiles"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "canonical_url": {
+        "description": "The page's canonical URL, absolute.",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "description": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "favicon_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "feeds": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "title": {
+              "type": [
+                "string",
+                "null"
+              ]
+            },
+            "type": {
+              "type": [
+                "string",
+                "null"
+              ]
+            },
+            "url": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "url",
+            "type",
+            "title"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "final_url": {
+        "description": "The address the page ended on, after its redirects.",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "generator": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "icons": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "rel": {
+              "type": "string"
+            },
+            "sizes": {
+              "type": [
+                "string",
+                "null"
+              ]
+            },
+            "type": {
+              "type": [
+                "string",
+                "null"
+              ]
+            },
+            "url": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "url",
+            "rel",
+            "sizes",
+            "type"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "image": {
+        "additionalProperties": false,
+        "properties": {
+          "alt": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "height": {
+            "type": [
+              "integer",
+              "null"
+            ]
+          },
+          "type": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "url": {
+            "type": "string"
+          },
+          "width": {
+            "type": [
+              "integer",
+              "null"
+            ]
+          }
+        },
+        "required": [
+          "url",
+          "width",
+          "height",
+          "alt",
+          "type"
+        ],
+        "type": [
+          "object",
+          "null"
+        ]
+      },
+      "keywords": {
+        "items": {
+          "type": "string"
+        },
+        "type": "array"
+      },
+      "language": {
+        "description": "ISO 639-1 code of the language the content is written in.",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "locale": {
+        "description": "The most specific language tag the page declares for that language, BCP 47 (en-US).",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "logo_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "manifest_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "modified_at": {
+        "description": "ISO 8601, as published_at.",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "oembed_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "open_graph": {
+        "additionalProperties": false,
+        "description": "The page's og: tags as declared.",
+        "properties": {
+          "description": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "image": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "locale": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "site_name": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "title": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "type": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "url": {
+            "type": [
+              "string",
+              "null"
+            ]
+          }
+        },
+        "required": [
+          "title",
+          "description",
+          "image",
+          "url",
+          "type",
+          "site_name",
+          "locale"
+        ],
+        "type": "object"
+      },
+      "published_at": {
+        "description": "ISO 8601: a date (2024-05-01) or a moment with its offset.",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "robots": {
+        "additionalProperties": false,
+        "properties": {
+          "follow": {
+            "type": "boolean"
+          },
+          "index": {
+            "type": "boolean"
+          }
+        },
+        "required": [
+          "index",
+          "follow"
+        ],
+        "type": "object"
+      },
+      "site_name": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "structured_data_types": {
+        "items": {
+          "type": "string"
+        },
+        "type": "array"
+      },
+      "theme_color": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "title": {
+        "description": "The page's own title, without the site's name around it.",
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "twitter": {
+        "additionalProperties": false,
+        "description": "The page's twitter: tags as declared.",
+        "properties": {
+          "card": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "creator": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "description": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "image": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "site": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "title": {
+            "type": [
+              "string",
+              "null"
+            ]
+          }
+        },
+        "required": [
+          "card",
+          "site",
+          "creator",
+          "title",
+          "description",
+          "image"
+        ],
+        "type": "object"
+      },
+      "type": {
+        "description": "The kind of page, as the page declares it.",
+        "enum": [
+          "article",
+          "website",
+          "profile",
+          "book",
+          "product",
+          "video",
+          "music",
+          "place",
+          "event",
+          "recipe",
+          null
+        ],
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "video": {
+        "additionalProperties": false,
+        "properties": {
+          "height": {
+            "type": [
+              "integer",
+              "null"
+            ]
+          },
+          "type": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "url": {
+            "type": "string"
+          },
+          "width": {
+            "type": [
+              "integer",
+              "null"
+            ]
+          }
+        },
+        "required": [
+          "url",
+          "type",
+          "width",
+          "height"
+        ],
+        "type": [
+          "object",
+          "null"
+        ]
+      }
+    },
+    "required": [
+      "final_url",
+      "canonical_url",
+      "title",
+      "description",
+      "site_name",
+      "type",
+      "language",
+      "locale",
+      "authors",
+      "published_at",
+      "modified_at",
+      "image",
+      "video",
+      "favicon_url",
+      "icons",
+      "logo_url",
+      "feeds",
+      "keywords",
+      "theme_color",
+      "generator",
+      "manifest_url",
+      "robots",
+      "open_graph",
+      "twitter",
+      "alternates",
+      "oembed_url",
+      "structured_data_types"
+    ],
+    "type": "object"
+  },
+  {
+    "properties": {
+      "labels": {
+        "type": "object"
+      },
+      "status": {
+        "const": "accepted",
+        "type": "string"
+      },
+      "token": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "token",
+      "status"
+    ],
+    "type": "object"
+  }
+]
  5. Changed1 schema field changed
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "properties": {
-      "additional_author_information": {
-        "type": [
-          "object",
-          "null"
-        ]
-      },
-      "author_name": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "description": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "favicon_url": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "feed_url": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "language": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "main_image_url": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "publication_date": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "title": {
-        "type": [
-          "string",
-          "null"
-        ]
-      }
-    },
-    "type": "object"
-  },
-  {
-    "properties": {
-      "labels": {
-        "type": "object"
-      },
-      "status": {
-        "const": "accepted",
-        "type": "string"
-      },
-      "token": {
-        "type": "string"
-      }
-    },
-    "type": "object"
-  }
-]New value: +[
+  {
+    "properties": {
+      "additional_author_information": {
+        "type": [
+          "object",
+          "null"
+        ]
+      },
+      "author_name": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "description": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "favicon_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "feed_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "language": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "main_image_url": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "publication_date": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "title": {
+        "type": [
+          "string",
+          "null"
+        ]
+      }
+    },
+    "type": "object"
+  },
+  {
+    "properties": {
+      "labels": {
+        "type": "object"
+      },
+      "status": {
+        "const": "accepted",
+        "type": "string"
+      },
+      "token": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "token",
+      "status"
+    ],
+    "type": "object"
+  }
+]
  6. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "anyOf": [
+    {
+      "properties": {
+        "additional_author_information": {
+          "type": [
+            "object",
+            "null"
+          ]
+        },
+        "author_name": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "description": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "favicon_url": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "feed_url": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "language": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "main_image_url": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "publication_date": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "title": {
+          "type": [
+            "string",
+            "null"
+          ]
+        }
+      },
+      "type": "object"
+    },
+    {
+      "properties": {
+        "labels": {
+          "type": "object"
+        },
+        "status": {
+          "const": "accepted",
+          "type": "string"
+        },
+        "token": {
+          "type": "string"
+        }
+      },
+      "type": "object"
+    }
+  ],
+  "type": "object"
+}
  7. First observed

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (destructiveHint false, idempotentHint true, openWorldHint true), and the description adds genuinely non-redundant behavior: the page is rendered before reading so JS-injected values appear, missing values come back as null or empty lists, and the call costs 1 credit. It does not restate the sync/async default or caching, which live in the schema, but that is acceptable given the annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded paragraph that leads with what is read and follows with the key set, then the rendering and null semantics and the cost. The long field enumeration is dense but earns its place by telling the agent what keys to expect; 'Exposes POST /meta' is the only mildly wasteful, implementation-facing clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter, nested-schema tool with an output schema, the description covers what the agent needs: what is extracted, that rendering happens, how absences are represented, and the credit cost. The remaining gap is routing guidance against the many sibling extraction tools, which is a usage concern rather than a completeness one.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every one of the ten parameters, including the nested page_options, carries its own explanatory text. The description adds nothing about parameters, so the baseline 3 applies — the schema is doing all the work here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Reads everything a page declares about itself') and enumerates the exact output surface: Open Graph, Twitter cards, meta tags, JSON-LD, microdata, links. That enumeration effectively separates it from siblings like extract_keywords or fetch_html, but no sibling is ever named explicitly, so differentiation is inferred rather than declared.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the purpose — an agent can infer 'use this to get a page's declared metadata' but there is no explicit when-to-use, no when-not-to-use, and no pointer to alternatives such as extract_data or fetch_markdown. The mention that JavaScript-injected values are included hints at the case where fetch_html would be insufficient, but that contrast is never made.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All URLpipe toolsInspect
Try in Browser
Let your AI agent see the webWeb-to-screenshot MCP for Claude, Cursor and Codex. No cookie banners. Free to start.snap-render.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources