Skip to main content
Glama
Octen-Team

octen-mcp

Official
by Octen-Team
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

URL Extract

extract
Read-only

Read one or more web pages by URL and return clean, LLM-ready markdown or text, handling hard sites with advanced mode.

Instructions

Read one or more web pages by URL and return clean, LLM-ready content (markdown or text). By default (no query) it returns each page's full content — this is what you want in almost all cases. Only pass query when the user explicitly asks to fetch relevance-ranked snippets for a specific topic; doing so returns highlights INSTEAD of the full body, so the content will be partial. Every result also includes a category (topical) and page_structure (typology) classification. Bare hosts like 'octen.ai' are auto-normalized to https. Cached when fresh.

Pick mode by how hard the URLs are: ordinary sites → standard (or omit), known hard sites → advanced, mixed/unsure → auto. If a standard result failed or came back empty/skeletal, retry just those URLs with mode "advanced".

Use this when you already have the URL(s). To find pages first, use search or broad_search.

keywords: read page, fetch url, scrape, page content, article text, parse webpage, extract, read article, url content, open link

Input Schema

NameRequiredDescriptionDefault
modeNoExtraction mode; omitted means standard. standard: fastest, cheapest; right for news, blogs, docs, product and static pages. On login-walled, JS-heavy or anti-bot sites it may return empty or skeletal content yet still report success. advanced: highest success rate (real browser, stronger anti-bot), slower, 2.5x price; for anti-bot/WAF sites, JS-heavy SPAs, social (Reddit, X, LinkedIn), academic (ResearchGate), dynamically loaded content. auto: picks per URL, escalating to advanced only where needed, billed per mode used; for mixed batches. With advanced/auto, raise `timeout` (e.g. 60). Each result's resolved_mode is the mode actually used.
urlsYesURLs to extract. 1-20 per call, each ≤2048 chars. Bare hosts ok.
queryNoOptional — leave UNSET in the normal case. When unset, each result returns the page's `full_content` (the complete text). Only set this when the user explicitly wants relevance-ranked snippets for a specific query/topic: setting it makes each result return `highlights` (ranked excerpts) and OMIT `full_content`, so the page body will be incomplete. Do not pass it just to focus a normal fetch.
formatNoOutput format. Default markdown.markdown
timeoutNoPer-URL timeout in seconds (1-60).
include_audioNoReturn audio URLs found on each page.
include_linksNoReturn links found on each page — to discover a site's pages or follow outbound references. `{}` uses the defaults. Each link carries is_external.
include_imagesNoReturn image resources found on each page (also enables `cover_image` when the page has one).
include_videosNoReturn video URLs found on each page.
max_age_secondsNoMaximum age of cached content in seconds. Default 24h. Lower this for time-sensitive pages (news / prices).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv0.6.0
    • addedInput schema / properties / include_links
      Added value: +{
+  "description": "Return links found on each page — to discover a site's pages or follow outbound references. `{}` uses the defaults. Each link carries is_external.",
+  "properties": {
+    "max_links": {
+      "description": "Maximum links per page, 1-1000. Default 200.",
+      "maximum": 1000,
+      "minimum": 1,
+      "type": "integer"
+    },
+    "scope": {
+      "description": "Which links are listed first (page order kept within each group). prefer_internal (default): same-site first; prefer_external: other registered domains first.",
+      "enum": [
+        "prefer_internal",
+        "prefer_external"
+      ],
+      "type": "string"
+    }
+  },
+  "type": "object"
+}
    • addedInput schema / properties / mode
      Added value: +{
+  "description": "Extraction mode; omitted means standard. standard: fastest, cheapest; right for news, blogs, docs, product and static pages. On login-walled, JS-heavy or anti-bot sites it may return empty or skeletal content yet still report success. advanced: highest success rate (real browser, stronger anti-bot), slower, 2.5x price; for anti-bot/WAF sites, JS-heavy SPAs, social (Reddit, X, LinkedIn), academic (ResearchGate), dynamically loaded content. auto: picks per URL, escalating to advanced only where needed, billed per mode used; for mixed batches. With advanced/auto, raise `timeout` (e.g. 60). Each result's resolved_mode is the mode actually used.",
+  "enum": [
+    "standard",
+    "advanced",
+    "auto"
+  ],
+  "type": "string"
+}
  2. Changed3 schema fields changedv0.5.0
    • addedInput schema / properties / max_age_seconds / maximum
      Added value: +31536000
    • changedInput schema / properties / urls / description
      Previous value: -"URLs to extract. 1-20 per call. Bare hosts ok."New value: +"URLs to extract. 1-20 per call, each ≤2048 chars. Bare hosts ok."
    • addedInput schema / properties / urls / items / maxLength
      Added value: +2048
  3. Addedv0.3.4
  4. Removedv0.3.3
  5. Changed2 schema fields changedv0.3.1
    • removedInput schema / properties / include_favicon
      Removed value: -{
-  "default": false,
-  "description": "Return each page's favicon URL.",
-  "type": "boolean"
-}
    • changedInput schema / properties / include_images / description
      Previous value: -"Return image URLs found on each page."New value: +"Return image resources found on each page (also enables `cover_image` when the page has one)."
  6. Changed1 schema field changedv0.2.2
    • changedInput schema / properties / query / description
      Previous value: -"Optional intent-focused keywords. When set, each result returns `highlights` (most relevant snippets, ranked) instead of `full_content`."New value: +"Optional — leave UNSET in the normal case. When unset, each result returns the page's `full_content` (the complete text). Only set this when the user explicitly wants relevance-ranked snippets for a specific query/topic: setting it makes each result return `highlights` (ranked excerpts) and OMIT `full_content`, so the page body will be incomplete. Do not pass it just to focus a normal fetch."
  7. First observedv0.1.0

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safe-read profile (readOnly, openWorld, non-destructive); the description adds substantial behavior beyond that: caching with TTL, auto-normalization of bare hosts, cost differential (2.5x for advanced), silent-success failure mode ('may return empty or skeletal content yet still report success'), per-result resolved_mode, and the content-partiality tradeoff when `query` is set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core contract (full content by default, query is the exception), then mode strategy, then routing. Slightly long, with the query caveat stated twice (description paragraph and schema) and a keyword tail that adds bulk rather than routing value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter tool with nested objects and no output schema, the description still covers the return shape (full_content vs highlights, category, page_structure, resolved_mode), caching semantics, and failure/recovery paths. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description nonetheless adds operational meaning beyond the schema — the escalation heuristic for `mode`, the retry-with-advanced recovery pattern, and raising `timeout` in tandem with advanced/auto. Some of the query/mode prose does duplicate the schema, keeping it short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource+output ('Read one or more web pages by URL and return clean, LLM-ready content (markdown or text)'). Explicitly differentiates from siblings: 'Use this when you already have the URL(s). To find pages first, use search or broad_search.' An agent can route between extract and the search family without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use/when-not guidance for the `query` parameter ('Only pass query when the user explicitly asks... returns highlights INSTEAD of the full body'), for `mode` selection ('ordinary sites → standard, known hard sites → advanced, mixed/unsure → auto'), plus a concrete retry heuristic ('If a standard result failed or came back empty/skeletal, retry just those URLs with mode "advanced"').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools