Skip to main content
Glama
web-basics-mcp

web-basics-mcp

by yoloyash

web-basics

A local MCP server for resilient web search and safe URL fetching. It runs over stdio, supports configured and keyless search providers, and leaves answer synthesis to the client.

Requirements

  • Node.js 20.18.1 or newer

  • A SearXNG instance or Brave Search API key is optional; keyless search can be enabled explicitly

fetch_url works without a configured search provider.

Related MCP server: Basic MCP Tools

Add To Codex

For automatic fallback with no required credentials:

codex mcp add web-basics \
  --env SEARCH_PROVIDER=auto \
  --env SEARCH_ALLOW_KEYLESS_FALLBACK=true \
  -- npx -y @yoloyash/web-basics

Automatic search uses only configured providers by default. Enabling SEARCH_ALLOW_KEYLESS_FALLBACK appends keyless Firecrawl, anonymous Exa MCP, and DuckDuckGo HTML. Searches are sequential and stop after the first provider returns renderable content. Empty responses fall through like provider errors. Keyless services are best-effort and may enforce their own rate limits or bot challenges.

To prioritize Brave and then a SearXNG instance before keyless fallback:

codex mcp add web-basics \
  --env SEARCH_PROVIDER=auto \
  --env BRAVE_SEARCH_API_KEY=your-subscription-token \
  --env SEARXNG_URL=http://127.0.0.1:8088 \
  --env SEARCH_ALLOW_KEYLESS_FALLBACK=true \
  -- npx -y @yoloyash/web-basics

To pin SearXNG without fallback:

codex mcp add web-basics \
  --env SEARCH_PROVIDER=searxng \
  --env SEARXNG_URL=http://127.0.0.1:8088 \
  -- npx -y @yoloyash/web-basics

For another stdio MCP client:

{
  "command": "npx",
  "args": ["-y", "@yoloyash/web-basics"],
  "env": {
    "SEARCH_PROVIDER": "searxng",
    "SEARXNG_URL": "http://127.0.0.1:8088"
  }
}

SEARCH_PROVIDER accepts auto, brave, searxng, firecrawl, exa, or duckduckgo. It defaults to searxng, and explicitly selected SearXNG defaults to http://127.0.0.1:8088. In automatic mode, Brave and SearXNG are included only when their corresponding configuration is set. SEARCH_ALLOW_KEYLESS_FALLBACK=true appends Firecrawl, Exa, and DuckDuckGo; it has no effect on explicitly selected providers.

Programmatic callers can enable the same policy with createWebBasics({ searchBackend: "auto", allowKeylessFallback: true }).

To use Brave Search instead:

codex mcp add web-basics \
  --env SEARCH_PROVIDER=brave \
  --env BRAVE_SEARCH_API_KEY=your-subscription-token \
  -- npx -y @yoloyash/web-basics

Brave uses its official Web Search API. SearXNG searches retain a bounded two-minute cache. Brave and keyless providers coalesce concurrent identical requests but do not retain completed responses.

Tools

Searches the selected provider or automatic fallback chain.

  • query: search query

  • limit: optional result count from 1 to 10; defaults to 5

  • recency: optional day, week, month, or year filter

  • max_tokens: optional provider answer token cap

  • temperature: optional provider sampling temperature

  • num_search_results: optional provider search breadth or local result cap from 1 to 10

The structured result uses a unified provider response contract:

{
  "response": {
    "provider": "firecrawl",
    "sources": [
      {
        "title": "Example result",
        "url": "https://example.com/result",
        "snippet": "Example snippet"
      }
    ],
    "requestId": "request-id-if-provided",
    "authMode": "keyless"
  }
}

response.provider identifies the provider that served the result. Providers may also return answer, citations, searchQueries, relatedQuestions, usage, model, requestId, and authMode. A successful response does not include previous failed attempts. If every provider fails, the MCP call returns Error: ... as normal text plus { response: { provider, sources: [] }, error } as structured content.

The root API returns the SearchResponse directly:

const web = createWebBasics({
  searchBackend: "auto",
  allowKeylessFallback: true,
});

const response = await web.webSearch({ query: "TypeScript 6", limit: 5 });
console.log(response.provider, response.sources);

fetch_url

Fetches one public HTTP(S) URL.

  • url: URL to fetch

  • start_index: optional character offset; defaults to 0

  • max_length: optional character limit from 1 to 20,000; defaults to 8,000

Supports readable web pages, PDFs, direct text formats, Reddit posts, X posts, YouTube video transcripts, and PNG, JPEG, WebP, or GIF images. When text is truncated, call the tool again with next_start_index.

Reddit posts (reddit.com/r/.../comments/<id>/..., reddit.com/comments/<id>, and redd.it/<id> links) are read from Reddit's public Atom feed, which is the only route Reddit still serves without a login. The result contains the post body, the outbound link for link posts, and top-level comments as Markdown. Reddit throttles anonymous feed requests to about one per minute per IP; a throttled call returns an HTTP 429 error that states how many seconds to wait. Successful responses are cached by post ID for up to one hour while respecting stricter upstream cache directives.

YouTube video URLs (watch?v=, youtu.be, shorts, embed, and live forms) return the title, channel, duration, description, and a timestamped transcript built from YouTube's public caption tracks. Manual English captions are preferred, then automatic English captions, then the first manual track in any language. Videos without caption tracks return their metadata and a note that no transcript is available. Results are cached by video ID for up to one hour.

X post URLs (x.com/<user>/status/<id>, twitter.com/<user>/status/<id>, and x.com/i/status/<id> forms) are read from the public syndication endpoint that powers embedded posts. The result contains the post text with expanded links, media URLs with alt text, the link card, like and reply counts, and the parent and quoted posts when present. If the syndication endpoint fails, the official oEmbed endpoint supplies the text alone and fallback_reason says so. X serves only the first 280 characters of long posts to logged-out clients; such results carry a note. Results use the default fetch cache keyed by post ID.

Both tools expose MCP output schemas and return structured content alongside text content.

Safety And Scope

Public provider requests and fetched URLs are limited to safe HTTP(S) destinations. The server rejects URL credentials, unsafe DNS results and redirects, unsupported content types, and oversized responses. User-managed SearXNG endpoints may be private.

Keyless fallback is opt-in because search queries are sent to third-party services. Its order is Firecrawl, Exa, then DuckDuckGo; select one of those providers explicitly to use only that service.

This package does not provide JavaScript rendering, browser automation, crawling, authenticated page fetching, proxy routing, bundled search infrastructure, or answer synthesis.

Development

npm ci
npm test
npm pack --dry-run

Normal tests do not access the public internet. Live smoke tests should explicitly select the provider being tested; Brave and SearXNG require their corresponding configuration.

Available Tools

2 tools
fetch_urlA
Read-onlyIdempotent

Fetch one public HTTP(S) URL. Returns clean Markdown for pages, Reddit posts, X posts, and YouTube video transcripts, extracted PDF text, direct text data, or a supported image. Use start_index to continue truncated text.

Inputs: url (required), max_length, start_index. Returns: url, links, title, content, +12 more

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 2 tools

    Disambiguation5/5

    fetch_url and web_search target clearly separate actions—retrieving a single known URL vs. discovering URLs via a query. There is no realistic way to confuse one for the other.

    Naming Consistency5/5

    Both names follow the same verb_noun snake_case pattern (fetch_url, web_search) and the verbs accurately signal their actions. Fully consistent.

    Tool Count3/5

    Two tools is very thin, but it covers the minimal retrieve-and-discover pair that defines a 'web basics' server. It borders on underspecified, since there is no way to select a provider, batch fetches, or crawl links, but the scope is intentionally minimal.

    Completeness4/5

    Fetch and search together cover the core read-only web workflow, with fold-out detail like start_index pagination, PDF/image handling, and YouTube/Reddit/X support baked into the descriptions. Gaps remain for multi-URL fetching, link extraction, or provider choice, which agents must work around by issuing repeated calls.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      A
      quality
      D
      maintenance
      MCP server that provides a search_web tool to query a self-hosted SearXNG instance and return structured web search results.
      1
      -
    • F
      license
      Not graded
      quality
      B
      maintenance
      A self-hosted MCP server providing private web search, web page fetching, and current date/time tools, powered by a bundled SearXNG instance for API-key-free local search.
      2
      -