web_fetch

Instructions

Implementation Reference

• src/tools/webFetch.ts:4-22 (handler)

  The webFetch function that implements the tool's core logic: fetches a URL with size/time limits, determines if the response is text or binary, and returns structured output.

  export async function webFetch(url: string) {
    const res = await fetchWithLimits(url, CONFIG.fetchTimeoutMs, CONFIG.maxFetchBytes);
    if (!res || !res.body) {
      return {
        finalUrl: url, status: res?.status || 0,
        contentType: res?.contentType || 'application/octet-stream',
        bodyText: null, bytesB64: null, fetchedAt: new Date().toISOString()
      };
    }
    const ct = (res.contentType || '').toLowerCase();
    const isText = ct.startsWith('text/') || ct.includes('html') || ct.includes('xml') || ct.includes('json');
    return {
      finalUrl: res.finalUrl || url, status: res.status,
      contentType: res.contentType,
      bodyText: isText ? res.body.toString('utf-8') : null,
      bytesB64: isText ? null : res.body.toString('base64'),
      fetchedAt: new Date().toISOString()
    };
  }

• src/server.ts:65-70 (schema)

  Input schema for the web_fetch tool: defines url (required), timeout, max_bytes, and headers fields.

  const webFetchShape = {
    url: z.string().url(),
    timeout: z.number().int().optional(),
    max_bytes: z.number().int().optional(),
    headers: z.record(z.string()).optional()
  };

• src/server.ts:78-84 (registration)

  Registration of the 'web_fetch' tool (alias of 'web.fetch') via server.tool(), binding the schema to the webFetch handler.

  server.tool('web_fetch', 'Alias of web.fetch',
    webFetchShape, OPEN,
    async ({ url, timeout, max_bytes }) => {
      const res = await webFetch(url);
      return { content: [{ type: 'text', text: JSON.stringify(res) }] };
    }
  );

• src/utils/http.ts:40-79 (helper)

  The fetchWithLimits helper: performs HTTP GET with SSRF protection, timeout via AbortController, and max byte limiting.

  export async function fetchWithLimits(urlStr: string, timeoutMs = CONFIG.fetchTimeoutMs, maxBytes = CONFIG.maxFetchBytes) {
    const u = new URL(urlStr);
    await assertNotPrivate(u);
  
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), timeoutMs);
    try {
      const res = await request(urlStr, {
        method: 'GET',
        headers: { 'user-agent': 'mcp-multitool/0.2', 'accept': '*/*' },
        signal: controller.signal,
        maxRedirections: 3
      });
  
      const status = res.statusCode;
      const headersRec: Record<string, string> = {};
      for (const [k, v] of Object.entries(res.headers)) {
        headersRec[k] = Array.isArray(v) ? v.join(', ') : String(v ?? '');
      }
  
      if (status >= 400) {
        return { status, headers: headersRec, body: null as any };
      }
  
      const chunks: Buffer[] = [];
      let total = 0;
      for await (const chunk of res.body) {
        const b: Buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk as any);
        total += b.length;
        if (total > maxBytes) break;
        chunks.push(b);
      }
      const buf = Buffer.concat(chunks);
      const contentType = headersRec['content-type'] || 'application/octet-stream';
      const finalUrl = headersRec['content-location'] || urlStr;
      return { status, headers: headersRec, body: buf, finalUrl, contentType };
    } finally {
      clearTimeout(timer);
    }
  }

• src/utils/http.ts:23-38 (helper)

  The assertNotPrivate helper: DNS-resolves and checks if the target IP is in private/reserved ranges (anti-SSRF).

  async function assertNotPrivate(url: URL) {
    const host = url.hostname;
    if (net.isIP(host)) {
      if (ipInRanges(host, BLOCKEDv4) || ipInRanges(host, BLOCKEDv6)) {
        throw new Error('Blocked private IP (SSRF)');
      }
      return;
    }
    const addrs = await dns.lookup(host, { all: true });
    for (const a of addrs) {
      if ((a.family === 4 && ipInRanges(a.address, BLOCKEDv4)) ||
          (a.family === 6 && ipInRanges(a.address, BLOCKEDv6))) {
        throw new Error('Blocked private IP (SSRF)');
      }
    }
  }