Skip to main content
Glama
webscraping-ai

WebScraping-AI MCP Server

Official

webscraping_ai_question

Ask a question about a web page to extract specific information. Provide the URL and question, and get an answer based on the page content.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesURL of the target page.
questionYesQuestion or instructions to ask the LLM model about the target page.
timeoutNoMaximum web page retrieval time in ms (20000 by default, maximum is 30000).
jsNoExecute on-page JavaScript using a headless browser (false by default).
js_timeoutNoMaximum JavaScript rendering time in ms (3000 by default).
wait_forNoCSS selector to wait for before returning the page content.
proxyNoType of proxy, datacenter or residential (datacenter by default).datacenter
countryNoCountry of the proxy to use (US by default).
custom_proxyNoYour own proxy URL in "http://user:password@host:port" format.
deviceNoType of device emulation.
error_on_404NoReturn error on 404 HTTP status on the target page (false by default).
error_on_redirectNoReturn error on redirect on the target page (false by default).
js_scriptNoCustom JavaScript code to execute on the target page.

Implementation Reference

  • src/index.js:231-246 (registration)
    Registration of the 'webscraping_ai_question' tool with the MCP server, including Zod schema for URL, question, and common options.
    server.tool(
      'webscraping_ai_question',
      {
        url: z.string().describe('URL of the target page.'),
        question: z.string().describe('Question or instructions to ask the LLM model about the target page.'),
        ...commonOptionsSchema
      },
      async ({ url, question, ...options }) => {
        try {
          const result = await client.question(url, question, options);
          return createSanitizedResponse(result, url);
        } catch (error) {
          return createSanitizedResponse(error.message, url, true);
        }
      }
    );
  • Handler function for the 'webscraping_ai_question' tool - calls client.question() and returns sanitized response.
    async ({ url, question, ...options }) => {
      try {
        const result = await client.question(url, question, options);
        return createSanitizedResponse(result, url);
      } catch (error) {
        return createSanitizedResponse(error.message, url, true);
      }
    }
  • Input schema for the tool: url (string), question (string), plus common options like timeout, js, proxy, etc.
    {
      url: z.string().describe('URL of the target page.'),
      question: z.string().describe('Question or instructions to ask the LLM model about the target page.'),
      ...commonOptionsSchema
  • Client.question() method on WebScrapingAIClient that makes the API request to '/ai/question' endpoint.
    async question(url, question, options = {}) {
      return this.request('/ai/question', {
        url,
        question,
        ...options
      });
    }
  • Common options schema shared across all tools, used by the question tool.
    const commonOptionsSchema = {
      timeout: z.number().optional().default(DEFAULT_TIMEOUT).describe(`Maximum web page retrieval time in ms (${DEFAULT_TIMEOUT} by default, maximum is 30000).`),
      js: z.boolean().optional().default(DEFAULT_JS_RENDERING).describe(`Execute on-page JavaScript using a headless browser (${DEFAULT_JS_RENDERING} by default).`),
      js_timeout: z.number().optional().default(DEFAULT_JS_TIMEOUT).describe(`Maximum JavaScript rendering time in ms (${DEFAULT_JS_TIMEOUT} by default).`),
      wait_for: z.string().optional().describe('CSS selector to wait for before returning the page content.'),
      proxy: z.enum(['datacenter', 'residential']).optional().default(DEFAULT_PROXY_TYPE).describe(`Type of proxy, datacenter or residential (${DEFAULT_PROXY_TYPE} by default).`),
      country: z.enum(['us', 'gb', 'de', 'it', 'fr', 'ca', 'es', 'ru', 'jp', 'kr', 'in']).optional().describe('Country of the proxy to use (US by default).'),
      custom_proxy: z.string().optional().describe('Your own proxy URL in "http://user:password@host:port" format.'),
      device: z.enum(['desktop', 'mobile', 'tablet']).optional().describe('Type of device emulation.'),
      error_on_404: z.boolean().optional().describe('Return error on 404 HTTP status on the target page (false by default).'),
      error_on_redirect: z.boolean().optional().describe('Return error on redirect on the target page (false by default).'),
      js_script: z.string().optional().describe('Custom JavaScript code to execute on the target page.')
    };
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/webscraping-ai/webscraping-ai-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server