Gremlin Web Scraper MCP

GremlinScraper is a lightweight HTTP MCP module designed to scrape visible text from any publicly accessible webpage. It runs locally, integrates directly with VS Code’s MCP system, and speaks plain JSON.

This is Part 1 of the GremlinOS Runtime Suite from StatikFinTech LLC.

🧠 Features

MCP-Compatible: Shows up in VS Code’s MCP list with metadata.
Simple API: POST a URL, receive clean text in return.
CORS-Ready: Built-in CORS support for cross-origin requests.
Logging: Uses loguru to log all activity to rotating files.
Timeouts + Error Handling: Gracefully deals with slow or weird sites.
Human UA Header: Doesn’t look like a bot (unless you read the name).

🔧 Usage

Clone or drop this repo into your .vscode/mcps/ or wherever your MCPs live.
Add "gremlinScraper" to .mcp.json.
Click “Start Server” in the VS Code MCP tab.
Or run it manually:
pip install -r requirements.txt python server.py

📦 Endpoints & Examples

1. POST /scrape

Fetch a single page’s visible text:

curl -X POST http://localhost:8742/scrape \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com"}'

Response:

{
  "text": "Example Domain\n\nThis domain is for use in illustrative examples in documents.\n..."
}

2. POST /crawl

Recursively crawl same-domain links:

curl -X POST http://localhost:8742/crawl \
  -H 'Content-Type: application/json' \
  -d '{
    "url":"https://example.com",
    "max_pages":10,
    "max_depth":2,
    "concurrency":5
  }'

Response:

{
  "https://example.com": "Example Domain\n\nThis domain is for use…",
  "https://example.com/about": "About Us\n\n…",
  "...": "…"
}

3. POST /crawl-stream

Stream each page as soon as it’s fetched:

curl -N -X POST http://localhost:8742/crawl-stream \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","max_pages":5}'

Response (NDJSON):

{"url":"https://example.com","text":"Example Domain\n…"}
{"url":"https://example.com/link1","text":"Link One\n…"}
…

4. GET /ping

Health check endpoint:

curl http://localhost:8742/ping

Response:

pong

5. GET /mcp/metadata

MCP discovery metadata:

curl http://localhost:8742/mcp/metadata

Response:

{
  "name":"Gremlin Web Scraper MCP",
  "description":"Scrapes and crawls text from URLs via HTTP endpoints…",
  "version":"0.0.1",
  "author":"StatikFinTech LLC",
  "tags":["scraping","crawl","MCP","runtime"],
  "endpoints":[…]
}

🗂 Metadata

Name: Gremlin Web Scraper MCP
Author: StatikFinTech LLC
License: MIT
Tags: #scraping, #crawl, #runtime, #gremlin

🐾 Future Add-ons

PDF / EPUB / Markdown parsing
Selective DOM element filtering
Scheduling/recurring crawl and scrap jobs
Direct Memory injection to GremlinGPT core

“Split. Streamlined. Sovereign.” StatikFinTech Systems • 2025

Caution

“Your qualifications are impressive...”

Coder Hiring Team (2025 Rejection Letter)

🔱 "This isn't rejection. It's proof they don't know how to build what comes next.

Still building what they can’t classify." 🔱 -StatikFinTech, LLC

Ascend Institute Traffic

The world’s first RS-RACS
Recursive, Self-Referential Autonomous Cognitive System

This server cannot be installed

security - not tested

license - permissive license

quality - not tested

How are these scores calculated?

local-only server

The server can only run on the client's local machine because it depends on local resources.

A lightweight HTTP module that enables scraping visible text from any publicly accessible webpage, integrating directly with VS Code's MCP system.

Related MCP Servers

UseScraper MCP Server
tanevanwifferen
A
security
A
license
A
quality
A TypeScript-based MCP server utilizing the UseScraper API to provide web scraping capabilities, allowing users to extract content from webpages in various formats.
Last updated -
1
2
JavaScript
MIT License
MD Webcrawl MCP
jmh108
-
security
A
license
-
quality
A Python-based MCP server that crawls websites to extract and save content as markdown files, with features for mapping website structure and links.
Last updated -
3
Python
MIT License
Website Scraper MCP Server
tolik-unicornrider
A
security
F
license
A
quality
An MCP server that extracts meaningful content from websites and converts HTML to high-quality Markdown, using Mozilla's Readability engine.
Last updated -
1
7,866
6
JavaScript
Vinted-scrapper
Gertje823
-
security
A
license
-
quality
This MCP scraps vinted for product info. Disclaimer: This script is designed for educational purposes only. It is intended to demonstrate web scraping techniques and should not be used for any commercial or personal gain. Please note that using this software may violate the terms of service of Vint
Last updated -
105
Python
GPL 3.0

View all related MCP servers

Gremlin Web Scraper MCP