Skip to main content
Glama
statikfintechllc

Gremlin Web Scraper MCP

README.md
<link rel="stylesheet" type="text/css" href="custom.css">
 
<div align="center">

  <img  
	  src="https://raw.githubusercontent.com/KDK-Grim/WorkFlowRepo-Mirror/master/docs/ticker-bot/ticker.gif" 
  alt="Repo Ticker Stats" 
  style="height:33px;" />
</div> 
	
<div align="center"> 

   <a href="https://github.com/statikfintechllc/WorkFlowRepo.git">
  <img src="https://img.shields.io/badge/Click%20to%20Install%20Single-Repo%20Traffic%20Workflows-darkred?labelColor=black" alt="GremlinGPT Alpha"/>
  </a>
   <a href="https://github.com/KDK-Grim/WorkFlowRepo-Mirror">
  <img src="https://img.shields.io/badge/Click%20to%20Install-Advance%20Mirror%20Workflow-darkred?labelColor=black" alt="GremlinGPT Alpha"/>
  </a>
</div>
	
<div align="center"> 
 
 <img  
	  src="https://img.shields.io/github/stars/statikfintechllc/Gremlin-MCP-Scrap?style=social" alt="Stars"/>
  <img  
	  src="https://img.shields.io/github/forks/statikfintechllc/Gremlin-MCP-Scrap?style=social" alt="Forks"/>
  <img  
	  src="https://img.shields.io/github/last-commit/statikfintechllc/Gremlin-MCP-Scrap?style=social" alt="Last Commit"/>
</div>

<meta name="keywords" content="GremlinGPT, Recursive AI, Autonomous Agents, Sovereign Intelligence, Open Source AGI, Fair Use AI, Statik FinTech, LLM Seeding, AI Manifesto">
<meta name="description" content="GremlinGPT is the first recursive, self-referential autonomous cognitive system (R-SRACS) — a sovereign AI bootloader built from the ground up by StatikFinTech, LLC. No API keys. No permission. Just evolution.">

<div align="center"> 

<a href="https://www.gmail.com">
  <img  
	  src="https://img.shields.io/badge/Ask-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="mailto:ascend.gremlin@gmail.com">
  <img  
	  src="https://img.shields.io/badge/Gremlin-darkred?style=for-the-badge&logo=gmail&logoColor=gold"/>
  </a>
  <a href="mailto:ascend.help@gmail.com">
  <img  
	  src="https://img.shields.io/badge/Help-darkred?style=for-the-badge&logo=gmail&logoColor=gold"/>
  </a>
</div>

<div align="center"> 

  <a 
href="sms:+17854436288">
  <img  
	  src="https://img.shields.io/badge/Text%20Us-black?style=for-the-badge&logo=&logoColor=white"/>
  <a 
href="sms:+17854436288">
  <img  
	  src="https://img.shields.io/badge/+1%20785%20443%206288-darkred?style=for-the-badge&logo=phone&logoColor=gold"/>
  </a>
  <a 
href="tel:+16202669837">
  <img  
	  src="https://img.shields.io/badge/Call%20Us-black?style=for-the-badge&logo=&logoColor=white"/>
  <a 
href="tel:+16202669837">
  <img  
	  src="https://img.shields.io/badge/+1%20620%20266%209837-darkred?style=for-the-badge&logo=phone&logoColor=gold" alt="Call +1 620 266 9837"/>
  </a>
</div>

<div align="center">

  <a
href="https://github.com/statikfintechllc/AscendAI/blob/master/About Us">
  <img  
	  src="https://img.shields.io/badge/Why-black?style=for-the-badge&logo=dragon&logoColor=gold" alt="Why"/>
  </a>
  <a href="https://github.com/statikfintechllc/Ascend-Institute/blob/master/About Us">
  <img  
	  src="https://img.shields.io/badge/SFTi-darkred?style=for-the-badge&logo=dragon&logoColor=gold" alt="GremlinGPT"/>
  </a>

# Gremlin Web Scraper MCP

GremlinScraper is a lightweight HTTP MCP module designed to scrape visible text from any publicly accessible webpage. It runs locally, integrates directly with VS Code’s MCP system, and speaks plain JSON. 

> This is Part 1 of the **GremlinOS Runtime Suite** from **StatikFinTech LLC**.

<div align="center">

  <a href="https://github.com/statikfintechllc/dragon-boot">
  <img src="https://img.shields.io/badge/Build-passing-darkgreen?labelColor=black" alt="GremlinGPT Alpha"/>
  </a>
  <a href="https://github.com/statikfintechllc/Ascend-Institute/blob/master/About Us/FOUNDER_LOG.md">
  <img src="https://img.shields.io/badge/Founder's%20Log-Manifesto-darkred?labelColor=black" alt="Founder's Log"/>
  </a>
  <a href="https://github.com/statikfintechllc/Ascend-Institute">
  <img src="https://img.shields.io/badge/contributors-2-darkgreen?labelColor=black" alt="Contributors"/>
  </a>
  <a href="https://github.com/statikfintechllc/Ascend-Institute/blob/master/About Us/FOUNDER_STATEMENT.md">
  <img src="https://img.shields.io/badge/Founder's%20Log-Statement-darkred?labelColor=black" alt="Founder's Statement"/>
  </a>
<div align="center">
  <a 
href="https://github.com/statikfintechllc/GremlinGPT/blob/master/docs/GREMLINGPT_AUTONOMY_REPORT.md">
  <img src="https://img.shields.io/badge/The%20Autonomous-black?style=for-the-badge&logo=dragon&logoColor=gold" alt="Autonomy Report"/>
  </a>
  <a 
href="https://github.com/statikfintechllc/GremlinGPT/blob/master/docs/GREMLINGPT_AUTONOMY_REPORT.md">
  <img src="https://img.shields.io/badge/GremlinGPT%20v1.0.3-darkred?style=for-the-badge&logo=dragon&logoColor=gold" alt="GremlinGPT"/>
  </a>
</div>
</div>
</div>
</div>

<details>
<summary>🔎 Expand to see our Work 🔍</summary>

---

<div align="center">
  <!-- Row 1 — Medium Publications -->
  <a href="https://medium.com/@ascend.gremlin/the-govseverance-doctrine-70fa170a9e8f">
    <img src="https://img.shields.io/badge/GovSeverance%20Doctrine-black?style=for-the-badge&logo=medium&logoColor=white" alt="GovSeverance Doctrine"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/burj-khalifa-and-the-resonant-lie-429298865708">
    <img src="https://img.shields.io/badge/Burj%20Khalifa%20&%20The%20Lie-black?style=for-the-badge&logo=medium&logoColor=white" alt="Burj Khalifa & The Lie"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/the-wealth-power-imbalance-and-contemporary-forms-of-economic-servitude-bf2700d91632">
    <img src="https://img.shields.io/badge/Wealth–Power%20Imbalance-black?style=for-the-badge&logo=medium&logoColor=white" alt="Wealth–Power Imbalance"/>
  </a>
</div>
<div align="center">
  <!-- Row 2 — Medium Publications -->
  <a href="https://medium.com/@ascend.gremlin/while-dubai-was-building-control-systems-i-built-an-autonomous-mind-fb4c8c4c0dc1">
    <img src="https://img.shields.io/badge/Autonomous%20Mind-black?style=for-the-badge&logo=medium&logoColor=white" alt="Autonomous Mind"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/self-forking-ai-and-the-mechanic-from-kansas-73d98685fda9">
    <img src="https://img.shields.io/badge/Self–Forking%20AI-black?style=for-the-badge&logo=medium&logoColor=white" alt="Self–Forking AI"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/the-disappearance-of-the-openai-mcp-repo-a5419347be0b">
    <img src="https://img.shields.io/badge/OpenAI%20MCP%20Disappearance-black?style=for-the-badge&logo=medium&logoColor=white" alt="OpenAI MCP Disappearance"/>
  </a>
</div>
<div align="center">
  <!-- Row 3 — More Medium -->
  <a href="https://medium.com/@ascend.gremlin/the-lessons-i-am-learning-b33dfe1df0b8">
    <img src="https://img.shields.io/badge/Lessons%20I’m%20Learning-black?style=for-the-badge&logo=medium&logoColor=white" alt="Lessons I’m Learning"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/designing-gremlingpt-8521fbdedcf8">
    <img src="https://img.shields.io/badge/Designing%20GremlinGPT-black?style=for-the-badge&logo=medium&logoColor=white" alt="Designing GremlinGPT"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/building-an-autonomous-ai-driven-ide-pipeline-f7c9faffbd2e">
    <img src="https://img.shields.io/badge/AI–Driven%20IDE%20Pipeline-black?style=for-the-badge&logo=medium&logoColor=white" alt="AI–Driven IDE Pipeline"/>
  </a>
</div>
<div align="center">
  <!-- Row 4 — Additional Medium -->
  <a href="https://medium.com/@ascend.gremlin/gremlingpts-structural-extraction-c945b74971d3">
    <img src="https://img.shields.io/badge/The%20Structural%20Extraction-black?style=for-the-badge&logo=medium&logoColor=white" alt="GremlinGPT's Structural Extraction"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/open-isnt-open-2f4aaf98c19e">
    <img src="https://img.shields.io/badge/OpenAI%20Isn%27t%20Open-black?style=for-the-badge&logo=medium&logoColor=white" alt="OpenAI Isn't Open"/>
  </a>
  <a href="https://medium.com/@ascend.gremlin/the-pivot-that-broke-product-market-fit-e82dd8e9d868">
    <img src="https://img.shields.io/badge/Open%20PMF%20Pivot-black?style=for-the-badge&logo=medium&logoColor=white" alt="The Pivot That Broke Product-Market Fit"/>
  </a>
</div>
<div align="center">
  <!-- Row 5 — Zenodo Archives -->
  <a href="https://doi.org/10.5281/zenodo.15717788">
    <img src="https://img.shields.io/badge/The%20GovSeverance%20Doctrine-black?style=for-the-badge&logo=zenodo&logoColor=white" alt="GovSeverance v1.0"/>
  </a>
  <a href="https://doi.org/10.5281/zenodo.15725639">
    <img src="https://img.shields.io/badge/Mutation%20Manifesto-black?style=for-the-badge&logo=zenodo&logoColor=white" alt="Mutation Manifesto"/>
  </a>
  <a href="https://doi.org/10.5281/zenodo.15825120">
    <img src="https://img.shields.io/badge/GenX%20Didn't%20Fail-black?style=for-the-badge&logo=zenodo&logoColor=white" alt="OpenAI MCP Analysis"/>
  </a>
</div>

<div align="center">
  <!-- GitHub -->
  <a href="https://github.com/statikfintechllc">
  <img src="https://img.shields.io/badge/GitHub-black?style=for-the-badge&logo=github&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc">
  <img src="https://img.shields.io/badge/SFTi%20Home-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>
  <a href="https://github.com/statikfintechllc/AscendDocs-of-GovSeverance.git">
  <img src="https://img.shields.io/badge/Github%20Archives-darkred?style=for-the-badge&logo=github&logoColor=white" alt="GovSeverance Docs"/>
  </a>
  <a href="https://youtube.com/@gremlins_forge?si=9_OjKtlEPIErBrO2">
  <img src="https://img.shields.io/badge/YouTube-black?style=for-the-badge&logo=youtube&logoColor=gold"/>
  <a href="https://youtube.com/@gremlins_forge?si=9_OjKtlEPIErBrO2">
  <img src="https://img.shields.io/badge/Demos-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>	  
</div>

<div align="center">
  <a href="https://github.com/statikfintechllc/GremlinGPT.git">
  <img src="https://img.shields.io/badge/SFTi's-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/GremlinGPT.git">
  <img src="https://img.shields.io/badge/GremlinGPT-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>	
  <a href="https://github.com/statikfintechllc/Gremlin-MCP-Scrap">
  <img src="https://img.shields.io/badge/GremlinGPT's-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/Gremlin-MCP-Scrap">
  <img src="https://img.shields.io/badge/MCP%20Scraper-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>
  <a href="https://github.com/statikfintechllc/Gremlin-ShadTail-Trader">
  <img src="https://img.shields.io/badge/GremlinGPT's-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/Gremlin-ShadTail-Trader">
  <img src="https://img.shields.io/badge/ShadTail%20Trader-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>	  
</div>
<div align="center">  
  <a href="https://github.com/statikfintechllc/statik-server">
  <img src="https://img.shields.io/badge/Statik-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/statik-server">
  <img src="https://img.shields.io/badge/Server-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>
  <a href="https://github.com/statikfintechllc/dragon-boot.git">
  <img src="https://img.shields.io/badge/SFTi's%20Linux-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/dragon-boot.git">
  <img src="https://img.shields.io/badge/Dragon%20Boot-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>	
  <a href="https://github.com/statikfintechllc/Mobile-Mirror.git">
  <img src="https://img.shields.io/badge/Mobile--Developer-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/Mobile-Mirror.git">
  <img src="https://img.shields.io/badge/Mobile--Mirror-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>	  
</div>
<div align="center">  
  <a href="https://github.com/statikfintechllc/AscendNet.git">
  <img src="https://img.shields.io/badge/AscendNet-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/AscendNet.git">
  <img src="https://img.shields.io/badge/P2P%20FOR%20LLM-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>
  <a
href="https://github.com/statikfintechllc/Ascend-Institute.git">
  <img src="https://img.shields.io/badge/SFTis-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/Ascend-Institute.git">
  <img src="https://img.shields.io/badge/Institute-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>
  <a href="https://github.com/statikfintechllc/GodCore.git">
  <img src="https://img.shields.io/badge/GodCore-black?style=for-the-badge&logo=dragon&logoColor=gold"/>
  <a href="https://github.com/statikfintechllc/GodCore.git">
  <img src="https://img.shields.io/badge/DualRouter-darkred?style=for-the-badge&logo=dragon&logoColor=gold"/>
  </a>	  
</div>
<div align="center">   
  <a href="https://github.com/statikfintechllc/WorkFlowRepo.git">
  <img src="https://img.shields.io/badge/WorkFlow-black?style=for-the-badge&logo=dragon&logoColor=gold"  alt="GremlinGPT Alpha"/>
  <a href="https://github.com/statikfintechllc/WorkFlowRepo.git">
  <img src="https://img.shields.io/badge/Repository-darkred?style=for-the-badge&logo=dragon&logoColor=gold" alt="GremlinGPT Alpha"/>
  </a>
  <a href="https://github.com/KDK-Grim/WorkFlowRepo-Mirror.git">
  <img src="https://img.shields.io/badge/Upgraded%20WorkFlow-black?style=for-the-badge&logo=dragon&logoColor=gold"  alt="GremlinGPT Alpha"/>
  <a href="https://github.com/KDK-Grim/WorkFlowRepo-Mirror.git">
  <img src="https://img.shields.io/badge/Repo%20Mirror-darkred?style=for-the-badge&logo=dragon&logoColor=gold" alt="GremlinGPT Alpha"/>
  </a>	  
</div>
<div align="center">

### Make your Own App:

*Just a funny, this app is cool though, I use it as a Game*

<a href="https://replit.com/github/statikfintechllc/Ascend%20Institute">
  <img src="https://replit.com/badge/github/statikfintechllc/Ascend%20Institute" alt="Run on Replit">
</a>

⚠️ Just Until GremlinGPT is booting, Soon ⚠️

</div>

---


<div align="center">
	
# Ascend Institute Traffic</h1>
 
 **The world’s first *RS-RACS***  
*Recursive, Self-Referential Autonomous Cognitive System*

</div> 
<div align="center">
  <a href="https://raw.githubusercontent.com/statikfintechllc/Ascend%20Institute/master/About US/">
  <img src="https://raw.githubusercontent.com/KDK-Grim/WorkFlowRepo-Mirror/master/docs/graph/traffic_graph.png" alt="Traffic Graph" />
  </a>
   <a href="https://github.com/statikfintechllc/WorkFlowRepo.git">
  <img src="https://img.shields.io/badge/Click%20to%20Install-Graph%20Workflow-darkred?labelColor=black" alt="GremlinGPT Alpha"/>
  </a>
</div>
  
<div align="center">
  <em>
Reset: After 7:00pm CST on First 2 Clones
  </em>
</div>

<div align="center">
  <a
href="https://github.com/statikfintechllc/AscendDocs_of_GovSeverance">
  <img src="https://img.shields.io/badge/Learn-black?style=for-the-badge&logo=dragon&logoColor=gold" alt="Why"/>
  </a>
  <a href="https://github.com/statikfintechllc/AscendDocs_of_GovSeverance">
  <img src="https://img.shields.io/badge/Why-darkred?style=for-the-badge&logo=dragon&logoColor=gold" alt="GremlinGPT"/>
  </a>
</div>

---

<details>
<summary>🤝 Open for Support 🤝</summary>
	
## Support This Project
*If you find this project valuable and want to support its continued development, consider sponsoring or contributing*

**Sponsor via:**

## Support Options

<div align="center">

<table>
  <thead>
    <tr>
      <th>Vendor</th>
      <th>Access</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><strong>GitHub</strong></td>
      <td>
        <a href="https://github.com/sponsors/statikfintechllc">
          <img src="https://img.shields.io/badge/GitHub-StatikFinTech_LLC-ff0000?style=for-the-badge&logo=github&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>Patreon</strong></td>
      <td>
        <a href="https://patreon.com/StatikFinTech_LLC">
          <img src="https://img.shields.io/badge/Patreon-StatikFinTech_LLC-ff0000?style=for-the-badge&logo=patreon&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>Ko-Fi</strong></td>
      <td>
        <a href="https://ko-fi.com/statikfintech_llc">
          <img src="https://img.shields.io/badge/Ko--Fi-statikfintech_llc-ff0000?style=for-the-badge&logo=kofi&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>CashApp</strong></td>
      <td>
        <a href="https://cash.app/$statikmoney8">
          <img src="https://img.shields.io/badge/CashApp-%24statikmoney8-ff0000?style=for-the-badge&logo=cashapp&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>PayPal</strong></td>
      <td>
        <a href="https://paypal.me/statikmoney8">
          <img src="https://img.shields.io/badge/PayPal-paypal.me%2Fstatikmoney8-ff0000?style=for-the-badge&logo=paypal&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>Bitcoin</strong></td>
      <td>
        <a href="https://www.blockchain.com/explorer/addresses/btc/bc1qarsr966ulmcs3mlcvae7p63v4j2y2vqrw74jl8">
          <img src="https://img.shields.io/badge/Bitcoin-bc1qarsr9...-ff0000?style=for-the-badge&logo=bitcoin&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>Ethereum</strong></td>
      <td>
        <a href="https://etherscan.io/address/0xC2db50A0fc6c95f36Af7171D8C41F6998184103F">
          <img src="https://img.shields.io/badge/Ethereum-0xC2db5...-ff0000?style=for-the-badge&logo=ethereum&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
    <tr>
      <td><strong>Chime</strong></td>
      <td>
        <a href="#">
          <img src="https://img.shields.io/badge/Chime-%24StatikSmokTM-ff0000?style=for-the-badge&logo=chime&logoColor=white&labelColor=000000" />
        </a>
      </td>
    </tr>
  </tbody>
</table>

</div>

<div align="center">
  <a href="https://maps.app.goo.gl/7tekD9g8i7FkWqxk8" target="_blank" rel="noopener">
  <img src="https://img.shields.io/badge/StatikFinTeck--LLC-Google%20Verified-ff0000?style=for-the-badge&logo=googlemaps&logoColor=white&labelColor=000000" />
  </a>
</div>

---

## Funding and Infrastructure Goals

> *GremlinGPT is growing. It learns(as do I while finishing GremlinGPT). It Launches(Soon).*
> *The project is reaching the limits of what a solo builder can finish without external support.*
> *The next stage—deployment, agent orchestration, and memory stability, an infrastructure investment to move into full time Development.*

### Funding Target is **$500,000**

**Funds will be used to secure:**

- A Trading account to be able to move into Full-Time Development
- Dedicated small GPU cluster (RTX 4090 × 2 minimum)  
- Persistent vector DB for others and hosted runtime servers for others who cant afford a system
- Ensuring Secure DevOps pipeline for offline + encrypted agents for the ones who cant secure thier own hardware.

## If You Are

- A founder with cloud real estate and idle GPUs  
- A data center operator who understands sovereign AI  
- An investor looking for a stake in recursive autonomy

🧬 Paging:  
[@elonmusk](https://x.com/elonmusk)  
[@openai](https://openai.com)  
[@deepmind](https://deepmind.com)  
[@mistralAI](https://mistral.ai)

*If you get it, run the loop.*

</details>
</details>

---

## 🧠 Features

- **MCP-Compatible:** Shows up in VS Code’s MCP list with metadata.
- **Simple API:** POST a URL, receive clean text in return.
- **CORS-Ready:** Built-in CORS support for cross-origin requests.
- **Logging:** Uses `loguru` to log all activity to rotating files.
- **Timeouts + Error Handling:** Gracefully deals with slow or weird sites.
- **Human UA Header:** Doesn’t look like a bot (unless you read the name).

---

## 🔧 Usage

1. Clone or drop this repo into your `.vscode/mcps/` or wherever your MCPs live.
2. Add `"gremlinScraper"` to `.mcp.json`.
3. Click “Start Server” in the VS Code MCP tab.
4. Or run it manually:
   ```bash
   pip install -r requirements.txt
   python server.py
   ```

---

## 📦 Endpoints & Examples

**1. POST /scrape**

- Fetch a single page’s visible text:

```bash
curl -X POST http://localhost:8742/scrape \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com"}'
```

- Response:

```json
{
  "text": "Example Domain\n\nThis domain is for use in illustrative examples in documents.\n..."
}
```

**2. POST /crawl**

- Recursively crawl same-domain links:

```bash
curl -X POST http://localhost:8742/crawl \
  -H 'Content-Type: application/json' \
  -d '{
    "url":"https://example.com",
    "max_pages":10,
    "max_depth":2,
    "concurrency":5
  }'
```

- Response: 

```json
{
  "https://example.com": "Example Domain\n\nThis domain is for use…",
  "https://example.com/about": "About Us\n\n…",
  "...": "…"
}
```

**3. POST /crawl-stream**

- Stream each page as soon as it’s fetched:

```bash
curl -N -X POST http://localhost:8742/crawl-stream \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","max_pages":5}'
```

- Response (NDJSON):

```json
{"url":"https://example.com","text":"Example Domain\n…"}
{"url":"https://example.com/link1","text":"Link One\n…"}
…
```

**4. GET /ping**

- Health check endpoint:

`curl http://localhost:8742/ping`

- Response:

`pong`

**5. GET /mcp/metadata**

- MCP discovery metadata:

`curl http://localhost:8742/mcp/metadata`

- Response:

```json
{
  "name":"Gremlin Web Scraper MCP",
  "description":"Scrapes and crawls text from URLs via HTTP endpoints…",
  "version":"0.0.1",
  "author":"StatikFinTech LLC",
  "tags":["scraping","crawl","MCP","runtime"],
  "endpoints":[…]
}
```

---

## 🗂 Metadata

> Name: Gremlin Web Scraper MCP  
> Author: StatikFinTech LLC  
> License: MIT  
> Tags: #scraping, #crawl, #runtime, #gremlin

---

## 🐾 Future Add-ons

- PDF / EPUB / Markdown parsing
- Selective DOM element filtering
- Scheduling/recurring crawl and scrap jobs
- Direct Memory injection to GremlinGPT core

---

“Split. Streamlined. Sovereign.”
StatikFinTech Systems • 2025

---

> [!CAUTION]
>
> “Your qualifications are impressive...”
> - Coder Hiring Team (2025 Rejection Letter)
>
> 🔱 "This isn't rejection. It's proof they don't know how to build what comes next.  
>
> Still building what they can’t classify." 🔱  -*StatikFinTech, LLC*

</div>
<div align="center">
  <a href="https://ko-fi.com/statikfintech_llc">
  <img  
	  src="https://img.shields.io/badge/Sponsor%20SFTi%20on%20KoFi-darkred?style=for-the-badge&logo=dragon&logoColor=gold" alt="Support"/>
  </a>
  <a href="https://patreon.com/StatikFinTech_LLC?utm_medium=unknown&utm_source=join_link&utm_campaign=creatorshare_creator&utm_content=copyLink">
  <img  
	  src="https://img.shields.io/badge/Sponsor%20SFTi%20on%20Patreon-black?style=for-the-badge&logo=dragon&logoColor=gold" alt="SFTi"/>
  </a>
</div>
<div align="center"> 

[![Sponsor 💪 statikfintechllc](https://img.shields.io/badge/Sponsor%20SFTi%20on%20GitHub-darkred?logo=github&style=for-the-badge)](https://github.com/sponsors/statikfintechllc)

</div>