Skip to main content
Glama
utaschulz1

mcp-crawl4ai2vectordb

by utaschulz1

mcp-crawl4ai2vectordb

MCP server for crawling web documentation and storing it in a Supabase vector database. Credits: craw4ai and Cole Medin's mcp server.

Responsibility: ingestion only — crawl, chunk, embed, store. No search, no reranking, no Neo4j.

Tools

Tool

Description

crawl_single_page

Crawl one URL and store its content

smart_crawl_url

Auto-detect URL type (sitemap / txt / webpage) and crawl accordingly

get_available_sources

List all sources stored in the database

delete_source

Delete a source and all its content from the database

Related MCP server: web-docs-mcp

Setup

cp .env.example .env
# fill in OPENAI_API_KEY, MODEL_CHOICE, SUPABASE_URL, SUPABASE_SERVICE_KEY
uv sync

Running

stdio (recommended — Claude Code manages the process):

Add to your MCP config:

{
  "mcpServers": {
    "crawl4ai2vectordb": {
      "type": "stdio",
      "command": "uv",
      "args": ["run", "python", "-u", "src/server.py"],
      "cwd": "/path/to/mcp_crawl4ai2vectordb"
    }
  }
}

or

{
  "mcpServers": {
    "crawl4ai2vectordb": {
      "type": "sse",
      "url": "http://localhost:8051/sse"
    }
  }
}

SSE (manual startup, for multi-client use):

TRANSPORT=sse uv run python -u src/server.py

Optional features

Env var

Default

Effect

USE_CONTEXTUAL_EMBEDDINGS

false

Prepend LLM-generated context to each chunk before embedding (improves retrieval accuracy, costs more)

USE_AGENTIC_RAG

false

Extract code blocks and store them separately in code_examples table

F
license - not found
-
quality - not tested
D
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/utaschulz1/mcp_crawl4ai2vectordb'

If you have feedback or need assistance with the MCP directory API, please join our Discord server