PocketMCP
PocketMCP is a lightweight, local-first MCP (Model Context Protocol) server that automatically watches folders, chunks and embeds files locally using Transformers.js with MiniLM, stores vectors in SQLite + sqlite-vec, and exposes semantic search capabilities to VS Code and Cursor. Designed for small machines (Intel N100, 16GB RAM) with zero external dependencies after initial model download.
🌟 Features
- 🔍 Semantic Search: Find content by meaning, not just keywords
- 📁 Auto-Ingestion: Watches folders and automatically processes new/changed files
- ⚡ Local-First: Runs completely offline after initial model download
- 🗄️ SQLite Storage: Fast, reliable vector storage with sqlite-vec extension
- 🔧 MCP Integration: Native support for VS Code and Cursor via MCP protocol
- 🌐 Web Interface: Built-in web tester for validation and manual testing
- 💾 Efficient: Designed for resource-constrained environments
- 🔄 Real-time: Debounced file watching with smart concurrency limits
📋 Table of Contents
🚀 Quick Start
1. Installation
# Clone or download the project
cd PocketMCP
# Install dependencies
pnpm install
# Setup environment
pnpm setup
# Or manually: cp .env.sample .env
2. Configuration
Edit .env file:
# SQLite database path
SQLITE_PATH=./data/index.db
# Directory to watch for file changes (optional)
WATCH_DIR=./kb
# Embedding model (default is recommended)
MODEL_ID=Xenova/all-MiniLM-L6-v2
# Chunking configuration
CHUNK_SIZE=1000
CHUNK_OVERLAP=120
3. Create Content Directory
# Create directory for your documents
mkdir -p kb
# Add some markdown or text files
echo "# My First Document" > kb/test.md
echo "This is a sample document for testing PocketMCP." >> kb/test.md
4. Start the Server
Option A: MCP Server Only
# Development mode (recommended for testing)
pnpm dev:mcp
# With file watching enabled
pnpm dev:mcp:watch
# Production mode
pnpm build && pnpm start
Option B: MCP Server + Web Tester
# Start both web interface and API server
pnpm dev
# Or start individual components
pnpm --filter @pocketmcp/api dev # API server only
pnpm --filter @pocketmcp/web dev # Web interface only
On first run, the server will download the MiniLM model (~100MB) and then process any files in your watch directory.
🌐 Web Tester
PocketMCP includes a comprehensive web interface for testing and validation.
Access Points
- Web Interface: http://127.0.0.1:5173
- API Server: http://127.0.0.1:5174
- Health Check: http://127.0.0.1:5174/health
Features
📊 Database Diagnostics Panel
- Real-time database status monitoring
- Table counts and vector dimensions
- SQLite WAL mode verification
- Error detection and reporting
- One-click smoke testing
🔍 Search Panel
- Interactive semantic search testing
- LIKE vs Vector search modes
- Configurable result count (top-K)
- Detailed result inspection
- Performance metrics (response time)
📄 Documents Panel
- Browse all indexed documents
- Pagination support
- Document metadata display
- Creation and update timestamps
🔎 Chunk Viewer
- Detailed chunk inspection modal
- Full text content display
- Metadata and offset information
- Copy-to-clipboard functionality
API Endpoints
| Endpoint | Method | Description |
|---|
/health | GET | Server health check |
/api/db/diag | GET | Database diagnostics |
/api/search | POST | Semantic search |
/api/chunk/:id | GET | Get specific chunk |
/api/docs | GET | List documents |
Example API Usage
Search Documents:
curl -X POST http://127.0.0.1:5174/api/search \
-H "Content-Type: application/json" \
-d '{"query": "machine learning", "top_k": 5, "mode": "like"}'
Get Diagnostics:
curl http://127.0.0.1:5174/api/db/diag | jq .
🔧 MCP Client Integration
Cursor Integration
- Open Cursor Settings → MCP
- Add a new server with these settings:
{
"command": "pnpm",
"args": ["dev:mcp"],
"cwd": "/path/to/PocketMCP",
"env": {
"SQLITE_PATH": "./data/index.db",
"WATCH_DIR": "./kb",
"MODEL_ID": "Xenova/all-MiniLM-L6-v2"
}
}
VS Code Integration
For VS Code clients that support MCP, add to your settings:
{
"mcpServers": {
"pocketmcp": {
"command": "pnpm",
"args": ["dev:mcp"],
"cwd": "/path/to/PocketMCP",
"env": {
"SQLITE_PATH": "./data/index.db",
"WATCH_DIR": "./kb",
"MODEL_ID": "Xenova/all-MiniLM-L6-v2"
}
}
}
}
Alternative: Direct Node Execution
{
"command": "node",
"args": ["dist/server.js"],
"cwd": "/path/to/PocketMCP",
"env": {
"SQLITE_PATH": "./data/index.db",
"WATCH_DIR": "./kb"
}
}
📚 API Reference
search
Search for similar content using semantic search.
{
"query": "machine learning algorithms",
"top_k": 5,
"filter": {
"doc_ids": ["doc_123", "doc_456"]
}
}
upsert_documents
Insert or update documents programmatically.
{
"docs": [
{
"text": "Your document content here...",
"external_id": "my_doc_1",
"title": "Important Notes",
"metadata": {}
}
]
}
delete_documents
Delete documents by ID.
{
"doc_ids": ["doc_123"],
"external_ids": ["my_doc_1"]
}
list_documents
List all documents with pagination.
{
"page": {
"limit": 20
}
}
MCP Resources
PocketMCP provides resource URIs for accessing specific chunks:
- Format:
mcp+doc://<doc_id>#<chunk_id> - Returns: Complete chunk data including text, offsets, and metadata
⚙️ Configuration
Environment Variables
| Variable | Default | Description |
|---|
SQLITE_PATH | ./data/index.db | Path to SQLite database file |
WATCH_DIR | (none) | Directory to watch for file changes |
MODEL_ID | Xenova/all-MiniLM-L6-v2 | Hugging Face model for embeddings |
CHUNK_SIZE | 1000 | Target chunk size in characters |
CHUNK_OVERLAP | 120 | Overlap between chunks in characters |
NODE_ENV | development | Environment mode |
VERBOSE_LOGGING | false | Enable detailed logs |
DEBUG_DOTENV | false | Enable dotenv debug output |
API_PORT | 5174 | Web API server port |
API_BIND | 127.0.0.1 | API server bind address |
Available Scripts
| Script | Description |
|---|
pnpm dev | Start web interface + API server |
pnpm dev:mcp | Start MCP server only |
pnpm dev:mcp:watch | Start MCP server with file watching |
pnpm dev:mcp:verbose | Start MCP server with verbose logging |
pnpm build | Build all components |
pnpm start | Start production MCP server |
pnpm setup | Create .env from template |
pnpm clean | Clean build artifacts and database |
Watch Directory Notes
WATCH_DIR is optional - if not set, only manual document upserts work- Choose any directory -
./kb is just a convention, use whatever makes sense - Supported files:
.md, .txt by default (configurable in code) - File filtering: Automatically ignores temp files,
.DS_Store, node_modules, etc. - Nested directories: Recursively watches all subdirectories
Supported File Types
Currently supports:
- Markdown (
.md) - Plain text (
.txt)
To add more file types, modify the supportedExtensions in the FileIngestManager configuration.
🛠️ Development
Project Structure
PocketMCP/ # Monorepo root
├── package.json # Workspace configuration
├── pnpm-workspace.yaml # pnpm workspace setup
├── .env # Environment variables
├── .env.sample # Environment template
├── apps/
│ ├── api/ # Express API server
│ │ ├── src/
│ │ │ ├── server.ts # Main API server
│ │ │ └── db.ts # Database manager
│ │ └── package.json
│ └── web/ # React + Vite frontend
│ ├── src/
│ │ ├── App.tsx # Main app component
│ │ ├── store.ts # Zustand state management
│ │ ├── api.ts # API client
│ │ └── components/ # UI components
│ └── package.json
├── src/ # Original MCP server
│ ├── server.ts # MCP server and main entry point
│ ├── db.ts # SQLite database with sqlite-vec
│ ├── embeddings.ts # Transformers.js embedding pipeline
│ ├── chunker.ts # Text chunking with sentence awareness
│ ├── ingest.ts # Generic document ingestion
│ ├── file-ingest.ts # File-specific ingestion logic
│ └── watcher.ts # File system watcher with debouncing
├── data/ # SQLite database storage
├── kb/ # Default watch directory (configurable)
└── README.md
Development Commands
# Install dependencies
pnpm install
# Run MCP server in development mode (hot reload)
pnpm dev:mcp
# Run web tester in development mode
pnpm dev
# Build for production
pnpm build
# Run production build
pnpm start
# Run with custom environment
WATCH_DIR=./my-docs CHUNK_SIZE=500 pnpm dev:mcp
Testing
# Test web tester functionality
./test-web-tester.sh
# Manual API testing
curl http://127.0.0.1:5174/health
curl http://127.0.0.1:5174/api/db/diag
🏗️ Architecture
- Sweet spot: 10K-100K chunks on modest hardware
- Query latency: Sub-100ms for
top_k <= 10 on typical corpora - Memory usage: ~100MB for model + minimal overhead per document
- Concurrency: Limited to 3 simultaneous file operations by default
- File size limit: 50MB per file (configurable)
🔧 Troubleshooting
Model Download Issues
If the embedding model fails to download:
- Check internet connection for initial download
- Model cache location:
~/.cache/huggingface/transformers/ - Clear cache and retry if needed
SQLite Extension Issues
If sqlite-vec fails to load:
- Ensure
sqlite-vec npm package is installed - Check that your system supports the required SQLite version
- The system automatically falls back to regular SQLite tables if vec0 virtual tables fail
File Watching Issues
- Files not being detected: Check file extensions and ignore patterns
- High CPU usage: Increase debounce time with larger
debounceMs values - Permission errors: Ensure read/write access to watch and data directories
Web Interface Issues
- API not accessible: Ensure API server is running on port 5174
- Database not found: Check
SQLITE_PATH environment variable - CORS errors: API server includes CORS headers for local development
Memory Issues
- Reduce
CHUNK_SIZE for lower memory usage - Process fewer files simultaneously by reducing
maxConcurrency - Consider using a smaller embedding model (though this requires code changes)
Common Error Messages
"Too many parameter values were provided"
- This was a known issue with sqlite-vec virtual tables, now fixed with automatic fallback
"Failed to load sqlite-vec extension"
- System automatically falls back to regular SQLite tables with JSON embeddings
"Database file does not exist"
- Run the MCP server first to create the database, or check the
SQLITE_PATH
📄 License
MIT License - see LICENSE file for details.
🤝 Contributing
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests if applicable
- Submit a pull request
🙏 Acknowledgments
- sqlite-vec for fast vector similarity search
- Transformers.js for local embedding generation
- Model Context Protocol for standardized tool integration
- Hugging Face for the MiniLM model
- React + Vite for the modern web interface
- TailwindCSS for beautiful, responsive styling