# Cline Rules for MCP Web Docs
This is a TypeScript MCP (Model Context Protocol) server project for crawling and indexing web documentation.
## Quick Reference
For comprehensive guidelines, see **AGENTS.md** in the project root.
## Project Overview
MCP Web Docs is a self-hosted MCP server that:
- Crawls and indexes documentation from any website
- Provides hybrid search (full-text + semantic/vector)
- Supports authentication for protected documentation sites
- Uses local embeddings (FastEmbed) for privacy
## Essential Rules
### Code Style
- TypeScript with strict mode enabled
- **Always use `.js` extensions in imports** (ESM requirement)
- Follow existing patterns in the codebase
- Run `npm run lint` and `npm run prettier` before committing
### Testing
- Tests are co-located: `*.test.ts` next to source files
- Use Vitest globals: `describe`, `it`, `expect`, `vi`
- Mock embeddings: `createMockEmbeddings()` from `src/__mocks__/embeddings.ts`
- Use `vi.hoisted()` for mocks configured before imports
### Critical Constraints
- **Never log to stdout** - MCP servers use stdio for JSON-RPC
- Use logger from `src/util/logger.ts` (writes to stderr)
- Validate inputs with Zod schemas from `src/util/security.ts`
- Use `validatePublicUrl()` for SSRF protection
### Key Commands
```bash
npm test # Run tests
npm run build # Build TypeScript
npm run lint # Run ESLint
npm run prettier # Format code
npm run test:coverage # Test with coverage
```
### File Structure
```
src/
├── index.ts # Main MCP server (WebDocsServer)
├── crawler/ # Web crawling (Playwright/Crawlee)
├── processor/ # Content processing
├── storage/ # SQLite + LanceDB
├── embeddings/ # FastEmbed vectors
├── indexing/ # Status tracking
└── util/ # Logger, security, helpers
```
### When Making Changes
1. Check existing patterns in the codebase first
2. Add Zod schema for new inputs in `src/util/security.ts`
3. Write tests for new functionality
4. Run `npm test` before completing
### Commit Conventions
This project uses **semantic-release** for automated versioning. Follow [Conventional Commits](https://www.conventionalcommits.org/):
- `feat`: New feature → Minor version bump
- `fix`: Bug fix → Patch version bump
- `chore`, `refactor`, `docs`, `test`: No release
**For feature PRs**: Use `feat` for the main commit, `chore`/`refactor` for follow-up fixes within the same PR (keeps release notes focused on the feature, not every small fix).
Commit bodies are included in release notes—use `-` bullet points for sub-features.
### Mocking Patterns
```typescript
// Hoisted mocks
const { mockFn } = vi.hoisted(() => ({ mockFn: vi.fn() }));
vi.mock('module', () => ({ default: mockFn }));
// Mock embeddings
import { createMockEmbeddings } from '../__mocks__/embeddings.js';
// Temp directories for tests
beforeEach(async () => {
tempDir = await mkdtemp(join(tmpdir(), 'test-'));
});
```
See **AGENTS.md** for detailed documentation.
