# Pre-commit Setup Guide
This guide helps you set up and use pre-commit hooks for the crawl4ai-mcp project.
## Installation
### 1. Install pre-commit tool
Using UV (recommended):
```bash
uv add --dev pre-commit
```
Or using pip:
```bash
pip install pre-commit
```
### 2. Install the hooks
```bash
# Install hooks for this repository
pre-commit install
# Also install for commit-msg hooks (conventional commits)
pre-commit install --hook-type commit-msg
# Install for pre-push hooks
pre-commit install --hook-type pre-push
```
## Usage
### Automatic execution
Once installed, hooks will run automatically:
- **Pre-commit hooks**: Run before each commit
- **Commit-msg hooks**: Validate commit message format
- **Pre-push hooks**: Run before pushing to remote
### Manual execution
Run all hooks on all files:
```bash
pre-commit run --all-files
```
Run specific hook:
```bash
pre-commit run ruff --all-files
pre-commit run bandit --all-files
```
Run only on staged files:
```bash
pre-commit run
```
### Skip hooks (when needed)
```bash
# Skip all pre-commit hooks
SKIP=pre-commit git commit -m "emergency fix"
# Skip specific hooks
SKIP=ruff,mypy git commit -m "work in progress"
# Skip pre-push hooks
git push --no-verify
```
## Hook Categories
### 🏠 Housekeeping Hooks
- **trailing-whitespace**: Removes trailing whitespace
- **end-of-file-fixer**: Ensures files end with newline
- **check-json/yaml/toml**: Validates file formats
- **check-merge-conflict**: Prevents committing merge conflicts
### 🐍 Python Code Quality
- **ruff lint**: Fast linting (replaces flake8, isort, etc.)
- **ruff format**: Fast formatting (replaces black)
- **mypy**: Optional type checking
- **pydocstyle**: Docstring conventions
### 🔒 Security & Safety
- **bandit**: Security vulnerability scanner
- **detect-secrets**: Prevents committing secrets
- **hadolint**: Dockerfile linting
### ✅ Development Workflow
- **conventional-pre-commit**: Enforces conventional commit messages
- **uv-lock-check**: Ensures uv.lock is up to date
- **pytest-check**: Runs quick tests before push
- **docker-build-test**: Validates Docker build before push
## Commit Message Format
This project uses [Conventional Commits](https://www.conventionalcommits.org/):
```
<type>[optional scope]: <description>
[optional body]
[optional footer(s)]
```
### Examples
```bash
git commit -m "feat(mcp): add new crawling endpoint"
git commit -m "fix(database): resolve connection timeout issue"
git commit -m "docs: update API documentation"
git commit -m "test: add integration tests for Qdrant"
git commit -m "refactor(utils): simplify error handling"
```
### Valid types
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation changes
- `test`: Adding or updating tests
- `refactor`: Code refactoring
- `perf`: Performance improvements
- `ci`: CI/CD changes
- `build`: Build system changes
- `chore`: Maintenance tasks
## Configuration Files
### Main configuration
- `.pre-commit-config.yaml`: Hook definitions and settings
- `pyproject.toml`: Tool configurations (ruff, pytest, mypy, etc.)
### Supporting files
- `.secrets.baseline`: Baseline for detect-secrets
- `.gitignore`: Files to ignore in version control
## Troubleshooting
### Common issues
**1. Hook installation fails:**
```bash
# Update pre-commit
pip install --upgrade pre-commit
# Reinstall hooks
pre-commit clean
pre-commit install
```
**2. Ruff/mypy not found:**
```bash
# Ensure UV environment is active
uv sync
uv run pre-commit run --all-files
```
**3. Docker build fails:**
```bash
# Test Docker build manually
docker build --target development -t crawl4ai-mcp:test .
```
**4. False positive in secret detection:**
```bash
# Update baseline
uv run detect-secrets scan --baseline .secrets.baseline --update
# Or add inline comment
password = "fake_password_for_testing" # pragma: allowlist secret
```
### Performance optimization
**Skip slow hooks in CI:**
The configuration automatically skips slow hooks (pytest, docker-build, mypy) in pre-commit.ci.
**Run specific hooks:**
```bash
# Only run fast formatting/linting
pre-commit run ruff ruff-format
# Only run security checks
pre-commit run bandit detect-secrets
```
## Integration with IDEs
### VS Code
Add to `.vscode/settings.json`:
```json
{
"python.linting.enabled": true,
"python.linting.ruffEnabled": true,
"python.formatting.provider": "none",
"[python]": {
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true
}
}
}
```
### PyCharm
1. Install Ruff plugin
2. Configure external tool for pre-commit
3. Set up code style to match ruff configuration
## Best Practices
1. **Run hooks locally**: Don't rely only on CI/CD
2. **Fix issues incrementally**: Address linting issues as you go
3. **Use descriptive commit messages**: Follow conventional commit format
4. **Test before pushing**: Let pre-push hooks catch issues early
5. **Keep configuration updated**: Regularly update hook versions
## Getting Help
- Pre-commit documentation: <https://pre-commit.com/>
- Ruff documentation: <https://docs.astral.sh/ruff/>
- Conventional Commits: <https://www.conventionalcommits.org/>
For project-specific issues, check the project's troubleshooting documentation or open an issue.
