mcp-server-bm25-code-search

English | 日本語

Fast, low-token BM25 local code search plugin & MCP server backed by SQLite FTS5 for AI coding agents (VS Code, Cursor, GitHub Copilot, ChatGPT & Codex, Kiro, Hermes Agent, OpenClaw, Grok Bot, NanoClaw, etc.), fully compliant with the Agent Plugins specification.

✨ Features

• 📦 Zero External Dependencies (Python Standard Library Only)
  Built entirely on sqlite3 (FTS5) and the Python standard library. Runs out-of-the-box without requiring third-party package installations (pip install).

• 🧩 Agent Plugins (v1.0.0) Compliant
  Conforms to the Agent Plugins standard. In supported clients (VS Code, Cursor, GitHub Copilot, ChatGPT & Codex, Kiro, Hermes Agent, OpenClaw, Grok Bot, NanoClaw), simply pointing to this repository automatically discovers and loads both the MCP server (mcp.json) and the search guidance skill (skills/) with zero configuration.

• 🔤 Code Identifier & Japanese/CJK Hybrid Tokenization
  Pre-processes code identifiers with subword splitting for getUserProfile (camelCase) and session_token (snake_case), combined with Python-side CJK 2-gram (bigram) tokenization for technical documentation and comments. Automatically applied to both FTS5 indexing and search queries.

• 📁 File Path Boost (3.0x)
  Leverages FTS5 column weighting bm25(code_fts, 3.0, 1.0) to weight file path matches 3.0x higher than file content matches, pinpointing target files in fewer search iterations.

• ⚡ Fast Incremental Indexing & Git Worktree Isolation
  Strict .gitignore compliance using git ls-files with ultra-fast incremental updates (0.1–0.5s during standard editing) via git diff / HEAD hash tracking. The index database .bm25_index.db is stored locally within the worktree and automatically ignored by .gitignore.

• 🔌 MCP 2026-07-28 & Hermes Native Support

  • MCP Native: Stateless stdio JSON-RPC server adhering to the MCP 2026-07-28 specification, with deterministic tool sorting for prompt cache optimization.

  • Hermes Agent: Includes a lightweight Function Calling adapter layer (hermes_adapter.py) for environments without native MCP support.

• 🛡️ Context Overflow Protection & Fallback Guidance
  Safe byte-length truncation (--max-bytes) preserving UTF-8 multi-byte character boundaries. Returns structured fallback messages prompting agents to switch to grep/glob when zero results are found.

Related MCP server: lynx-mcp

📁 Directory Structure

mcp-server-bm25-code-search/
├── plugin.json            # Agent Plugins v1.0.0 manifest
├── mcp.json               # Agent Plugins v1.0.0 MCP configuration
├── skills/                # Agent Skills (agent search guidelines & prompt)
│   └── bm25-search/
│       └── SKILL.md
├── bm25_search/
│   ├── db.py              # SQLite FTS5 v2 schema (chunks / code_fts / triggers)
│   ├── tokenizer.py       # Pre-tokenizer (camelCase / snake_case / CJK 2-gram)
│   ├── indexer.py         # Indexer (git ls-files, 80/20 chunking, incremental sync)
│   ├── search.py          # Search engine & CLI interface
│   ├── mcp_server.py      # MCP 2026-07-28 stateless stdio server
│   └── hermes_adapter.py  # Function Calling adapter for Hermes Agent
├── bin/
│   └── cli.js             # Node.js CLI / npx runner wrapper
├── docs/
│   ├── specification.md   # Detailed specification
│   └── plans/             # Design documents
└── tests/                 # pytest test suite

🚀 Getting Started

1. Installation via Agent Plugins (Recommended / Zero-Config)

In Agent Plugins compatible clients (VS Code, Cursor, GitHub Copilot, ChatGPT & Codex, Kiro, Hermes Agent, OpenClaw, Grok Bot, NanoClaw), simply adding this repository directory or loading it as a plugin automatically recognizes both the MCP server (mcp.json) and the search guidance skill (skills/):

Official setup documentation for supported clients:

• VS Code: Agent Plugins in VS Code

• Cursor: Cursor Plugins

• GitHub Copilot: Copilot Agent Plugins

• ChatGPT & Codex: OpenAI Plugin Developers

• Kiro: Kiro Powers

• Hermes Agent: Hermes Portable Plugins

• OpenClaw: OpenClaw Plugin Bundles

• Grok Bot: Grok Bot Automations

• NanoClaw: NanoClaw Templates

2. Running Search via CLI

python bm25_search/search.py "<search query>" --top-k 5 --format markdown --max-bytes 4000

Key Options:

• <query>: Search query (supports Japanese, CJK, camelCase, snake_case)

• --top-k: Maximum number of search results to return (default: 5)

• --format: Output format, markdown or json (default: markdown)

• --max-bytes: Maximum output bytes; safely truncated preserving multibyte characters (default: 4000)

• --mode: Token conjunction mode, OR or AND (default: OR)

• --db: Path to SQLite index DB file (default: .bm25_index.db)

3. Running as a Standalone MCP Server (uvx / npx / Manual)

The server communicates via Stdio and automatically indexes the project codebase. You can launch it instantly with uvx or npx. If arguments are omitted, the server automatically detects the current working directory as the project root and synchronizes the incremental index (.bm25_index.db).

CLI Options

💡 Automatic Project Root Detection with --db:
When specifying --db <path> (e.g. --db /path/to/project/.bm25_index.db) without an explicit --root, the parent directory of the DB file is automatically detected as the project root.
This allows global or shared agent configurations to easily target specific projects while maintaining seamless Auto Sync and search functionality (specifying --root explicitly will take precedence).

① Using uvx (uv / Python)

{
  "mcpServers": {
    "bm25-code-search": {
      "command": "uvx",
      "args": ["mcp-server-bm25-code-search"],
      "alwaysAllow": ["search"]
    }
  }
}

② Using npx (Node.js / npm)

{
  "mcpServers": {
    "bm25-code-search": {
      "command": "npx",
      "args": ["-y", "mcp-server-bm25-code-search"],
      "alwaysAllow": ["search"]
    }
  }
}

③ Using Local Python Directly

{
  "mcpServers": {
    "bm25-code-search": {
      "command": "python",
      "args": [
        "/path/to/mcp-server-bm25-code-search/bm25_search/mcp_server.py",
        "--stdio"
      ],
      "alwaysAllow": [
        "search"
      ]
    }
  }
}

④ Specifying a Target Project DB via --db

{
  "mcpServers": {
    "bm25-code-search": {
      "command": "uvx",
      "args": [
        "mcp-server-bm25-code-search",
        "--db",
        "/path/to/my-project/.bm25_index.db"
      ],
      "alwaysAllow": ["search"]
    }
  }
}

Note: The parent directory /path/to/my-project is automatically recognized as the project root for indexing and synchronization.

💡 Agent Instruction Guideline (AGENTS.md / CLAUDE.md)

To prevent AI agents from repeatedly spamming grep and wasting context tokens, adding the following instruction to your project's AGENTS.md, CLAUDE.md, or system prompt is strongly recommended:

## Code Search Policy
- When exploring code or investigating features across the codebase, always prioritize the MCP tool `search` (BM25 Code Search) first.
- Only fall back to `grep_search` or `glob` if `search` returns zero results or when exact literal matches for a specific symbol are required.

4. Setup in Claude Code (Manual Configuration)

Because Claude Code does not natively support the Agent Plugins standard, automatic manifest discovery (plugin.json / mcp.json) is not available. Configure it using Claude Code's MCP server registration:

💡 Note on Deferred Tools in Claude Code:
Claude Code treats MCP tools as "Deferred Tools" to conserve context. The agent will load the tool schema before execution. Adding the instruction above to CLAUDE.md or placing the skill in .claude/skills/bm25-search/SKILL.md ensures consistent tool selection.

Method A: claude mcp add CLI Command (Recommended)

Run one of the following commands in your project root (--scope project generates a .mcp.json file to commit to git; --scope user registers it globally):

# Using uvx (uv / Python)
claude mcp add bm25-code-search --scope project -- uvx mcp-server-bm25-code-search

# Using npx (Node.js / npm)
claude mcp add bm25-code-search --scope project -- npx -y mcp-server-bm25-code-search

# Using local Python directly
claude mcp add bm25-code-search --scope project -e PYTHONUTF8=1 -- python /path/to/mcp-server-bm25-code-search/bm25_search/mcp_server.py --stdio

Verify the configuration with claude mcp list or the /mcp command inside a Claude Code session.

Method B: Direct .mcp.json File

Place a .mcp.json file in your project root (identical schema to the MCP configuration above):

{
  "mcpServers": {
    "bm25-code-search": {
      "command": "uvx",
      "args": ["mcp-server-bm25-code-search"]
    }
  }
}

5. Using Hermes Agent Adapter

For Hermes Agent environments without MCP support, use the bm25_search.hermes_adapter module:

from bm25_search.hermes_adapter import hermes_function_schema, run_hermes_tool

# Get Hermes Tool Schema
schema = hermes_function_schema()

# Execute Function Call from Hermes
response = run_hermes_tool({
    "name": "bm25_search",
    "arguments": {
        "query": "getUserProfile",
        "top_k": 5
    }
})

🧪 Running Tests

Run the unit and integration test suite using pytest:

pytest tests/

📄 Documentation

• Specification (docs/specification.md)

• Design Specification (docs/plans/bm25-multi-agent-search-skill-design.md)

📚 References & Links

• Agent Plugins Specification: Agent Plugins Specification (agentplugins/agent-plugins-spec) / agent-plugins.org

• Agent Skills Specification: Agent Skills Specification

• Paper: Wang et al., "BM25 Wins at Scale: Evaluating Agentic Search over Enterprise Corpora" (2026)
  https://arxiv.org/abs/2607.26497

• Article: Hidetoshi Sudo (KnowledgeSense, Inc.), "Using BM25 to reduce Codex token consumption by 30%" (Zenn, 2026)
  https://zenn.dev/knowledgesense/articles/9e55a3bb67729c

⚖️ License

This project is licensed under the MIT License.