Trove
Provides tools to discover repositories from GitHub accounts/orgs, clone/fetch them, and track changed files via the GitHub API, with rate limiting, pagination, and retries.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Trovefind the authentication flow in our code and docs"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Trove — Knowledge Engine for AI Development
A local-first knowledge engine with hybrid search, source connectors, plugin extensibility, and MCP integration for Claude Code. Trove discovers, indexes, and searches your documents, code, and web content so AI tools always have the right context.
Features
Search & Retrieval
Hybrid Search — BM25 keyword + vector semantic search with cross-encoder re-ranking
RRF Merge Strategy — Reciprocal Rank Fusion as alternative to weighted merging
MMR Diversity — Maximal Marginal Relevance reduces redundant results
HyDE — Hypothetical Document Embeddings for better vector search on vague queries
Multi-Query / RAG-Fusion — Generate query variants and merge results via RRF
Search Quality Indicators — Scores, timing, source diversity, and strategy metadata
Configurable Embeddings — Choose embedding model per client (MiniLM, BGE, E5)
Stemming & Synonyms — Porter/Snowball stemmer and domain-specific synonyms
Agentic/Recursive Search — Automatically refines queries when results are insufficient
Reference Resolution — Detects "see Appendix X" and fetches referenced content
Query Contextualization — Understands follow-up questions in conversation context
Search Result Caching — LRU + TTL cache with filter-aware keys
Score Normalization — MinMax, Z-score, or raw normalization for merge strategies
Source Connectors
GitHub Connector — Discover repos from GitHub accounts/orgs, clone/fetch, track changed files. API client with rate limiting, pagination, and retries. Tokens via environment variables only.
Web Crawler — BFS crawl with scope control, HTML-to-Markdown conversion (markdownify + trafilatura), incremental cache via ETag/content hash.
Auto-sync — Sources sync before ingestion when
sync_on_ingestis configured. Manual sync via CLI or MCP.Incremental Updates — Only changed files are re-crawled and reindexed.
Plugin System
6 Extension Points — Custom embedding models, chunkers, parsers, rerankers, LLM providers, and merge strategies
Plugin Registry — Register and lookup plugins by name with duplicate detection
Auto-Discovery — Drop a
.pyfile or package inplugins/and it's loaded on startupBuilt-in Refactoring — LLM providers and rerankers already implement plugin interfaces
Ingestion & Indexing
Producer-Consumer Pipeline — Parallel file parsing (ThreadPool) → GPU batch embedding → progressive ChromaDB indexing
Fast File Collection —
git ls-filesfor git repos (near-instant),os.scandirfallback, parallel change detectionCtrl+C Safe — Each GPU batch is indexed immediately; interrupt loses at most one batch, resume skips already-indexed files
Code Indexing — Tree-sitter AST parsing for Python, TypeScript, JavaScript, PHP, HTML, CSS, YAML, JSON, Gherkin, Dart
Semantic Chunking — Split by embedding similarity drops (alternative to title-based)
Recursive Chunking — Priority-ordered separators with fallback
Contextual Embeddings — Prepend document context to chunks before embedding
Chunk Quality Validation — Min-length, alphanumeric, and content-hash deduplication
Response Generation
Inline Citations — LLM responses cite sources with
[1],[2]referencesConfigurable Reranker — Local CrossEncoder or API-based (Cohere-compatible) with automatic fallback
Token Budgeting — Separate context budgets for document, code, and reference chunks
Context Pre-Filtering — Drop low-scoring chunks before LLM context
Knowledge & Agents
Knowledge Graph — Entity/relationship extraction, graph-augmented search (local/global/hybrid/mix modes)
Persistent Memory — SQLite-backed store with vector search, TTL expiration, per-client or global
Workflow DAGs — Multi-step task pipelines with dependency ordering and auto-status tracking
Agent Templates — YAML-defined personas (code-reviewer, security-reviewer, etc.)
Local Agent Runner — Search for context, inject system prompt, generate via LM Studio
Prompt Discovery — 10k+ indexed templates + live prompts.chat API search
Related MCP server: coderag
Stack
Layer | Library | License |
Document Processing | Unstructured.io | Apache 2.0 |
Keyword Search | rank-bm25 | Apache 2.0 |
Stemming/NLP | NLTK | Apache 2.0 |
Vector Search | ChromaDB + sentence-transformers | Apache 2.0 |
Re-ranking | sentence-transformers (cross-encoder) or API | Apache 2.0 |
Knowledge Graph | NetworkX | BSD |
GitHub API | httpx | BSD |
HTML → Markdown | markdownify + trafilatura | MIT / Apache 2.0 |
LLM (local) | LM Studio via OpenAI-compatible API | - |
LLM (cloud) | Claude via Anthropic client | - |
MCP Server | mcp | MIT |
100% local capable. Zero API costs (with LM Studio). Licensed under BSL 1.1 (converts to MIT).
Quick Start
Requires Python 3.10+
# Get the code and install dependencies
git clone https://github.com/tomgould/trove.git
cd trove
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
# Create a new client (interactive wizard)
python scripts/create_client.py
# Option A: Add local documents
cp /path/to/docs/* clients/myproject/documents/
# Option B: Connect external sources
python scripts/sources.py connect github --client myproject --owner myorg
python scripts/sources.py crawl --client myproject --url https://docs.example.com
# Ingest and index
python scripts/ingest.py --client myproject
# Use via MCP server (see MCP Server section below)
python scripts/mcp_server.pyCreating a Client
The easiest way to create a new client is with the interactive wizard:
python scripts/create_client.pyThe wizard will prompt you for:
Client name - Display name (e.g., "Acme Corp RAG")
Machine name - Directory name (auto-generated from display name)
Description - Optional description
Embedding model - MiniLM (fast) or BGE (better quality)
LLM settings - Provider (lmstudio/claude), context limits, max tokens
Search settings - BM25/vector weights, top_k, rerank settings (optional)
Tokenization - Stemmer selection (Porter, Snowball, or none)
Reference domain - Compliance, legal, technical, or academic patterns
Audit settings - Enable/disable logging, log level
System prompt - Custom prompt or use default
After completion, it creates the full directory structure and config files:
clients/your_client/
├── client.yaml # Configuration
├── documents/ # Add your source files here
├── index/ # Search index (created on ingest)
└── prompts/ # System prompt
└── system.txtManual Setup
Alternatively, create a client manually:
mkdir -p clients/myproject/{documents,index,prompts,data}Then create clients/myproject/client.yaml (see Client Configuration below).
How It Works
1. Document Ingestion
PDF/DOCX/XLSX/CSV/MBOX/MD → Unstructured → Chunks with metadata → Vector DB + BM25 IndexSupported file types:
PDF - Text extraction with tables, lists preserved
Word - DOCX, DOC
Spreadsheets - XLSX, XLS, CSV
Email archives - MBOX (Gmail exports, etc.)
Markdown - Ideal for scraped API docs
HTML - Web pages
Plain text - TXT files
Features:
Document structure preserved (headings, sections)
Automatic chunking by section
Email metadata preserved (subject, from, to, date)
Recursive directory scanning (all subdirectories)
2. Search (Hybrid + Agentic)
Query → Contextualize (if follow-up)
→ BM25 (keyword) + Vector (semantic) in parallel
→ Merge (weighted or RRF)
→ Cross-encoder re-rank (adaptive depth)
→ MMR diversity (if enabled)
→ Reference resolution (fetch appendices/sections)
→ [If recursive: analyze → retry with better query]
→ Final results + quality indicatorsBM25 finds exact matches ("Appendix C" → actual Appendix C)
Vector finds semantic matches ("equivalent jurisdictions" → related content)
Merge via weighted scores (default) or RRF rank fusion
Cross-encoder picks the best results (adaptive depth reduces cost on high-confidence matches)
MMR reduces redundancy by penalizing similarity to already-selected chunks
Reference resolver fetches content when chunks say "see Appendix X"
3. Response Generation
Top chunks → Context filter (drop low-scoring)
→ Token budgeting (doc/code/reference allocations)
→ LLM prompt with citation instructions
→ Answer with inline [1] [2] citations
→ Citation footer + Follow-up suggestionsProject Structure
Trove/
├── core/
│ ├── ingest.py # Document processing with Unstructured
│ ├── pipeline.py # Producer-consumer ingestion pipeline
│ ├── search/ # Hybrid search package
│ │ ├── types.py # SearchQuality, SearchResult dataclasses
│ │ ├── tokenizer.py # EnhancedTokenizer, stemming, stop words
│ │ ├── cache.py # Search result LRU+TTL cache
│ │ ├── merge.py # merge_search_results() for multi-index merging
│ │ └── searcher.py # HybridSearcher (BM25 + vector + re-ranking + recursive)
│ ├── llm.py # LLM providers (LM Studio, Claude)
│ ├── client.py # Client configuration + validation
│ ├── refinement.py # Query refinement & contextualization
│ ├── reference_resolver.py # Appendix/section reference resolution
│ ├── reference_config.py # Configurable reference patterns per domain
│ ├── logging_config.py # Audit configuration loading
│ ├── reranker.py # Reranker backends (local, API, fallback)
│ ├── evaluation.py # Retrieval metrics + LLM-as-judge RAG quality
│ ├── chunking.py # Chunking strategies (title, semantic, recursive)
│ ├── ingest_progress.py # Progress tracking and resumable ingestion
│ ├── ingest_job.py # Ingestion job management for MCP tools
│ ├── memory.py # Persistent memory store (SQLite + vector search)
│ ├── workflow.py # Workflow DAG engine (task pipelines)
│ ├── agents.py # Agent template loader (YAML configs)
│ ├── runner.py # Local agent runner (search → RAG → LLM generate)
│ ├── plugins/ # Plugin system
│ │ ├── interfaces.py # ABCs: BaseEmbedder, BaseChunker, BaseParser, etc.
│ │ ├── registry.py # Plugin registry (register/lookup by name)
│ │ └── loader.py # Auto-discovery from plugins/ directory
│ ├── sources/ # Source connectors
│ │ ├── base.py # BaseSourceConnector ABC, SyncResult
│ │ ├── github_connector.py # GitHub repo discovery, clone, sync
│ │ ├── github_client.py # GitHub API client (rate limiting, pagination)
│ │ ├── github_models.py # GitHubSourceConfig, RepoInfo
│ │ ├── git_ops.py # Git clone/fetch with token auth
│ │ ├── web_crawler.py # BFS web crawler with scope control
│ │ ├── web_converter.py # HTML → clean Markdown conversion
│ │ └── web_cache.py # Incremental crawling cache (ETag/content hash)
│ ├── code/ # Code indexing subsystem
│ │ ├── chunker.py # Code file chunking orchestration
│ │ ├── parser.py # Tree-sitter parser management
│ │ ├── git_utils.py # Git repository detection
│ │ ├── incremental.py # Incremental indexing state
│ │ └── extractors/ # Language-specific extractors
│ └── graph/ # Knowledge graph subsystem
│ ├── models.py # Entity, Relationship dataclasses
│ ├── store.py # KnowledgeGraph (NetworkX + JSON)
│ ├── extractor.py # LLM-based entity/relationship extraction
│ ├── search.py # Graph search modes
│ └── export.py # Stats and D3.js export
├── plugins/ # User plugin directory (drop .py files here)
├── mcp_server/
│ ├── server.py # MCP server entry point, tool definitions, dispatcher
│ ├── shared.py # Shared state, cache, helpers
│ └── handlers/ # Tool handler modules (clients, search, ingest, graph, memory, workflows, agents, prompts, sources)
├── agents/ # Global agent YAML templates
├── clients/
│ ├── {name}/
│ │ ├── client.yaml # Config (including sources: section)
│ │ ├── documents/ # Local source files
│ │ ├── sources/ # Auto-managed source directories
│ │ │ ├── github/ # Cloned git repos
│ │ │ └── web/ # Crawled markdown files
│ │ ├── index/ # Search indices
│ │ ├── prompts/ # System prompts
│ │ └── tests/ # Evaluation datasets
│ └── prompts/
│ └── api_client.py # prompts.chat REST API client
├── scripts/
│ ├── create_client.py # Interactive client creation wizard
│ ├── ingest.py # CLI for document ingestion
│ ├── search.py # CLI search tool
│ ├── evaluate.py # CLI evaluation tool
│ ├── build_graph.py # CLI for building knowledge graphs
│ ├── sources.py # CLI for source connectors (GitHub, web)
│ ├── mcp_server.py # MCP server launch script
│ ├── run_agent.py # CLI for running agents locally
│ ├── watch_and_reindex.py # Source sync + file watcher
│ └── sync_prompts.py # Sync prompts from prompts.chat API
└── tests/
└── test_core.py # 724 testsClient Configuration
# clients/example/client.yaml
name: "My RAG System"
machine_name: example
paths:
documents: "documents"
index: "index"
prompts: "prompts"
# Embedding model settings
embedding:
model: "all-MiniLM-L6-v2" # or "BAAI/bge-base-en-v1.5" for better quality
dimension: 384 # 384 for MiniLM, 768 for BGE
query_prefix: "" # Use "query: " for E5 models
passage_prefix: "" # Use "passage: " for E5 models
contextualize: false # Prepend doc context to chunks before embedding (requires reindex)
hnsw_ef_construction: 200 # HNSW build-time neighbor list (requires reindex)
hnsw_max_neighbors: 16 # HNSW max connections per node (requires reindex)
hnsw_ef_search: 100 # HNSW query-time neighbor list (no reindex needed)
search:
bm25_weight: 0.5
vector_weight: 0.5
top_k: 20
rerank_top_n: 50
# Merge strategy: "weighted" (default) or "rrf" (rank fusion)
merge_strategy: "weighted"
rrf_k: 60 # RRF constant (only used with merge_strategy: "rrf")
# MMR diversity: null (disabled) or 0.0-1.0 (0.0=max diversity, 1.0=pure relevance)
mmr_lambda: null
# Adaptive reranking: reduce rerank depth on high-confidence matches
adaptive_rerank: true
adaptive_rerank_threshold: 0.9 # Combined score threshold to trigger
adaptive_rerank_depth: 15 # Reduced depth when triggered
# BM25 tuning
bm25_k1: 1.5 # Term frequency saturation (>0)
bm25_b: 0.75 # Document length normalization (0-1)
score_normalization: "minmax" # "minmax", "zscore", or "raw"
# Advanced query processing (requires LLM provider)
hyde: false # Generate hypothetical doc for vector search
multi_query: false # Generate query variants and merge via RRF
multi_query_count: 3 # Number of query variants (2-7)
# Search result caching
cache_ttl: 0 # TTL in seconds (0 = disabled)
cache_max_size: 128 # Max cached queries per client
# Reranker backend: "local" (CrossEncoder) or "api" (Cohere-compatible)
reranker: "local"
reranker_model: null # Custom reranker model (null=default)
reranker_endpoint: null # API endpoint for "api" reranker
# Tokenization settings for BM25
tokenization:
stemmer: "porter" # "porter", "snowball", or null
synonyms: # Domain-specific synonyms
aml: ["anti-money laundering", "financial crime"]
kyc: ["know your customer"]
# Reference resolution settings
references:
enabled: true
mode: "extend" # "extend" adds to domain defaults, "replace" uses only custom
domain: "compliance" # "compliance", "legal", "technical", or "academic"
# Optional: custom patterns
patterns:
custom_ref:
priority: high
patterns:
- '(?:see\s+)?REF-(\d+)'
query_templates:
- "Reference {id}"
llm:
provider: "lmstudio" # OpenAI-compatible (recommended for LM Studio)
lmstudio_model: "mistralai/mistral-nemo-instruct-2407" # must be set explicitly
max_chunks: 10
max_context_chars: 12000
max_tokens: 2000
citations: false # Enable inline [N] citations in responses
context_filter: false # Drop low-scoring chunks before LLM context
context_filter_threshold: 0.1 # Minimum rerank score for context inclusion
doc_budget_ratio: 0.6 # Token budget share for document chunks
code_budget_ratio: 0.3 # Token budget share for code chunks
reference_budget_ratio: 0.1 # Token budget share for reference chunks
# Optional: Knowledge graph (entity/relationship extraction)
graph:
enabled: false # Opt-in per client
entity_types: # Entity types to extract
- person
- organization
- concept
- location
extract_batch_size: 10 # Chunks per LLM call
max_description_chars: 200 # Truncate descriptions
search_mode: "naive" # Default search: naive, local, global, hybrid, mix
# Optional: Chunking strategy
chunking:
strategy: "title" # "title", "semantic", or "recursive"
max_chunk_size: 1500
min_chunk_size: 100
overlap: 100 # Overlap between chunks (recursive strategy)
semantic_threshold: 0.5 # Cosine similarity for semantic split points
separators: ["\n\n", "\n", ". ", " "] # Priority-ordered separators (recursive strategy)
# Optional: Source connectors (auto-fetch from GitHub/web)
sources:
github:
- owner: myorg # GitHub user or org
token_env: GITHUB_TOKEN # Env var name (never store tokens in YAML)
include_repos: [api, frontend] # Optional whitelist
exclude_repos: [legacy-app] # Optional blacklist
sync_on_ingest: true # Auto-sync before reindexing
- repo: owner/repo # Single repo shorthand
token_env: GITHUB_TOKEN
web:
- url: https://docs.example.com
stay_in_path: true # Don't leave starting URL path
max_pages: 500 # Safety limit
sync_on_ingest: true
# Optional: Code indexing (tree-sitter AST parsing)
code:
enabled: true
paths: ["src/"] # Directories to index (relative to client root)
extensions: [".py", ".ts", ".js"] # File extensions to include
exclude_patterns: ["**/test_*", "**/__pycache__/**"]
# Optional: Code priority in search results (when docs may be outdated)
# Add to search: section
# code_priority: true # Boost code chunks above docs
# code_priority_boost: 0.5 # Score boost for code chunks
# cross_reference_code: true # Auto-fetch related code for doc results
# Optional: Audit settings
audit:
enabled: true
log_level: "full" # "full", "minimal", or "none"
include_chunks: true
include_scores: true
include_timing: trueEmbedding Models
Model | Dimensions | Speed | Quality | Notes |
| 384 | Fast | Good | Default, good for most use cases |
| 768 | Medium | Better | Better retrieval, recommended for production |
| 768 | Medium | Better | Requires query/passage prefixes |
Reference Domains
Domain | Patterns Detected |
| Appendix, Annex, Schedule, Section, Table |
| Exhibit, Clause, Article, Recital, Appendix, Schedule |
| RFC, Issue #, PR #, Section |
| Equation, Theorem, Lemma, Definition, Figure, Table, Chapter |
CLI Commands
Ingestion
# Ingest documents for a client
python scripts/ingest.py --client myproject
# Rebuild from client's documents directory
python scripts/ingest.py --client myproject --rebuild
# Rebuild ALL clients at once
python scripts/ingest.py --all
# Rebuild all clients, documents only (skip code)
python scripts/ingest.py --all --no-code
# Resume an interrupted ingestion (skips already-processed files)
python scripts/ingest.py --client myproject --rebuild --resume
# Force BM25 cache rebuild (use after changing tokenization settings)
python scripts/ingest.py --client myproject --rebuild --force-bm25-rebuild
# List available clients
python scripts/ingest.py --listSearch
# Basic search
python scripts/search.py --client myproject "your search query"
# Verbose output with timing and metadata
python scripts/search.py --client myproject --verbose "your query"
# Use RRF merge instead of weighted
python scripts/search.py --client myproject --merge-strategy rrf "your query"
# Enable MMR diversity (0.0=max diversity, 1.0=pure relevance)
python scripts/search.py --client myproject --mmr-lambda 0.7 "your query"
# Disable adaptive reranking
python scripts/search.py --client myproject --no-adaptive "your query"
# JSON output (for scripting/comparison)
python scripts/search.py --client myproject --json "your query"
# Full content (no truncation)
python scripts/search.py --client myproject --full "your query"
# List available clients
python scripts/search.py --listSource Connectors
Connect external sources (GitHub repos, websites) to a client. Source files are stored in clients/{name}/sources/ and indexed alongside documents/.
# Connect a GitHub org's repos to a client
python scripts/sources.py connect github --client myproject --owner myorg
# Connect a single repo
python scripts/sources.py connect github --client myproject --repo owner/repo
# Filter repos
python scripts/sources.py connect github --client myproject --owner myorg \
--include api frontend --exclude legacy --public-only
# Crawl a website into a client
python scripts/sources.py crawl --client myproject --url https://docs.example.com
# Crawl with options
python scripts/sources.py crawl --client myproject --url https://api.example.com/docs \
--max-pages 200 --wait-time 2.0 --no-stay-in-path
# Sync all configured sources for a client
python scripts/sources.py sync --client myproject
# Sync all clients at once
python scripts/sources.py sync --all
# List configured sources and on-disk state
python scripts/sources.py list --client myprojectAfter connecting sources, ingest them:
python scripts/ingest.py --client myproject --rebuildSources with sync_on_ingest: true in client.yaml automatically sync before each ingestion.
Automatic Reindexing (File Watcher)
The watcher compares files on disk against a manifest saved during ingestion. It detects new, modified, and deleted files — only changed files are reindexed. If a client has source_sync configured, files are synced from the source directory before checking for changes.
# Dry-run to see what changed
python scripts/watch_and_reindex.py --dry-run
# Run for a specific client (sync + reindex)
python scripts/watch_and_reindex.py --client myproject
# Set up as cron (every 15 minutes)
*/15 * * * * cd /path/to/trove && ./venv/bin/python scripts/watch_and_reindex.pySource Sync
Clients can auto-sync files from an external directory into their documents/ folder. Add to client.yaml:
source_sync:
enabled: true
source: ../.. # Relative to client directory
include: ["*.py", "*.md", "*.txt", "*.yaml", "*.json"]
exclude: [clients/, venv/, .git/, __pycache__/]The sync copies new/updated files, removes deleted files, and respects include/exclude patterns. It runs automatically as part of watch_and_reindex.py.
Evaluation
Measure search quality with the evaluation tool. Requires a test dataset at clients/{name}/tests/eval_dataset.json.
# Run retrieval evaluation
python scripts/evaluate.py --client myproject
# Filter by tags
python scripts/evaluate.py --client myproject --tags search
# Include LLM-as-judge RAG quality metrics (requires LM Studio running)
python scripts/evaluate.py --client myproject --rag
# A/B comparison of merge strategies
python scripts/evaluate.py --client myproject --compare "weighted,rrf"
# JSON output for scripting
python scripts/evaluate.py --client myproject --json
# Write markdown report
python scripts/evaluate.py --client myproject --output reports/eval.md
# List clients with eval datasets
python scripts/evaluate.py --listEmbedding Benchmark
Compare embedding models on your own evaluation dataset.
# Compare two models on a client's eval dataset
python scripts/benchmark_embeddings.py --client myproject \
--models "all-MiniLM-L6-v2,BAAI/bge-base-en-v1.5"
# Save results to JSON
python scripts/benchmark_embeddings.py --client myproject \
--models "all-MiniLM-L6-v2" --output results.jsonKnowledge Graph
Build a knowledge graph from indexed document chunks using LLM entity/relationship extraction.
# Build graph for a client (incremental - skips already-processed chunks)
python scripts/build_graph.py --client myproject
# Rebuild from scratch
python scripts/build_graph.py --client myproject --rebuild
# Dry-run (show chunk count and estimated LLM calls)
python scripts/build_graph.py --client myproject --dry-run
# Test with a sample of 50 random chunks
python scripts/build_graph.py --client myproject --sample 50
# Build for all graph-enabled clients
python scripts/build_graph.py --all
# Build even if graph.enabled is false in client.yaml
python scripts/build_graph.py --client myproject --force
# Graph saves after every batch — safe to cancel mid-run, resumes on next incremental runLocal Agent Runner
Run Trove agents end-to-end using a local LLM (LM Studio). Combines agent system prompt + RAG context search + LLM generation.
# List available agents
python scripts/run_agent.py --list-agents
# List available clients
python scripts/run_agent.py --list-clients
# One-shot query
python scripts/run_agent.py --agent code-reviewer --client example "Review the search pipeline"
# Interactive REPL with conversation history
python scripts/run_agent.py --agent code-reviewer --client example --interactive
# JSON output for scripting
python scripts/run_agent.py --agent security-reviewer --client myproject --top-k 15 --json "Check auth patterns"Available agents: code-reviewer, security-reviewer, performance-analyzer, documentation-writer, test-generator, architecture-analyst
Prompt Sync
Sync new and trending prompts from the live prompts.chat API into the local index.
# Preview what would be synced
python scripts/sync_prompts.py --dry-run
# Sync newest + popular prompts (2 pages per sort order)
python scripts/sync_prompts.py
# Sync more pages for deeper coverage
python scripts/sync_prompts.py --pages 100
# Sync and trigger incremental reindex
python scripts/sync_prompts.py --reindexOvernight Rebuild
Reindex all clients and rebuild knowledge graphs in one run. Requires LM Studio running with the configured model loaded.
# Incremental (changes only — fast on most nights)
./overnight-rebuild.sh
# Full rebuild from scratch
./overnight-rebuild.sh --rebuildEdit the script to uncomment/comment clients in the REINDEX_CLIENTS and GRAPH_CLIENTS arrays.
Client Creation
# Interactive wizard
python scripts/create_client.pyMCP Server (Claude Code Integration)
Trove includes an MCP (Model Context Protocol) server that allows Claude Code to query your indexed documentation directly.
Use Case
Scrape API documentation and convert to markdown
Create a Trove client for the API (
python scripts/create_client.py)Ingest the docs (
python scripts/ingest.py --client api_name)Configure Claude Code to use the Trove MCP server
Claude Code can now search your API docs when helping you code
Setup
Add to your Claude Code settings (~/.claude/settings.json) or project .mcp.json:
{
"mcpServers": {
"trove": {
"command": "python",
"args": ["/path/to/trove/scripts/mcp_server.py"],
"env": {
"TROVE_DEFAULT_CLIENT": "your_client_name"
}
}
}
}Available Tools
Tool | Description |
| List all available Trove clients/projects |
| Get info about the currently loaded client |
| Load a specific client for searching |
| Search documents with mode presets and context budgets |
| Load and search a client in one call |
| Search across multiple clients simultaneously (cross-client RRF merge) |
| Search code only, with language/symbol type filtering |
| List git repositories in the loaded client's index |
| Search within a specific repository |
| Find a function, class, or method by name |
| Search and return concise summaries instead of full content |
| Ingest a single file into a client's index |
| Fetch and ingest content from a URL |
| Trigger a full background reindex for a client |
| Check status/progress of an ingestion job |
| Get knowledge graph statistics (entity/relationship counts, top entities) |
| Export knowledge graph as D3.js-compatible JSON |
| Search knowledge graph entities by name |
| Traverse knowledge graph relationships from an entity |
| Build/rebuild a knowledge graph (background) |
| Connect GitHub repos to a client (discover, clone, sync) |
| Crawl a website and save pages as Markdown |
| Sync all configured sources for a client |
| List configured sources with sync status |
| Store a key-value memory entry (with optional TTL and vector embedding) |
| Search memory by semantic similarity or keyword |
| List stored memories in a namespace |
| Delete a memory entry |
| Create a multi-step workflow DAG with dependencies |
| Get workflow status and step progress |
| Update a workflow step status (running/completed/failed/skipped) |
| List all workflows with progress summaries |
| List available agent templates |
| Get full agent template details |
| Run an agent end-to-end: search context, apply system prompt, generate via local LLM |
| Combined local index + live prompts.chat API search (deduplicated) |
| Search live prompts.chat API with type/category/tag/sort filters |
| Fetch full prompt by ID with optional variable substitution |
Example Usage in Claude Code
# List available documentation projects
Use trove_list_clients to see what's available
# Search API documentation
Use trove_search_client with client_name="stripe_api" and query="how to create a payment intent"
# Search currently loaded client
Use trove_search with query="authentication flow"Prompt Discovery Examples
The prompt discovery tools search 10k+ indexed prompt templates and the live prompts.chat API.
Find a prompt (combined local + live search):
Find me a prompt template I can use as a senior code reviewerI need an AI prompt for generating midjourney-style portraitsBrowse live prompts with filters:
Search prompts.chat for the newest IMAGE generation promptsWhat are the most popular prompts on prompts.chat right now?Fetch a full prompt with variable substitution:
Fetch prompt cmj1zb0yc000ivl0rc9ka596e and set Position to "Data Engineer"Use the /prompt-finder skill shortcut:
/prompt-finder react code review assistant
/prompt-finder cartoon illustration of a Mini Cooper
/prompt-finder SQL expert that explains query optimizationMulti-step workflow:
Is there a prompt template that acts as a job interviewer?
Fetch the full version and customize it for a DevOps Engineer position.Creating Skills That Use Trove
You can create custom Claude Code skills in other projects that query Trove for documentation context. See docs/CLAUDE_CODE_INTEGRATION.md for:
Setting up global MCP configuration
Creating project-specific skills
Example skill templates (including
/prompt-finderfor AI prompt discovery)Multi-client workflows
Development
See CLAUDE.md for architecture details, code guidelines, and module reference.
Running Tests
pytest tests/ -vThis server cannot be deployed
Maintenance
Related MCP Connectors
Cloud or self-hosted knowledge for AI agents: hybrid search, reranking, GraphRAG, scoped MCP tools.
Intelligent context infrastructure for AI teams: knowledge graph, sessions, tasks, documents.
Your company's brain for AI agents. Cited, permission-aware knowledge across every system.
Code intelligence for coding agents: semantic, AST, graph, and full-text search. 279+ languages.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI coding assistants to search and retrieve information from a locally ingested knowledge base using hybrid search, grounded in user-curated documentation.16MIT
- AlicenseNot gradedqualityAmaintenanceEnables private source-code repositories to be indexed and searched via hybrid retrieval (symbol, lexical, semantic, and code-graph), then packages token-budgeted, structure-aware context for LLMs such as Claude.73 PyPIApache 2.0
- AlicenseBqualityAmaintenanceEnables AI coding agents to build and query local repository memory, returning ranked files, context packs, relationship maps, semantic recall, and feedback-tuned search without uploading code.91,459 PyPI1MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI coding agents to query a local semantic code graph for surgical context, cross-file resolution, framework-aware routes, and code intelligence while keeping indexes automatically up to date.77,341 npmMIT