# Local-first Knowledge Base

A private, local-first evidence search system for TXT, CSV, Google Sheets, Zoho Mail and MBOX sources. It normalizes source documents into SQLite FTS5 and exposes cited results through a local web UI, CLI and MCP server.

The runtime does not call a paid LLM API. It returns evidence with source links instead of generating unsupported answers.

## Features

- SQLite FTS5 index with rare-term scoring, character similarity, recency and reciprocal-rank fusion;
- idempotent TXT, CSV and Google Sheets ingestion;
- Google Sheets connector restricted to `spreadsheets.readonly`, one configured spreadsheet and configured sheet IDs;
- GET-only Zoho Mail connector with exact account and folder matching;
- local web UI bound to loopback only;
- MCP tools for cited retrieval and source status;
- no-answer gate for weak or unrelated matches;
- no model API keys or token-based LLM costs.

## Quick start

Requirements: Python 3.11+ and Node.js only for the JavaScript syntax check.

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

cp data/lukasz_knowledge_base/config.example.json \
  data/lukasz_knowledge_base/config.json
cp data/lukasz_knowledge_base/internal_context.example.txt \
  data/lukasz_knowledge_base/internal_context.txt

python execution/lukasz_knowledge_base.py init
python execution/lukasz_knowledge_base.py ingest-txt \
  data/lukasz_knowledge_base/internal_context.txt
python execution/lukasz_knowledge_base.py serve
```

Open `http://127.0.0.1:8765`.

The server refuses to bind to a public interface because the MVP has no remote authentication.

## Google Sheets

1. Create a dedicated Google service account.
2. Share only the required spreadsheet with that identity as Viewer.
3. Save its credential locally as `service_account.json`. This filename is ignored by Git.
4. Set the spreadsheet URL and allowed `sheet_gids` in the private `config.json`.
5. Run:

```bash
python execution/lukasz_knowledge_base.py sync-sheet
```

The connector constructs credentials with only:

```text
https://www.googleapis.com/auth/spreadsheets.readonly
```

It uses spreadsheet metadata and value reads; it contains no Sheets write operation.

## Zoho Mail

Create a dedicated folder such as `Knowledge Base` and a dedicated OAuth grant with read scopes only:

```text
ZohoMail.accounts.READ
ZohoMail.folders.READ
ZohoMail.messages.READ
```

Provide credentials through local `LOCAL_KB_ZOHO_*` environment variables or an ignored token file, then run:

```bash
python execution/lukasz_knowledge_base.py sync-zoho
```

The default configuration blocks `Inbox`. Email source text is marked as untrusted evidence.

## Search

```bash
python execution/lukasz_knowledge_base.py status
python execution/lukasz_knowledge_base.py search "What is our response process?"
python execution/lukasz_knowledge_base.py search "What is the offer?" --source internal_txt
```

Each result includes a snippet, source type, metadata, score and source URL. When the lexical evidence is too weak, the result list is empty.

## MCP

Example MCP client entry:

```json
{
  "mcpServers": {
    "local-knowledge-base": {
      "command": "python",
      "args": ["/absolute/path/execution/lukasz_knowledge_base_mcp.py"],
      "env": {
        "LOCAL_KB_CONFIG": "/absolute/path/data/lukasz_knowledge_base/config.json"
      }
    }
  }
}
```

Available tools:

- `search_knowledge`
- `list_knowledge_sources`
- `knowledge_base_status`

## Architecture

```text
TXT / CSV / Google Sheets / Zoho / MBOX
                    |
        read-only source connectors
                    |
     normalized documents + provenance
                    |
    SQLite FTS5 + hybrid lexical ranking
                    |
      cited evidence + no-answer gate
                    |
             CLI / Web / MCP
```

## Secret safety

Never commit private `config.json`, `.env`, service-account JSON, OAuth tokens, MBOX exports, internal context or generated databases. The included `.gitignore` excludes these files, but secret scanning and credential rotation remain the operator's responsibility.

Run the same checks used before publication:

```bash
python -m pip install -r requirements-dev.txt
python -m pytest -q
node --check execution/lukasz_knowledge_base_web/app.js
gitleaks detect --no-git --source . --redact
```

## License

MIT