# web-tools-mcp-server (`mcptools`)

An all-in-one **MCP server for local LLMs** (Gemma, gpt-oss, Qwen, etc. via
LM Studio, Ollama, or any MCP client). It gives a model the capabilities small
local models lack: live web access, precise math, the current date, shell + code
execution, filesystem and SQLite access, a persistent memory, and semantic
search (RAG) over your own files using LM Studio embeddings.

- **Transport:** stdio (local).
- **Config:** `~/.config/mcptools/config.json` (auto-created on first run).
- **No cloud keys required** — search is Startpage (+ Google headless
  fallback), weather is open-meteo, embeddings are your local LM Studio.
- **39 tools** in 10 config-gated groups.

---

## Table of contents

- [Tools](#tools-39-in-10-groups)
- [Tool reference](#tool-reference)
- [Install & build](#install--build)
- [Run](#run)
- [Register with a client](#register-with-a-client) — [Claude Code](#claude-code) · [OpenCode](#opencode) · [Generic](#generic-mcp-client)
- [Configuration](#configuration-confmcptoolsconfigjson)
- [RAG / LM Studio](#rag--lm-studio)
- [Development](#development)

---

## Tools (39, in 10 groups)

| Group | Tools | Notes |
|-------|-------|-------|
| **web** | `web_search`, `fetch_url`, `extract_links`, `get_page_metadata`, `http_request` | Mojeek; HTML→text/markdown |
| **compute** | `calculator`, `current_datetime` | mathjs; IANA timezones |
| **system** | `read_file`, `write_file`, `list_directory`, `run_command`, `execute_code`, `notify`, `server_info` | **unrestricted**, timeout-guarded; python/node/bash |
| **files** | `search_files`, `find_files`, `edit_file`, `edit_lines`, `pdf_to_json`, `download_file` | grep/glob/exact + line-based edits without shell quoting; PDF→JSON; binary-safe downloads |
| **data** | `sqlite_query`, `json_query` | any local `.db`; path queries into JSON/CSV |
| **knowledge** | `wikipedia_lookup`, `weather` | keyless |
| **feeds** | `read_rss`, `hackernews`, `arxiv_search`, `youtube_transcript` | keyless live sources |
| **memory** | `memory_save`, `memory_recall`, `memory_list`, `memory_delete` | persistent SQLite at `<dataDir>/memory.db` |
| **tasks** | `todo_add`, `todo_list`, `todo_done` | persistent checklist at `<dataDir>/todos.db` |
| **rag** | `rag_index`, `rag_search`, `rag_list`, `rag_delete` | embeddings via LM Studio; vectors in `<dataDir>/rag.db` |

> ⚠️ **Security:** `run_command`, `execute_code`, `write_file`, `edit_file`, `download_file` and unrestricted
> filesystem access run with your user's full privileges (only timeouts, no
> sandbox — as configured). Run this server only with clients/models you trust,
> or disable those groups in the config (see [Configuration](#configuration-confmcptoolsconfigjson)).

---

## Tool reference

Every tool returns human-readable text plus `structuredContent` for programmatic
clients. Responses are capped to `limits.characterLimit` and truncated with a note.

### web
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `web_search` | `query`, `limit` (1–20, def 5), `response_format` | Ranked results: title, url, snippet |
| `fetch_url` | `url`, `format` (`text`\|`markdown`) | Cleaned page content (scripts/nav stripped) |
| `extract_links` | `url`, `limit` (1–200, def 50) | Absolute links with anchor text |
| `get_page_metadata` | `url` | title, description, canonical, OpenGraph |
| `http_request` | `url`, `method`, `headers`, `body` | status, headers, body (any REST API) |

### compute
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `calculator` | `expression` | mathjs result (functions, constants, units) |
| `current_datetime` | `timezone` (IANA, optional) | ISO + human time + unix seconds |

### system
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `read_file` | `path`, `offset?`, `max_lines?`, `line_numbers?` | File contents, pageable with line numbers (PDFs auto-detected, text extracted per page) |
| `write_file` | `path`, `content`, `append?` | Bytes written (creates parent dirs) |
| `list_directory` | `path` (def `.`) | Entries with type + size |
| `run_command` | `command`, `cwd?` | exit code, stdout, stderr (system shell) |
| `execute_code` | `language` (`python`\|`node`\|`bash`), `code`, `stdin?` | exit code, stdout, stderr |
| `notify` | `title`, `message`, `urgency?` | Desktop notification via `notify-send` |
| `server_info` | — | mcptools version, enabled/disabled groups, config + data paths |

### files
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `search_files` | `path`, `pattern` (regex), `glob?`, `context_lines?`, `max_results?` | Matches as `file:line: text` (skips binaries, `.git`, `node_modules`) |
| `find_files` | `path`, `pattern` (glob, e.g. `**/*.ts`), `max_results?` | Matching paths with sizes |
| `edit_file` | `path`, `old_string`, `new_string`, `replace_all?`, `dry_run?` | Exact in-place replacement (errors if not found / ambiguous); returns edited region |
| `edit_lines` | `path`, `mode` (`replace`\|`insert_before`\|`insert_after`\|`delete`), `start_line`, `end_line?`, `new_text?`, `dry_run?` | Line-based edits (use `read_file` with `line_numbers` first) |
| `pdf_to_json` | `path`, `output_path?` | PDF as JSON: metadata + per-page text (write to file for big PDFs, then `json_query` it) |
| `download_file` | `url`, `path` | Binary-safe download to disk (pairs with `read_file` for PDFs) |

### data
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `sqlite_query` | `db_path`, `sql`, `params?`, `read_only?` | Rows (SELECT) or change count |
| `json_query` | `path` (.json/.csv), `query` (e.g. `.users[].name`), `limit?` | Matching slice as JSON |

### knowledge
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `wikipedia_lookup` | `query`, `lang` (def `en`) | Article title, summary, url |
| `weather` | `location`, `days` (1–7, def 3) | Current conditions + daily forecast |

### feeds
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `read_rss` | `url`, `limit?` | Feed items: title, link, date, summary (RSS 2.0 + Atom) |
| `hackernews` | `query` (empty = front page), `sort?`, `limit?` | Stories with points, comments, discussion link (Algolia, keyless) |
| `arxiv_search` | `query`, `sort?` (`relevance`\|`latest`), `limit?` | Papers with abstract + `pdf_url` (download + `read_file` to read) |
| `youtube_transcript` | `video` (URL or ID), `lang?` | Full caption transcript as text |

### memory
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `memory_save` | `key`, `value`, `tags?` | Confirmation (upsert) |
| `memory_recall` | `key?` \| `search?`, `limit?` | Matching notes |
| `memory_list` | — | All keys with tags + timestamps |
| `memory_delete` | `key` | Confirmation |

### tasks
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `todo_add` | `text` | New task id |
| `todo_list` | `include_done?` | Checklist with ids |
| `todo_done` | `id`, `remove?` | Marks done (or deletes with `remove: true`) |

### rag
| Tool | Key arguments | Returns |
|------|---------------|---------|
| `rag_index` | `path` (file or dir), `max_files?` (def 200) | Files + chunks indexed (skips on backend error) |
| `rag_search` | `query`, `top_k` (1–20, def 5) | Ranked passages with source + score |
| `rag_list` | _(none)_ | Indexed sources with chunk counts + store totals |
| `rag_delete` | `source` (path from `rag_list`) | Chunks deleted + updated store totals |

---

## Install & build

```bash
cd ~/Code/mcptools
npm install
npm run build      # compiles to dist/
```

Requires **Node ≥ 22** (uses the built-in `node:sqlite`). `python3` is needed
for `execute_code` with `language: "python"`.

## Run

```bash
node dist/index.js     # speaks MCP over stdio
```

The first run writes a default `~/.config/mcptools/config.json`.

---

## Register with a client

### Install script (recommended)
```bash
npm run install:claude     # Claude Code (~/.claude.json, user scope)
npm run install:opencode   # OpenCode (~/.config/opencode/opencode.json)
npm run install:all        # both
```

Merges the `mcptools` entry into the client's user-level config (everything
else in the file is preserved; re-running updates the entry in place). Build
first — the script points the client at `dist/index.js`.

### Claude Code (manual)
```bash
claude mcp add mcptools -- node ./dist/index.js
```

### OpenCode
Add to `~/.config/opencode/config.json` (global) or a project `opencode.json`,
under the `mcp` key:

```json
{
  "mcp": {
    "mcptools": {
      "type": "local",
      "command": ["node", "./dist/index.js"],
      "enabled": true
    }
  }
}
```

Restart OpenCode (or start a new session) to spawn the server. Verify with
`/mcp` inside OpenCode, or list tools from the TUI.

### Generic MCP client (`mcpServers` JSON)
```json
{
  "mcpServers": {
    "mcptools": {
      "command": "node",
      "args": ["./dist/index.js"]
    }
  }
}
```

> 💡 Small local models choose tools more reliably with **fewer** of them. If 39
> tools is too many for your model, disable groups in the config rather than the
> client — see below.

---

## Configuration (`~/.config/mcptools/config.json`)

```jsonc
{
  "groups": {                 // turn whole tool groups on/off
    "web": true, "compute": true, "system": true, "data": true,
    "knowledge": true, "memory": true, "rag": true
  },
  "tools": {                  // per-tool overrides win over group, e.g.:
    // "run_command": false,
    // "write_file": false
  },
  "limits": {
    "timeoutMs": 30000,       // network/exec timeout
    "characterLimit": 25000,  // max chars per tool response
    "maxOutputBytes": 1000000 // max bytes captured from a process/file/fetch
  },
  "lmstudio": {
    "baseUrl": "http://localhost:1234/v1",
    "embeddingModel": "text-embedding-nomic-embed-text-v1.5"
  },
  "paths": { "dataDir": "~/.config/mcptools" }
}
```

- **Per-tool overrides** (`tools`) take precedence over the group switch.
- Hot env overrides: `MCPTOOLS_LMSTUDIO_URL`, `MCPTOOLS_EMBED_MODEL`.
- For a **safe, read-only-ish** profile set `"system": false` (and optionally
  `"data": false`) in `groups`.

A copy of the defaults lives in [`config.example.json`](./config.example.json).

---

## RAG / embedding server

`rag_index` and `rag_search` call any OpenAI-compatible
`/v1/embeddings` endpoint. At startup, the server **auto-detects** which
embedding backend is running and uses it:

1. The URL in `lmstudio.baseUrl` (or `MCPTOOLS_LMSTUDIO_URL` env var) — primary.
2. `http://localhost:1234/v1` — LM Studio.
3. `http://localhost:8089/v1` — highllama / llama.cpp.

For each, it hits `/v1/models` with a 2 s timeout, picks the most promising
embedding model (configured id, then anything with `embed`/`bge`/`minilm`/`e5`
in its id), and **verifies with a real probe** `/v1/embeddings` call that the
model actually returns a numeric vector. This rejects llama.cpp servers that
are currently serving a chat model (the common case — llama.cpp loads one
model at a time, so if it's busy with `gemma-4-26B` for chat, it can't also
serve embeddings). For single-model servers with a name that doesn't match
the patterns, the same probe is used as a fallback so custom-named embedding
models are still discovered. The chosen endpoint is logged to stderr on
startup, e.g.:

```
[mcptools/rag] using LM Studio (1234) with model "text-embedding-nomic-embed-text-v1.5"
```

If nothing is reachable, the configured values are used and the embedding call
will fail with a clear error.

### ⚠️ Embedding model crashing on GPU — load it on CPU

Observed with `text-embedding-nomic-embed-text-v1.5`: the model crashes
mid-request ("The model has crashed... Exit code: null") and then fails to reload
("unable to allocate CUDA0 buffer"). **Root cause is GPU/CUDA, not the input** —
the same chunks embed fine on CPU. This tiny (~84 MB) model runs fast on CPU.

Load it CPU-only (the `lms` CLI is the only place that sets the GPU layer-offload
ratio — the REST API does not):

```bash
~/.lmstudio/bin/lms unload --all
~/.lmstudio/bin/lms load text-embedding-nomic-embed-text-v1.5 --gpu off -y
```

Loads in ~0.5 s and embeds reliably. An explicitly-loaded model has **no TTL** and
stays resident until LM Studio restarts, so JIT won't reload it on the GPU.

**Persisting it (headless / no GUI).** The GUI "per-model default" *is* the
correct knob and applies to JIT loading, but LM Studio only writes it from the
GUI (into an undocumented internal file — don't hand-edit it). Headless options:

- Run the `lms load ... --gpu off` command at login (e.g. a `systemd --user`
  oneshot unit, or append it to your existing `~/.lmstudio/load-optimized.sh`).
- Or just run it once per LM Studio session before using RAG.

If you have the GUI on another machine: **My Models → ⚙️ → set GPU offload = 0**.

### Defensive hardening (any backend)
- Inputs are embedded **one at a time** (batched array input crashes some backends).
- Each embed call **retries with backoff**.
- `rag_index` **skips** any chunk that still fails and reports the count, instead
  of aborting the whole index.

---

## Development

```bash
npm run dev        # tsx watch (no build step)
npm run build      # type-check + compile
npm run clean      # remove dist/
```

Project layout:
```
src/
  index.ts            entry: load config, register enabled tools, stdio
  config.ts           config load/merge + defaults (~/.config/mcptools)
  registry.ts         group/tool enable gating
  types.ts            shared types
  services/           search (mojeek), lmstudio (embeddings), vectorstore
  tools/              web, compute, system, data, knowledge, memory, rag
  util/               http, html, exec, sqlite, format, errors
```

Data files created at runtime under `paths.dataDir` (default
`~/.config/mcptools`): `config.json`, `memory.db`, `rag.db`.