Skip to main content
Glama
jstrick9

MCP Web Research Agent

by jstrick9
README.md
# MCP Web Research Agent for macOS

A Python [Model Context Protocol](https://modelcontextprotocol.io) (MCP) agent/server that gives a local AI assistant tools for:

- `search_web` — public web search via DuckDuckGo HTML results
- `fetch_url` — fetch a public web page and extract readable text
- `save_note` — save research notes as Markdown/text files in a folder you choose

The project also includes `agent.py`, a small local bridge that connects **Ollama** to the MCP server. Ollama runs the LLM; this project provides the MCP tools and the tool-calling agent loop.

> Note: Ollama itself is a model server, not a native MCP client. To use Ollama with MCP tools, run `agent.py` here or another MCP bridge/client.

## What you need

- MacBook Pro with macOS
- Python 3.11 or newer (the `mcp` package requires Python 3.10+; setup prefers 3.13/3.12/3.11)
- [Ollama](https://ollama.com) installed and running
- A tool-calling local model. Recommended starting point:
  - `qwen2.5:7b` for 16 GB RAM Macs
  - `qwen2.5:14b` if you have enough RAM/performance
  - `qwen3:14b` if your Ollama version supports it well

## 1. Install

Open Terminal and run:

```bash
cd ~/Projects
git clone <your-repo-url> mcp-web-research-agent  # or copy this folder here
cd ~/Projects/mcp-web-research-agent

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
```

If you do not have Python 3.11+:

```bash
brew install python
```

### Confirm the install works (no Ollama needed)

These two checks start the real MCP servers and drive the real agent loop
against a mock Ollama endpoint. They need no network and no downloaded model,
so they are the fastest way to confirm a fresh clone is healthy:

```bash
./.venv/bin/python tests/e2e_mcp.py
bash tests/e2e_agents.sh
```

You should see `ALL CHECKS PASSED` and `ALL BRIDGE AGENT CHECKS PASSED`.
Together they exercise all 32 MCP tools plus one full tool call through each
bridge agent.

## 2. Install and start Ollama

Install Ollama from <https://ollama.com> or with Homebrew:

```bash
brew install --cask ollama
```

Open the Ollama app once, then pull a model:

```bash
ollama pull qwen2.5:7b
ollama serve
```

In another Terminal tab, verify Ollama is running:

```bash
curl http://localhost:11434/api/tags
```

## 3. Run the local Ollama MCP agent

From the project folder:

```bash
cd ~/Projects/mcp-web-research-agent
source .venv/bin/activate
python agent.py
```

Then ask something like:

```text
Research recent MCP news, open the two best sources, summarize them, and save the summary as mcp-news.md.
```

One-shot mode:

```bash
python agent.py "Research current MCP SDK best practices and save notes."
```

Use a different model:

```bash
python agent.py --model qwen2.5:14b
```

Choose where notes are saved:

```bash
python agent.py --notes-dir ~/Documents/research-notes
```

## 4. Use with Claude Desktop

If you want Claude Desktop to connect directly to the MCP server, edit:

```text
~/Library/Application Support/Claude/claude_desktop_config.json
```

Add:

```json
{
  "mcpServers": {
    "web-research": {
      "command": "/Users/YOUR_USERNAME/Projects/mcp-web-research-agent/.venv/bin/python",
      "args": [
        "/Users/YOUR_USERNAME/Projects/mcp-web-research-agent/server.py"
      ],
      "env": {
        "MCP_NOTES_DIR": "/Users/YOUR_USERNAME/Documents/MCP-research-notes"
      }
    }
  }
}
```

Replace `YOUR_USERNAME` with your Mac username. Create the file if it does not exist. Restart Claude Desktop after editing.

## 5. Use with Cursor

Create or edit `.cursor/mcp.json` in a workspace:

```json
{
  "mcpServers": {
    "web-research": {
      "command": "/Users/YOUR_USERNAME/Projects/mcp-web-research-agent/.venv/bin/python",
      "args": [
        "/Users/YOUR_USERNAME/Projects/mcp-web-research-agent/server.py"
      ],
      "env": {
        "MCP_NOTES_DIR": "/Users/YOUR_USERNAME/Documents/MCP-research-notes"
      }
    }
  }
}
```

Then restart Cursor or reload its MCP settings.

## Tool reference

### `search_web(query: str, max_results: int = 5)`

Returns search results as JSON:

```json
{
  "query": "Model Context Protocol",
  "results": [
    {
      "title": "Example",
      "url": "https://example.com",
      "snippet": "..."
    }
  ]
}
```

### `fetch_url(url: str, max_chars: int = 8000)`

Fetches an `http`/`https` URL and returns extracted text. It avoids JavaScript rendering, so it works best on normal HTML pages.

### `save_note(filename: str, content: str)`

Saves a note to `MCP_NOTES_DIR`. The default directory is:

```text
~/MCPWebResearch/notes
```

The tool sanitizes filenames and blocks path traversal.

## Configuration

Environment variables:

- `OLLAMA_MODEL` — default model used by `agent.py`; default is `qwen2.5:7b`
- `OLLAMA_URL` — OpenAI-compatible Ollama chat endpoint; default is `http://localhost:11434/v1/chat/completions`
- `MCP_NOTES_DIR` — directory for saved notes

Example:

```bash
export OLLAMA_MODEL=qwen2.5:14b
export MCP_NOTES_DIR=~/Documents/research-notes
python agent.py
```

## Troubleshooting

### `Connection refused` to `localhost:11434`

Ollama is not running. Start it with:

```bash
ollama serve
```

### The agent does not call tools

Use a model with strong tool-calling support. `qwen2.5:7b`, `qwen2.5:14b`, and similar Qwen models are good starting points.

### A page returns little text

Some websites block non-browser clients or require JavaScript. Try a different source, or use `search_web` and `fetch_url` together.

### Claude Desktop does not show the server

Double-check that:

- The Python path points to `.venv/bin/python` inside this project
- The `server.py` path is absolute
- The JSON file has valid syntax
- You fully restarted Claude Desktop

## Files

- `server.py` — MCP server with web research tools
- `agent.py` — local Ollama-powered MCP client/agent loop
- `requirements.txt` — Python dependencies

## Safety notes

- This server can fetch public URLs and search the public web.
- It can write files only into `MCP_NOTES_DIR`.
- It does not execute shell commands.
- Review saved notes and citations before relying on them.

---

# Second MCP agent: Local Planner

The repo now includes a second MCP server/agent: `local-planner`. It stores projects, tasks, and Markdown notes on disk.

## What it does

Tools:

- `create_project(name, description)`
- `list_projects()`
- `create_task(project, title, notes, priority, due_date, status)`
- `list_tasks(project, status)`
- `update_task(project, task_id, ...)`
- `complete_task(project, task_id)`
- `delete_task(project, task_id)`
- `save_project_note(project, content, append)`
- `get_daily_focus(for_date)`

Default data directory:

```text
~/MCPPlanner
```

Override it with:

```bash
export MCP_PLANNER_DIR=~/Documents/my-planner
```

## Run the Ollama planner

```bash
bash run-planner.sh
```

One-shot:

```bash
bash run-planner.sh "Create a project called Weekend Yard Work with tasks for mowing, trimming bushes, and buying mulch."
```

Use a different model:

```bash
bash run-planner.sh --model qwen2.5:14b
```

Store planner data elsewhere:

```bash
bash run-planner.sh --data-dir ~/Documents/planner-data
```

## Good planner prompts

```text
Create a project called Home Network Upgrade and break it into at least six tasks with priorities.
```

```text
Look at my daily focus and tell me what I should work on first.
```

```text
Create a moving checklist project with tasks, due dates, and notes.
```

```text
Mark the first task in the Weekend Yard Work project complete and tell me what remains.
```

## Claude Desktop config for planner

Use:

```text
claude_desktop_config.planner.example.json
```

Add it to:

```text
~/Library/Application Support/Claude/claude_desktop_config.json
```

You can merge both servers under `mcpServers` so Claude sees web research and planning tools.

## Cursor config for planner

Use:

```text
cursor-mcp.planner.example.json
```

## Files

- `planner_server.py` — MCP server for projects/tasks/notes
- `planner_agent.py` — Ollama bridge/agent for the planner
- `run-planner.sh` — launcher

---

# Third MCP agent: Health & Habit Tracker

The repo includes a third MCP server/agent for tracking habits, workouts, meals, and body measurements. All data is stored locally under `MCP_HEALTH_DIR` (default `~/MCPHealth`).

> This tool is for personal tracking only and does not provide medical advice.

## Tools

- `create_habit(name, description, target_per_week, unit)`
- `list_habits(active_only)`
- `log_habit(log_date, value, notes, habit_id|habit_name)`
- `log_workout(activity, duration_minutes, log_date, intensity, calories, distance_km, notes)`
- `log_meal(description, meal_type, log_date, calories, protein_g, carbs_g, fat_g, notes)`
- `log_measurement(weight_kg, log_date, body_fat_pct, waist_cm, notes)`
- `list_logs(log_type, from_date, to_date, limit)`
- `delete_log(log_id)`
- `save_health_note(content, append)`
- `get_daily_summary(for_date)`
- `get_weekly_report(for_date)`

## Run with Ollama

```bash
bash run-health.sh
```

One-shot:

```bash
bash run-health.sh "Create habits for a 30-min walk, drinking water, and stretching, then log today's walk and a lunch salad."
```

Use a different model or data directory:

```bash
bash run-health.sh --model qwen2.5:14b --data-dir ~/Documents/health-data
```

## Good prompts

```text
Create habits for walking 5 days per week, drinking 80 oz of water, and stretching daily.
```

```text
Log a 45-minute moderate run today that burned 420 calories and covered 6 km.
```

```text
Log my breakfast: oatmeal with banana and peanut butter, about 520 calories and 22 grams of protein.
```

```text
Give me today's health summary and list habits I still need to complete.
```

```text
Give me my weekly report and tell me which habits I'm behind on.
```

## Claude Desktop / Cursor configs

- `claude_desktop_config.health.example.json`
- `cursor-mcp.health.example.json`

You can run all four MCP servers together (web research, planner, health, knowledge base) by listing each under `mcpServers`.

## Files

- `health_server.py` — MCP server
- `health_agent.py` — Ollama bridge/agent
- `run-health.sh` — launcher

---

# Fourth MCP agent: Personal Knowledge Base

A searchable long-term memory that ties the other three agents together. Your research agent, planner, and health tracker all *write* notes, but nothing could search them. This agent indexes those folders plus anything you save manually, with real full-text search.

Search uses SQLite's FTS5 extension with BM25 relevance ranking (both ship with Python, so there are no new dependencies). If FTS5 is unavailable on a platform, the server automatically falls back to substring search and reports `"search_mode": "substring"`.

Data is stored under `MCP_KB_DIR` (default `~/MCPKnowledge`) in a single SQLite file, `kb.db`.

## Tools

- `save_snippet(content, title, tags, source_url, source_path, source_type)`
- `search_kb(query, tag, limit, content_chars)`
- `list_snippets(tag, source_type, limit, content_chars)`
- `get_snippet(snippet_id)`
- `delete_snippet(snippet_id)`
- `list_tags()`
- `rename_tag(old_tag, new_tag)`
- `ingest_notes(directories, tag, recursive)`
- `kb_stats()`

## Import notes from your other agents

This is the highest-value first step. It pulls `.md` and `.txt` files into the index:

```text
Ingest my notes from ~/MCPWebResearch/notes, ~/MCPPlanner, and ~/MCPHealth with the tag imported.
```

Re-running is safe: unchanged files are skipped, changed files are updated in place, and nothing is duplicated. Skips `.git`, `.venv`, and `node_modules`.

## Run with Ollama

```bash
bash run-kb.sh
```

One-shot:

```bash
bash run-kb.sh "Ingest my research notes, then summarize everything I have saved about MCP."
```

Different model or database location:

```bash
bash run-kb.sh --model qwen2.5:14b --data-dir ~/Documents/knowledge
```

## Good prompts

```text
Save this: FastMCP exposes Python functions as MCP tools via a decorator. Tag it python and mcp.
```

```text
Search my knowledge base for stdio transport and cite the source of each result.
```

```text
What do I have tagged mcp? List titles and sources.
```

```text
Rename the tag py to python everywhere.
```

```text
Give me knowledge base stats: how many entries, which sources, and my top tags.
```

## Search syntax

`search_kb` passes your query to SQLite FTS5, so operators work:

| Query | Meaning |
|---|---|
| `stdio transport` | entries containing both words |
| `"stdio transport"` | that exact phrase |
| `mcp OR sourdough` | either word |
| `mcp NOT health` | `mcp` but not `health` |
| `protocol*` | prefix match |

If a query contains malformed FTS5 syntax, the server retries it as literal quoted phrases rather than failing, so unusual punctuation never produces an error.

## Claude Desktop / Cursor configs

- `claude_desktop_config.kb.example.json`
- `cursor-mcp.kb.example.json`

You can run all four MCP servers together by listing each under `mcpServers`.

## Files

- `kb_server.py` — MCP server with SQLite FTS5 search
- `kb_agent.py` — Ollama bridge/agent
- `run-kb.sh` — launcher

## Safety notes

- The database and all writes stay inside `MCP_KB_DIR`.
- `ingest_notes` reads only the folders you explicitly pass to it.
- It reads `.md` and `.txt` files only, and never executes shell commands.
- Notes may contain personal information. Back up `kb.db` like any other data file.