# mcp-agent

> **SimpleMCP Agent** — Calendar & Task management with kernel-level scraping.

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                           USER / CLI                                │
└───────────────────────────────┬─────────────────────────────────────┘
                                │
                         ┌──────▼──────┐
                         │  MCP Host   │   mcp_host/host.py
                         │  (REPL/API) │
                         └──────┬──────┘
                                │
                         ┌──────▼──────┐
                         │    Agent    │   mcp_host/agent.py
                         │ (ReAct loop)│   DocMost-first system prompt
                         └──────┬──────┘
                                │
               ┌────────────────┼────────────────┐
               │                │                │
        ┌──────▼──────┐  ┌──────▼──────┐        │
        │  MCP Client │  │  Anthropic  │        │
        └──────┬──────┘  │    LLM      │        │
               │  stdio  └─────────────┘        │
        ┌──────▼────────────────────────┐        │
        │         MCP Server            │        │
        │  Tools                        │        │
        │  ├── calendar (list/search)   │        │
        │  ├── tasks (CRUD)             │        │
        │  └── docmost (source-of-truth)│        │
        └───────┬───────────────┬───────┘        │
                │               │                │
        ┌───────▼──────┐ ┌──────▼────────────┐   │
        │Kernel Scraper│ │ DocMost REST API  │   │
        │ (httpx+BS4)  │ │ integrations/     │   │
        └───────┬──────┘ │ docmost/client.py │   │
                │        └──────────────────-┘   │
        ┌───────▼──────┐          │               │
        │Calendar Feed │ ┌────────▼──────────┐    │
        │ (ICS / HTML) │ │ DocMost Instance  │    │
        └──────────────┘ │ (self-hosted wiki)│    │
                         └───────────────────┘    │
```

---

## Layer Responsibilities

| Layer | Module | Role |
|---|---|---|
| **Config** | `config/` | Pydantic settings, `.env` loading |
| **Scraper — Kernel** | `scraper/kernel/` | Raw async HTTP, no browser |
| **Scraper — Session** | `scraper/kernel/session_manager.py` | Cookie jar, auth tokens, idle recycling |
| **Scraper — Time** | `scraper/kernel/time_manager.py` | Rate limiting, politeness delays |
| **Scraper — Parser** | `scraper/parsers/calendar_parser.py` | BS4 kernel parsing (JSON-LD → Microdata → data-* → ICS) |
| **DocMost Client** | `integrations/docmost/client.py` | REST API client for DocMost wiki |
| **DocMost Models** | `integrations/docmost/models.py` | Page, Space, SearchResult dataclasses |
| **DocMost Content** | `integrations/docmost/content.py` | ProseMirror JSON → Markdown converter |
| **MCP Server** | `mcp_server/` | Tool & resource registration, MCP protocol |
| **MCP Client** | `mcp_client/` | Server connection, tool invocation, schema conversion |
| **LLM** | `llm/` | Abstract BaseLLM + Anthropic implementation |
| **Agent** | `mcp_host/agent.py` | ReAct loop; DocMost-first system prompt |
| **Host** | `mcp_host/host.py` | Lifecycle, REPL, programmatic API |
| **Use Case** | `use_cases/calendar_tasks/` | Demo workflow, entry point |

---

## DocMost — Source of Truth

DocMost is the team wiki. The agent is instructed to treat it as the **authoritative
source of truth** for all factual questions. The integration has two modes:

### Mode 1 — REST API (default, any DocMost version)

```
DOCMOST_BASE_URL=https://docs.yourcompany.com
DOCMOST_API_KEY=<personal API key from Settings > API keys>
DOCMOST_DEFAULT_SPACE_ID=<optional space UUID>
```

The REST layer (`integrations/docmost/`) handles:
- Full-text search → `POST /api/search`
- Page read (ProseMirror JSON → Markdown) → `POST /api/pages/info`
- Page create/update (Markdown input) → `POST /api/pages/create`, `/api/pages/update`
- List spaces and pages → `GET /api/spaces`, `POST /api/pages`

### Mode 2 — Native MCP (DocMost enterprise)

DocMost exposes its own MCP endpoint at `{BASE_URL}/mcp`. If you have an
enterprise license and MCP enabled (Settings > AI & MCP > MCP tab), set:

```
DOCMOST_USE_NATIVE_MCP=true
```

Then add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "docmost": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://docs.yourcompany.com/mcp",
               "--header", "Authorization: Bearer YOUR_API_KEY"]
    }
  }
}
```

### DocMost tools exposed via MCP

| Tool | Description |
|---|---|
| `docmost_search` | **Primary source-of-truth lookup** — call before answering factual questions |
| `docmost_get_page` | Retrieve full page content as Markdown |
| `docmost_list_spaces` | Discover available knowledge spaces |
| `docmost_list_pages` | Browse pages in a space (paginated) |
| `docmost_create_page` | Write new wiki page (meeting notes, summaries) |
| `docmost_update_page` | Append or replace content on existing page |
| `docmost_delete_page` | Remove a page |

### Agent source-of-truth flow

```
User asks a question
        │
        ▼
 docmost_search("keywords from question")
        │
        ├─ Results found → docmost_get_page(page_id) → ground answer in wiki
        │
        └─ No results → fall back to calendar/task data + flag as inferred
        │
        ▼
 Produce response citing wiki page title + URL
        │
        ▼
 (Optional) docmost_create_page / docmost_update_page to persist findings
```

---

---

## Kernel Scraping vs Layout Scraping

This project deliberately uses **kernel-level scraping** — we operate at the
semantic/structural layer of HTML, not the visual/CSS layer.

| Approach | Targets | Stability |
|---|---|---|
| **Layout scraping** | CSS classes, visual position | ❌ Breaks on every redesign |
| **Kernel scraping** ✅ | JSON-LD, Microdata, `data-*`, ICS, ARIA | ✅ Survives redesigns |

### Parser strategy (priority order)

```
1. JSON-LD        <script type="application/ld+json"> Event schema
2. Microdata      itemtype="https://schema.org/Event"
3. data-* attrs   data-event-id, data-start-time, data-event-title …
4. ICS/iCal       Raw text; no external library needed
```

---

## Tools exposed by the MCP Server

| Tool | Description |
|---|---|
| `list_events` | Upcoming events within N days |
| `get_event_by_uid` | Single event by UID |
| `search_events` | Full-text search across title + description |
| `get_todays_schedule` | Today's events |
| `create_task` | Create a task (optionally linked to an event) |
| `list_tasks` | List tasks with status/priority filters |
| `update_task` | Update any task field |
| `delete_task` | Delete a task |

---

## Quick Start

```bash
# 1. Install
bash scripts/install.sh

# 2. Configure
#    Edit .env — set ANTHROPIC_API_KEY and CALENDAR_FEED_URL

# 3. Run interactive agent
source .venv/bin/activate
mcp-agent

# 4. Or run the demo sequence
mcp-agent --demo

# 5. Or run just the MCP server (for use with Claude Desktop etc.)
mcp-server
```

---

## Project Structure

```
mcp-agent/
├── config/                     # Pydantic settings
│   └── settings.py
│
├── integrations/               # ← NEW: third-party integrations
│   └── docmost/
│       ├── client.py           # DocMost REST API async client
│       ├── models.py           # Page, Space, SearchResult dataclasses
│       └── content.py          # ProseMirror JSON → Markdown converter
│
├── scraper/                    # Kernel scraper layer
│   ├── kernel/
│   │   ├── http_client.py
│   │   ├── session_manager.py
│   │   └── time_manager.py
│   └── parsers/
│       └── calendar_parser.py
│
├── mcp_server/                 # MCP Server
│   ├── server.py
│   ├── tools/
│   │   ├── calendar.py
│   │   ├── tasks.py
│   │   └── docmost.py          # ← NEW: DocMost tool handlers
│   └── resources/
│       └── calendar_resource.py
│
├── mcp_client/
│   └── client.py
│
├── llm/
│   ├── base.py
│   └── anthropic_llm.py
│
├── mcp_host/
│   ├── agent.py                # DocMost-first system prompt
│   └── host.py
│
├── use_cases/
│   └── calendar_tasks/
│       └── workflow.py
│
├── tests/
│   ├── test_scraper.py
│   ├── test_session_time.py
│   ├── test_tasks.py
│   └── test_docmost.py         # ← NEW
│
├── scripts/install.sh
├── pyproject.toml
└── .env.example
```

---

## Running Tests

```bash
pytest tests/ -v
```

All tests are offline — no network or API key required.

---

## Claude Desktop Integration

Add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "calendar-agent": {
      "command": "python",
      "args": ["/path/to/mcp-agent/mcp_server/server.py"],
      "env": {
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "CALENDAR_FEED_URL": "https://your-calendar.example.com/feed.ics"
      }
    }
  }
}
```