# Andalusia AI MCP Server

This MCP server exposes six Andalusia AI hospital metric tools to Claude:

1. `resolve_entities`
2. `search_members`
3. `query_cube`
4. `get_measure_definition`
5. `resolve_why_question`
6. `execute_investigation`

It is a thin wrapper around two FastAPI services:

- Librarian: `http://197.164.100.18:18080`
- Tools: `http://197.164.100.18:28081`
- Investigator: `http://197.164.100.18:28082`

This repository itself is not Dockerized. In production, this MCP service runs
as a Node.js process under systemd. The Andalusia backend services it calls may
run separately in Docker on the same server. For example, on the current host:

```text
MCP service                 systemd: andalusia-ai-mcp.service
Librarian backend           Docker: librarian-resolver-1
Tools backend               Docker: claude_project-tools-1
Librarian vector database   Docker: librarian-qdrant-1
```

When debugging a tool failure, check both MCP logs and the relevant backend
container logs.

Backend calls use the shared `X-Librarian-Secret` header. Keep the real secret in `.env`; do not commit it.
Non-secret app settings live in `config.yml`.
Tool descriptions live in `tool-descriptions.yml`.

## Project Layout

```text
config.yml                  Non-secret runtime settings
.env                        Secrets only
tool-descriptions.yml       Editable Claude-facing tool guidance
src/config.ts               Loads config.yml plus env overrides
src/toolDescriptions.ts     Loads tool-descriptions.yml
src/andalusiaClient.ts      Backend API client
src/mcpServer.ts            Creates the MCP server and registers tools
src/tools/index.ts          Tool registry: controls which tools are enabled
src/tools/*.ts              One file per MCP tool
```

## Setup

```bash
npm install
cp config.example.yml config.yml
cp .env.example .env
npm run build
```

Edit `config.yml` for non-secret settings:

```yaml
andalusiaLibrarianBaseUrl: http://197.164.100.18:18080
andalusiaToolsBaseUrl: http://197.164.100.18:28081
andalusiaInvestigatorBaseUrl: http://197.164.100.18:28082
andalusiaRequestTimeoutMs: 100000
andalusiaInvestigatorRequestTimeoutMs: 500000
monitoringEventLogFile: /var/log/andalusia-ai-mcp/monitoring-events.jsonl
monitoringEventLogMaxBytes: 52428800
monitoringEventLogReplayMaxBytes: 10485760
monitoringEventLogReplayMaxEvents: 10000
monitoringEventLogReplayMaxAgeMs: 86400000
monitoringMaxMetricBuckets: 200
oauthStateFile: /var/lib/andalusia-ai-mcp/oauth-state.json
oauthMaxClients: 1000
oauthMaxAuthorizationCodes: 1000
oauthMaxAccessTokens: 5000
oauthMaxRefreshTokens: 5000
```

Edit `.env` for secrets:

```bash
ANDALUSIA_LIBRARIAN_SECRET=your-shared-secret
```

Environment variables still override `config.yml`, which is useful for one-off
local runs and deployment systems.

## Adding Tools

Each MCP tool lives in its own file under `src/tools/` and implements the
`AndalusiaTool` interface from `src/tools/types.ts`.

To add a tool:

1. Add the backend method to `src/andalusiaClient.ts`.
2. Add a new `src/tools/<toolName>.ts` file with its `name`, `title`,
   `descriptionKey`, `inputSchema`, and `handler`.
3. Add the tool factory to `src/tools/index.ts`. This is the registry that
   decides which tools the MCP server exposes.
4. Add matching editable guidance to `tool-descriptions.yml`.

Example registry entry:

```ts
export function createAndalusiaTools(context: ToolContext): AnyAndalusiaTool[] {
  return [
    createResolveEntitiesTool(context),
    createSearchMembersTool(context),
    createQueryCubeTool(context),
    createGetMeasureDefinitionTool(context),
    createResolveWhyQuestionTool(context),
    createExecuteInvestigationTool(context)
  ];
}
```

If a tool is not added to `src/tools/index.ts`, it will not be registered with
the MCP server even if its file exists.

## Guides

- [Andalusia tools](docs/andalusia-tools.md)
- [Connecting to Claude](docs/connect-claude.md)
- [Deploy on HTTPS 443](docs/deploy-443.md)
- [CI pipeline](docs/ci.md)
- [Backend troubleshooting](docs/backend-troubleshooting.md)

## HTTP Mode

```bash
MCP_TRANSPORT=http MCP_HTTP_PORT=3000 npm start
```

Then connect Claude with Streamable HTTP:

```bash
claude mcp add --transport http andalusia-ai http://127.0.0.1:3000/mcp
```

For hosted Claude connector usage, set `MCP_PUBLIC_URL` to the public `/mcp`
URL and set `MCP_HTTP_BEARER_TOKEN`. Claude discovers OAuth metadata, registers
dynamically, and prompts for that token as the connector code during sign-in.

## Production Monitoring Dashboard

HTTP mode exposes built-in production monitoring endpoints:

- Production dashboard: `https://ai-mcp.andalusiagroup.net/dashboard`
- `/dashboard` - human-facing browser UI for production monitoring. Open this
  page to inspect MCP usage, tool calls, latency, failures, uptime, memory,
  backend configuration, JSON-RPC methods, and recent errors.
- `/dashboard/metrics` - JSON API used by the dashboard. Use this endpoint for
  debugging, scripts, smoke tests, or custom internal tools that need the full
  structured monitoring snapshot.
- `/metrics` - Prometheus-compatible text endpoint. Use this endpoint as the
  scrape target for Prometheus, Grafana, Datadog agents, or other monitoring
  systems.
- `/ready` - readiness check for required Andalusia backend configuration.

Quick reference:

```text
/dashboard          Browser UI for humans
/dashboard/metrics  JSON monitoring snapshot for the dashboard and scripts
/metrics            Prometheus-compatible metrics for monitoring platforms
/ready              Readiness check for deploy/load-balancer health gates
```

Application logs are written to stdout/stderr and captured by systemd. Enable
persistent journald storage on the production host so `journalctl` logs survive
reboots:

```bash
sudo install -d -m 2755 -o root -g systemd-journal /var/log/journal
sudo systemctl restart systemd-journald
```

When `MCP_HTTP_BEARER_TOKEN` is configured, `/dashboard/metrics` and `/metrics`
require the same bearer token:

```bash
curl -H "Authorization: Bearer $MCP_HTTP_BEARER_TOKEN" \
  https://ai-mcp.andalusiagroup.net/dashboard/metrics

curl -H "Authorization: Bearer $MCP_HTTP_BEARER_TOKEN" \
  https://ai-mcp.andalusiagroup.net/metrics
```

The dashboard keeps bounded working counters in memory. On startup, it replays
recent bounded history from the JSONL event log, then continues with live
updates.
When `monitoringEventLogFile` is configured, each HTTP request, tool call, and
server error is also appended to a bounded JSONL event log. The file rotates to
`.1` when it reaches `monitoringEventLogMaxBytes`.

The dashboard does not depend only on the current process runtime state. Runtime
counters are used while the service is running, but after a restart the
dashboard rebuilds recent monitoring results from the persistent JSONL event log
on disk before continuing with live updates.

Read the live monitoring event log:

```bash
sudo tail -f /var/log/andalusia-ai-mcp/monitoring-events.jsonl
```

Read recent monitoring events:

```bash
sudo tail -n 100 /var/log/andalusia-ai-mcp/monitoring-events.jsonl
```

Pretty-print recent JSONL events:

```bash
sudo tail -n 50 /var/log/andalusia-ai-mcp/monitoring-events.jsonl | jq
```

Read the previous rotated monitoring log:

```bash
sudo tail -n 100 /var/log/andalusia-ai-mcp/monitoring-events.jsonl.1
```

Read application stdout/stderr logs captured by systemd:

```bash
sudo journalctl -u andalusia-ai-mcp -n 200
sudo journalctl -u andalusia-ai-mcp -f
sudo journalctl -u andalusia-ai-mcp --since today
```

For long-term retention and alerting, scrape `/metrics` with your monitoring
platform and alert on server errors, tool failures, high P95 latency, missing
readiness checks, and process restarts. Use the JSONL event log or journald for
forensics and recent dashboard replay after process restarts.

Update needed: add an explicit retention policy for filesystem logs. The MCP
monitoring event log currently rotates by size, not by age. For normal
operations, keep raw monitoring and journald logs for about 30 days unless a
compliance requirement says otherwise. A production `logrotate` policy can be
used for the JSONL file, for example:

```text
/var/log/andalusia-ai-mcp/monitoring-events.jsonl {
  daily
  rotate 30
  compress
  missingok
  notifempty
  copytruncate
  create 0640 ai ai
}
```

Also configure journald retention on the host, for example with
`MaxRetentionSec=30day` in `journald.conf` if that matches the server policy.

OAuth clients, access tokens, and refresh tokens can be persisted with
`oauthStateFile`. The service still keeps bounded working maps in memory while
running, but the configured caps prevent unbounded growth and the state file
allows Claude connector sessions to survive process restarts.

Tool usage rows are driven by the MCP tool registry in `src/tools/index.ts`.
When a new tool is added to `createAndalusiaTools`, the dashboard automatically
includes it and starts its call count at `0` after the service restarts.

## Troubleshooting Backend Failures

The MCP durable monitoring log shows MCP-level failures:

```bash
sudo jq -r 'select(.type=="tool_call" and .success==false) | [.ts, .toolName, .durationMs, .error] | @tsv' \
  /var/log/andalusia-ai-mcp/monitoring-events.jsonl
```

Example:

```text
resolve_entities  librarian POST /resolve_entities failed with HTTP 500
```

That means the MCP service called the Librarian backend and the backend returned
HTTP 500. To find the root cause, check the Librarian Docker logs:

```bash
docker logs librarian-resolver-1 2>&1 | rg -C 5 '402 Payment Required|openrouter.ai/api/v1/embeddings|HTTPStatusError'
```

If Docker requires sudo:

```bash
sudo docker logs librarian-resolver-1 2>&1 | grep -C 5 "402 Payment Required"
```

A known failure is:

```text
httpx.HTTPStatusError: Client error '402 Payment Required'
for url 'https://openrouter.ai/api/v1/embeddings'
```

This is not a Claude quota error. It means the Librarian backend's OpenRouter
embedding provider returned `402 Payment Required`, usually because OpenRouter
billing/credits/API-key access needs attention.

## Publish To Production

Run from the project directory on the production host:

```bash
cd /home/ai/Workspace/yasser/Claude_Mcp
npm run typecheck
npm run build
```

Install the non-secret runtime files:

```bash
sudo install -d -m 750 -o root -g ai /etc/andalusia-ai-mcp
sudo install -d -m 750 -o ai -g ai /var/lib/andalusia-ai-mcp
sudo install -d -m 750 -o ai -g ai /var/log/andalusia-ai-mcp
sudo install -m 640 -o root -g ai config.yml /etc/andalusia-ai-mcp/config.yml
sudo install -m 640 -o root -g ai tool-descriptions.yml /etc/andalusia-ai-mcp/tool-descriptions.yml
```

Do not overwrite the production env file unless secrets changed. It should
already contain:

```bash
MCP_HTTP_BEARER_TOKEN=your-connector-code
ANDALUSIA_LIBRARIAN_SECRET=your-shared-secret
```

Install service/proxy config and restart:

```bash
sudo cp deploy/andalusia-ai-mcp.service /etc/systemd/system/andalusia-ai-mcp.service
sudo cp deploy/andalusia-ai-mcp.nginx.conf /etc/nginx/conf.d/andalusia-ai-mcp.conf
sudo systemctl daemon-reload
sudo nginx -t
sudo systemctl reload nginx
sudo systemctl reset-failed andalusia-ai-mcp
sudo systemctl restart andalusia-ai-mcp
sudo systemctl status andalusia-ai-mcp --no-pager -l
```

Verify:

```bash
curl -sS http://127.0.0.1:3001/health
curl -i https://ai-mcp.andalusiagroup.net/health
curl -i https://ai-mcp.andalusiagroup.net/.well-known/oauth-authorization-server
```

Expected health response:

```json
{"ok":true,"name":"andalusia-ai-mcp"}
```

## Public HTTPS

For `https://ai-mcp.andalusiagroup.net/mcp`, run the Node server privately on
`127.0.0.1:3001` and let Nginx terminate TLS on port `443`.

This repo is configured to use the existing `*.andalusiagroup.net` wildcard
certificate:

```text
/etc/nginx/certs/fullchain.crt
/etc/nginx/certs/private.key
```

Install the wildcard certificate and Nginx config on the server.
On this host, Docker already owns port `80`, so the systemd Nginx config is
443-only and does not provide an HTTP-to-HTTPS redirect.

```bash
sudo dnf install -y nginx
command -v nginx
sudo systemctl enable --now nginx

sudo install -d -m 700 -o root -g root /etc/nginx/certs
sudo install -m 644 /path/to/wildcard/fullchain.crt /etc/nginx/certs/fullchain.crt
sudo install -m 600 /path/to/wildcard/private.key /etc/nginx/certs/private.key

sudo cp deploy/andalusia-ai-mcp.nginx.conf /etc/nginx/conf.d/andalusia-ai-mcp.conf
sudo nginx -t
sudo systemctl reload nginx
```

If `command -v nginx` prints nothing or `nginx.service does not exist`, Nginx is
not installed on the host. Install it first, then rerun the Nginx commands.
If `install: cannot stat ...` appears, replace `/path/to/wildcard/...` with the
real certificate and private key paths.
Files under `/etc/pki/ca-trust/...` are system CA bundles, not the
`*.andalusiagroup.net` server certificate or its private key.
If `nginx` fails with `bind() to 0.0.0.0:80 failed` and `ss` shows
`docker-proxy`, port `80` is already handled by a Docker container. In that
case, remove the package default `listen 80` server from
`/etc/nginx/nginx.conf` and run systemd Nginx as a 443-only TLS proxy, or stop
the container before using systemd Nginx on both ports.

Verify locally and over HTTPS:

```bash
curl http://127.0.0.1:3001/health
curl -vk https://ai-mcp.andalusiagroup.net/mcp
```

For the full production checklist, including systemd and bearer-token setup, see
[Deploy on HTTPS 443](docs/deploy-443.md).