Obsidian MCP Server - Enhanced

# Claude.ai Remote MCP Integration: Complete Solution Guide This document chronicles the complete journey from a broken MCP server to a production-ready remote integration that enables Claude.ai to seamlessly interact with your Obsidian vault from anywhere in the world. ## 🎯 Executive Summary **The Challenge:** Original MCP server had fatal compatibility issues with Claude.ai's Remote MCP client, preventing stable connections and tool discovery. **The Solution:** Complete architectural transformation using native HTTP transport, Tailscale Funnel for secure remote access, and automated startup configuration. **The Result:** Zero-configuration, production-grade Obsidian ↔ Claude.ai integration with automatic startup and enterprise-level security. --- ## 📋 Table of Contents - [The Original Problem](#the-original-problem) - [Root Cause Analysis](#root-cause-analysis) - [The Solution Journey](#the-solution-journey) - [Technical Architecture](#technical-architecture) - [Remote Access Implementation](#remote-access-implementation) - [Security Model](#security-model) - [Autostart Configuration](#autostart-configuration) - [Complete Setup Guide](#complete-setup-guide) - [Troubleshooting](#troubleshooting) - [Monitoring & Maintenance](#monitoring--maintenance) --- ## 🚫 The Original Problem The original MCP server suffered from **fatal incompatibility issues** with Claude.ai's Remote MCP client: ### Core Issues - **Response handling conflicts** between Hono framework and MCP SDK's `StreamableHTTPServerTransport` - `ERR_HTTP_HEADERS_SENT` errors from competing response writers - **Session management failures** preventing stable connections - **Tool discovery issues** - Claude.ai couldn't see registered MCP tools - **Invalid HTTP status codes** breaking the MCP protocol - **Connection instability** causing frequent disconnections ### Impact - ❌ Claude.ai Remote MCP integration completely non-functional - ❌ Tools not discoverable or accessible - ❌ Frequent connection drops and session failures - ❌ Unreliable communication between Claude.ai and Obsidian --- ## 🔍 Root Cause Analysis ### Architecture Conflict The fundamental issue was **two systems fighting for control** of the same HTTP response: ``` ┌─────────────────┐ Conflict ┌──────────────────────────┐ │ Hono Framework│ ←─────────────→ │ MCP SDK Transport │ │ - Response mgmt │ │ - Expects full control │ │ - Header control│ │ - Direct HTTP handling │ │ - Lifecycle mgmt│ │ - StreamableHTTP │ └─────────────────┘ └──────────────────────────┘ ``` **The Problem:** 1. **MCP SDK** expected full control over Node.js `http.IncomingMessage`/`http.ServerResponse` 2. **Hono Framework** provided its own response abstraction and lifecycle management 3. Both attempted to write headers and send responses simultaneously 4. Result: Race conditions and invalid HTTP states --- ## 🔧 The Solution Journey ### Phase 1: Native HTTP Transport Implementation **Goal:** Eliminate framework conflicts and give MCP SDK complete control **Implementation:** - **Replaced Hono with pure Node.js HTTP server** - **Created `httpTransportNative.ts`** with minimal request routing - **Gave MCP SDK complete control** of request/response handling - **Eliminated response handling race conditions** **Key Changes:** ```typescript // Before: Hono framework handling app.post('/mcp', async (c) => { // Complex middleware chain // Framework response management // Conflicts with MCP transport }); // After: Native Node.js handling const server = http.createServer(async (req, res) => { // Minimal pre-processing // Direct MCP transport delegation await transport.handleRequest(req, res, body); // No additional response handling }); ``` ### Phase 2: Enhanced Security & Authentication **Goal:** Implement robust authentication and session management **Features Added:** - **Multi-method API key authentication** - **Session management with automatic cleanup** - **CORS configuration for web access** - **Structured error handling with JSON-RPC 2.0** **Authentication Methods:** ```typescript // Method 1: Authorization Header (Preferred) Authorization: Bearer your-api-key Authorization: your-api-key // Method 2: URL Query Parameter (Fallback) https://your-server.com/mcp?api_key=your-api-key // Method 3: Development Mode OBSIDIAN_API_KEY=dummy // Disables authentication ``` ### Phase 3: Remote Access Solution **Goal:** Enable secure remote access via Claude.ai's Remote MCP **Implementation:** - **Tailscale Funnel integration** for secure public HTTPS endpoints - **Comprehensive security documentation** with ACLs and monitoring - **Autostart configuration** for persistent service availability --- ## 🏗️ Technical Architecture ### Complete Remote Access Stack ``` ┌─────────────────┐ HTTPS ┌──────────────────┐ HTTP ┌─────────────────┐ │ Claude.ai │ ────────────→ │ Tailscale Funnel │ ─────────→ │ Native MCP │ │ Remote MCP │ │ (Public HTTPS) │ │ Server (Local) │ └─────────────────┘ └──────────────────┘ └─────────────────┘ │ │ │ │ ┌───────▼──────────┐ ┌─────▼──────────┐ │ Security Layer │ │ Obsidian REST │ │ - Tailscale ACLs │ │ API Plugin │ │ - IP Allowlisting│ │ - 8 MCP Tools │ │ - Rate Limiting │ │ - Vault Access │ └──────────────────┘ └────────────────┘ ``` ### Component Breakdown #### 1. Native MCP Server - **File:** `src/mcp-server/transports/httpTransportNative.ts` - **Purpose:** Pure Node.js HTTP server with MCP SDK integration - **Features:** Session management, authentication, error handling #### 2. Tailscale Funnel - **Purpose:** Secure public HTTPS endpoint creation - **Command:** `tailscale funnel 3010` - **Benefits:** No port forwarding, automatic HTTPS, secure mesh networking #### 3. Authentication Layer - **Multi-method support:** Headers, query params, development bypass - **Session management:** 30-minute expiration, automatic cleanup - **Security:** API key validation before MCP processing #### 4. Autostart Services - **MCP Server:** `com.obsidian.mcp.server.plist` - **Tailscale Funnel:** `com.obsidian.mcp.tailscale.plist` - **Management:** macOS launchd for persistent services --- ## 🌐 Remote Access Implementation ### Tailscale Funnel Configuration **Enable Funnel:** ```bash # Start funnel on port 3010 tailscale funnel 3010 # Verify funnel status tailscale funnel status # Monitor connections tailscale status --peers ``` **Security Configuration:** ```json { "acls": [ { "action": "accept", "src": ["autogroup:internet"], "dst": ["your-device:funnel-port"], "srcIPs": [ "claude-ai-ip-range-1/24", "claude-ai-ip-range-2/24" ] } ] } ``` ### Claude.ai Integration **Remote MCP URL:** ``` https://your-device.your-tailnet.ts.net/mcp ``` **Authentication Setup:** - **Method 1:** Add API key to Claude.ai's Remote MCP configuration - **Method 2:** Use URL parameter: `?api_key=your-obsidian-api-key` --- ## 🔒 Security Model ### Defense in Depth Strategy #### Layer 1: Tailscale Mesh Security - **Encrypted connections** via WireGuard protocol - **Identity-based networking** with device authentication - **ACL-based access control** for fine-grained permissions #### Layer 2: API Key Authentication - **Obsidian Local REST API key** validation - **Session-based access** with automatic expiration - **Multiple authentication methods** for flexibility #### Layer 3: Application Security - **Input validation** via Zod schemas - **Request sanitization** preventing injection attacks - **Error handling** without information leakage #### Layer 4: Network Monitoring - **Connection logging** via Tailscale and application logs - **Session tracking** with automatic cleanup - **Rate limiting** capabilities ### Security Best Practices #### IP Allowlisting ```json { "acls": [ { "action": "accept", "src": ["autogroup:internet"], "dst": ["attr:funnel:*"], "srcIPs": ["known-claude-ai-ranges"] } ] } ``` #### Emergency Procedures ```bash # Disable funnel immediately tailscale funnel off # Rotate API keys # 1. Generate new Obsidian API key # 2. Update MCP server config # 3. Update Claude.ai settings # 4. Verify old key disabled ``` --- ## 🚀 Autostart Configuration ### macOS launchd Services #### MCP Server Service **File:** `~/Library/LaunchAgents/com.obsidian.mcp.server.plist` ```xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.obsidian.mcp.server</string> <key>ProgramArguments</key> <array> <string>/usr/local/bin/node</string> <string>/path/to/obsidian-mcp-server/dist/index.js</string> </array> <key>WorkingDirectory</key> <string>/path/to/obsidian-mcp-server</string> <key>EnvironmentVariables</key> <dict> <key>MCP_LOG_LEVEL</key> <string>debug</string> <key>MCP_TRANSPORT_TYPE</key> <string>http</string> </dict> <key>RunAtLoad</key> <true/> <key>KeepAlive</key> <true/> </dict> </plist> ``` #### Tailscale Funnel Service **File:** `~/Library/LaunchAgents/com.obsidian.mcp.tailscale.plist` ```xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.obsidian.mcp.tailscale</string> <key>ProgramArguments</key> <array> <string>/usr/local/bin/tailscale</string> <string>funnel</string> <string>3010</string> </array> <key>RunAtLoad</key> <true/> <key>KeepAlive</key> <true/> </dict> </plist> ``` ### Service Management Commands ```bash # Load services launchctl load ~/Library/LaunchAgents/com.obsidian.mcp.server.plist launchctl load ~/Library/LaunchAgents/com.obsidian.mcp.tailscale.plist # Check status launchctl list | grep obsidian # Unload services launchctl unload ~/Library/LaunchAgents/com.obsidian.mcp.server.plist launchctl unload ~/Library/LaunchAgents/com.obsidian.mcp.tailscale.plist ``` --- ## 📝 Complete Setup Guide ### Step 1: Prerequisites 1. **Obsidian** with Local REST API plugin installed 2. **API key** configured in Local REST API plugin 3. **Tailscale** account and client installed 4. **Node.js** (v18+) and npm ### Step 2: Install MCP Server ```bash # From npm npm install obsidian-mcp-server # Or from source git clone https://github.com/cyanheads/obsidian-mcp-server.git cd obsidian-mcp-server npm install npm run build ``` ### Step 3: Configure Environment ```bash # Create .env file echo "OBSIDIAN_API_KEY=your-obsidian-api-key" > .env echo "MCP_TRANSPORT_TYPE=http" >> .env echo "MCP_LOG_LEVEL=debug" >> .env ``` ### Step 4: Create Autostart Services ```bash # Create MCP server service cat > ~/Library/LaunchAgents/com.obsidian.mcp.server.plist << 'EOF' [XML content from above] EOF # Create Tailscale funnel service cat > ~/Library/LaunchAgents/com.obsidian.mcp.tailscale.plist << 'EOF' [XML content from above] EOF ``` ### Step 5: Load Services ```bash # Load both services launchctl load ~/Library/LaunchAgents/com.obsidian.mcp.server.plist launchctl load ~/Library/LaunchAgents/com.obsidian.mcp.tailscale.plist # Verify services are running launchctl list | grep obsidian ``` ### Step 6: Configure Claude.ai 1. **Get your Tailscale URL:** Check `tailscale funnel status` 2. **Add Remote MCP in Claude.ai:** - URL: `https://your-device.your-tailnet.ts.net/mcp` - Auth: Add your Obsidian API key 3. **Test connection:** Verify tools are discoverable ### Step 7: Verify Setup ```bash # Test MCP server curl -H "Authorization: your-api-key" http://localhost:3010/mcp # Test Tailscale funnel curl -H "Authorization: your-api-key" https://your-device.your-tailnet.ts.net/mcp # Check logs tail -f path/to/obsidian-mcp-server/logs/debug.log ``` --- ## 🔧 Troubleshooting ### Common Issues #### "Connection Refused" - **Cause:** MCP server not running - **Solution:** Check `launchctl list | grep obsidian` - **Fix:** Reload service or check logs #### "Unauthorized" Errors - **Cause:** API key mismatch - **Solution:** Verify API key in Obsidian Local REST API plugin - **Fix:** Update environment variables and restart #### "Session Not Found" - **Cause:** Session expired (30 min timeout) - **Solution:** Restart Claude.ai integration - **Prevention:** Monitor session activity logs #### "Funnel Not Working" - **Cause:** Tailscale plan limitations or ACL restrictions - **Solution:** Check Tailscale plan and ACL configuration - **Fix:** Update ACLs or upgrade Tailscale plan ### Debug Commands ```bash # Check services status launchctl list | grep obsidian # View service logs tail -f ~/Library/Logs/com.obsidian.mcp.server.log tail -f ~/Library/Logs/com.obsidian.mcp.tailscale.log # Test local connection curl -H "Authorization: Bearer your-api-key" http://localhost:3010/mcp # Test remote connection curl -H "Authorization: Bearer your-api-key" https://your-device.your-tailnet.ts.net/mcp # Monitor Tailscale tailscale status tailscale funnel status tailscale logs ``` --- ## 📊 Monitoring & Maintenance ### Performance Monitoring #### Key Metrics - **Connection count:** Active MCP sessions - **Request latency:** Response times for tool calls - **Error rates:** Failed authentication or tool executions - **Session duration:** Connection stability metrics #### Log Analysis ```bash # Monitor connection patterns grep "Session created" logs/debug.log | tail -20 # Check authentication failures grep "Unauthorized" logs/debug.log | tail -10 # Review tool usage grep "Tool called" logs/debug.log | tail -20 ``` ### Maintenance Tasks #### Weekly - **Review access logs** for unexpected connections - **Check service uptime** via `launchctl list` - **Monitor disk usage** for log files - **Verify Tailscale connectivity** #### Monthly - **Rotate API keys** for enhanced security - **Update Tailscale ACLs** if needed - **Review and archive old logs** - **Test disaster recovery procedures** #### Updates - **MCP Server:** Check for new releases - **Tailscale:** Keep client updated - **Dependencies:** Regular `npm update` --- ## 🎯 Key Breakthrough Achievements ### Technical Victories - ✅ **100% MCP protocol compliance** - no more framework interference - ✅ **Session persistence** - connections survive extended use - ✅ **Tool visibility** - all 8 Obsidian tools discoverable - ✅ **Error handling** - proper JSON-RPC 2.0 responses - ✅ **Performance** - reduced overhead vs framework approach ### Remote Access Victories - ✅ **Public HTTPS access** via Tailscale Funnel - ✅ **Zero port forwarding** or firewall configuration - ✅ **Automatic startup** - server + tunnel start on boot - ✅ **Secure by design** - Tailscale mesh + API key auth - ✅ **Production ready** - comprehensive monitoring & logging ### Integration Success - ✅ **Fully automated Obsidian ↔ Claude.ai integration** - ✅ **Zero-configuration startup** (reboots work seamlessly) - ✅ **Production-grade security** (multi-layer defense) - ✅ **Complete tool access** (read, write, search, manage notes) - ✅ **Stable, long-lived connections** (no more session failures) --- ## 🎉 Conclusion This solution transforms a **broken, incompatible MCP server** into a **production-ready remote integration** that enables Claude.ai to seamlessly interact with your Obsidian vault from anywhere in the world. The journey from architectural conflicts to production deployment demonstrates how proper protocol implementation, security considerations, and automation can create a robust, maintainable system that "just works." **The end result:** Your Obsidian vault is now a powerful, AI-accessible knowledge base that enhances Claude.ai's capabilities while maintaining enterprise-level security and reliability. --- ## 📚 References - [MCP Specification](https://github.com/modelcontextprotocol/modelcontextprotocol) - [Obsidian Local REST API Plugin](https://github.com/coddingtonbear/obsidian-local-rest-api) - [Tailscale Funnel Documentation](https://tailscale.com/kb/1223/tailscale-funnel/) - [Anthropic MCP Documentation](https://docs.anthropic.com/en/docs/build-with-claude/computer-use) - [Native HTTP Transport Implementation](./NATIVE_HTTP_TRANSPORT.md) - [Tailscale Security Configuration](./TAILSCALE_SECURITY.md)