Pathfinder MCP Server

# TOON Format Investigation ## Overview TOON (Token-Optimized Output Notation) is a proposed format for reducing token consumption in LLM responses while maintaining information fidelity. This document summarizes research findings and applicability to Pathfinder MCP. ## Current State TOON is not yet a standardized format. The concept emerges from the need to: 1. **Reduce output tokens** - Shorter responses cost less and are faster 2. **Maintain structure** - Keep information organized and parseable 3. **Preserve semantics** - Ensure meaning is not lost in compression ## Approaches Investigated ### 1. Abbreviated Syntax Using shortened keys and compact notation: ``` # Standard JSON (45 tokens) {"session_id": "abc123", "phase": "research", "status": "completed"} # TOON-style (28 tokens) {sid:"abc123",ph:"res",st:"done"} ``` **Pros**: 30-40% token reduction **Cons**: Requires schema awareness, less human-readable ### 2. Structured Compression Using delimiters and positional encoding: ``` # Standard Session: abc123 Phase: research Status: completed Tokens: 1500 # TOON-style abc123|research|completed|1500 ``` **Pros**: 50%+ reduction for structured data **Cons**: Brittle, requires strict ordering ### 3. Semantic Compression Using domain-specific abbreviations: ``` # Standard MCP response The session has been created with ID 'abc123'. Current phase is 'research'. Please use save_research to document findings. # Compressed Session abc123 created. Phase: research. Next: save_research(findings). ``` **Pros**: Natural language compatible, 40-60% reduction **Cons**: Requires consistent abbreviation rules ## Recommendations for Pathfinder ### Short-term (v1) 1. **Concise tool responses** - Return minimal but complete information 2. **Use codes over messages** - `"code": "SESSION_NOT_FOUND"` vs verbose error 3. **Structured artifacts** - YAML frontmatter enables efficient parsing ### Medium-term (v2) 1. **Response templates** - Pre-defined compact response formats 2. **Context-aware verbosity** - Verbose for errors, terse for success 3. **Batch operations** - Combine multiple updates into single response ### Long-term (v3) 1. **Custom MCP response format** - If TOON standardizes, implement adapter 2. **Streaming compression** - Compress during SSE transport 3. **Client negotiation** - Let client request verbosity level ## Compatibility Considerations ### Cursor Client - Cursor parses JSON responses directly - No known support for custom compression formats - Best approach: structured JSON with minimal keys ### MCP Protocol - Protocol expects standard JSON-RPC responses - Custom formats would need wrapper/adapter - Resources and prompts can use any text format ## Token Savings Estimate For typical Pathfinder session: | Operation | Standard | Optimized | Savings | |-----------|----------|-----------|---------| | start_research | ~150 tokens | ~80 tokens | 47% | | save_research | ~200 tokens | ~100 tokens | 50% | | start_plan | ~180 tokens | ~90 tokens | 50% | | implement_phase | ~250 tokens | ~120 tokens | 52% | **Average savings: ~50% per tool response** ## Implementation Status - [x] Concise error codes implemented - [x] Minimal response structures - [x] Context status as compact dict - [ ] Response templates (future) - [ ] TOON adapter (pending standardization) ## References - Context engineering principles (Anthropic) - Token optimization patterns (OpenAI Cookbook) - MCP Protocol specification