README.md•6.15 kB
# Token Analyzer MCP
[](https://github.com/cordlesssteve/token-analyzer-mcp/actions/workflows/ci.yml)
[](https://www.npmjs.com/package/token-analyzer-mcp)
[](https://opensource.org/licenses/MIT)
A tool for independent verification of token counts and context volume in Model Context Protocol (MCP) server configurations. Provides accurate token consumption analysis to validate and optimize your Claude Code setup.
## Purpose
This tool serves as an **independent verification system** for token usage in MCP configurations. Rather than relying on estimates or built-in metrics, it provides precise token counting and context volume analysis to:
- Verify actual token consumption vs. expected usage
- Validate context window utilization against the 200k token limit
- Identify servers consuming excessive tokens
- Provide independent confirmation of token optimization efforts
## Features
- **Independent Token Verification** - Accurate token counting separate from MCP server self-reporting
- **Context Volume Analysis** - Precise measurement of total context consumption
- **MCP Configuration Scanning** - Automatic discovery and analysis of configured servers
- **Overhead Calculation** - Breakdown of token usage by server and tool definitions
- **Verification Reports** - Detailed analysis for validating token usage claims
- **Quick Estimation Mode** - Fast overhead checking without full server connections
## Installation
```bash
npm install -g token-analyzer-mcp
```
## Usage
### Basic Verification
```bash
# Perform independent token count verification
token-analyzer-mcp analyze
# Quick context overhead verification
token-analyzer-mcp quick
# Validate MCP configuration setup
token-analyzer-mcp config
# Verify analyzer dependencies
token-analyzer-mcp doctor
```
### Verification Reports
```bash
# Generate detailed verification report
token-analyzer-mcp analyze \
--output verification.json \
--report token-analysis.txt \
--csv usage-breakdown.csv
```
## Configuration Discovery
The analyzer automatically discovers MCP configurations from standard locations:
- `~/.claude/claude_desktop_config.json`
- `~/.config/claude-desktop/claude_desktop_config.json`
- Windows: `%APPDATA%/Claude/claude_desktop_config.json`
- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
### Supported Configuration Format
```json
{
"mcpServers": {
"server-name": {
"command": "node",
"args": ["path/to/server.js"],
"disabled": false
}
}
}
```
## Sample Verification Output
```
Token Analyzer - Independent Verification v1.0.0
Verifying MCP server token consumption...
Phase 1: Configuration Discovery
✓ Found 6 servers in ~/.claude/claude_desktop_config.json
✓ 4 active servers to verify
Phase 2: Schema Extraction
✓ document-organizer: 12 tools extracted
✓ conversation-search: 15 tools extracted
✓ claude-telemetry: 8 tools extracted
Phase 3: Independent Token Verification
✓ Token verification complete
Verification Results:
Built-in Tools: 2,250 tokens (1.13%)
MCP Servers: 8,540 tokens (4.27%)
Total Overhead: 10,790 tokens (5.40%)
Available Context: 189,210 tokens (94.60%)
Status: ✅ Context usage within acceptable limits
```
## Commands
- `analyze` - Full independent verification with detailed breakdown
- `quick` - Fast context overhead verification without server connections
- `config` - Validate MCP configuration accessibility and format
- `doctor` - Verify analyzer dependencies and permissions
## Verification Accuracy
- **Connection Timeout**: 10 seconds per server to ensure accurate results
- **Retry Logic**: Up to 2 retry attempts for reliable token counting
- **Independent Counting**: Separate token calculation engine from MCP servers
- **Cross-Validation**: Multiple counting methods for accuracy verification
## Troubleshooting
### Configuration Not Found
```bash
# Verify configuration locations
token-analyzer-mcp doctor
# Check file permissions and accessibility
ls -la ~/.claude/claude_desktop_config.json
```
### Server Connection Issues
```bash
# Test individual server accessibility
node /path/to/server/index.js
# Run verification with debug output
token-analyzer-mcp analyze --debug
```
### Token Count Discrepancies
```bash
# Generate detailed verification report
token-analyzer-mcp analyze --summary
# Compare with previous verification results
token-analyzer-mcp analyze --report comparison.txt
```
## Development
```bash
# Clone repository
git clone https://github.com/cordlesssteve/token-analyzer-mcp.git
cd token-analyzer-mcp
# Install dependencies
npm install
# Run tests
npm test
# Run analyzer locally
node src/index.js analyze
```
## API Reference
### TokenVerificationEngine
- `performIndependentVerification()` - Execute comprehensive token verification
- `verifyServerTokens(servers)` - Independently verify token consumption
- `validateContextUsage(tokens)` - Check context volume against limits
- `generateVerificationReport(analysis)` - Create verification documentation
### ConfigurationValidator
- `discoverConfigurations()` - Find and validate MCP configuration files
- `verifyServerAccessibility(server)` - Test server connection and response
- `validateConfigurationFormat(config)` - Check configuration file structure
- `calculateBaselineOverhead()` - Measure built-in tool token consumption
## Contributing
1. Fork the repository
2. Create a feature branch: `git checkout -b feature-name`
3. Add verification improvements and tests
4. Run the test suite: `npm test`
5. Submit a pull request
## License
MIT License - see [LICENSE](LICENSE) file for details.
## Support
- [Issues](https://github.com/cordlesssteve/token-analyzer-mcp/issues)
- [Discussions](https://github.com/cordlesssteve/token-analyzer-mcp/discussions)
- [Documentation](https://github.com/cordlesssteve/token-analyzer-mcp/wiki)
- [Changelog](CHANGELOG.md)