================================================================================
MCP vs Native Tool Usage Analysis
Comparing Claude Code workflows with and without MCP
================================================================================
1. Scenario: Find the BM25Indexer class definition
----------------------------------------------------------------------
MCP Approach:
Steps: 1
Tokens: 100
Time: 0.1s
Workflow: symbol_lookup(symbol)
Native Approach:
Steps: 2
Tokens: 5,500
Time: 0.7s
Workflow: grep(pattern) -> read_full(file)
Improvements with MCP:
Token reduction: 98.2%
Time reduction: 85.7%
Fewer steps: 50.0%
2. Scenario: Fix a bug in the search method of SQLiteStore
----------------------------------------------------------------------
MCP Approach:
Steps: 4
Tokens: 750
Time: 0.5s
Workflow: symbol_lookup(symbol) -> search_code(query) -> read_partial(file) -> edit(old)
Native Approach:
Steps: 4
Tokens: 9,150
Time: 2.2s
Workflow: find(name) -> grep(pattern) -> read_full(file) -> edit(old)
Improvements with MCP:
Token reduction: 91.8%
Time reduction: 77.3%
Fewer steps: 0.0%
3. Scenario: Understand how reranking works in the system
----------------------------------------------------------------------
MCP Approach:
Steps: 3
Tokens: 1,000
Time: 0.5s
Workflow: search_code(query) -> search_code(query) -> read_partial(file)
Native Approach:
Steps: 4
Tokens: 11,800
Time: 1.9s
Workflow: grep(pattern) -> read_full(file) -> read_full(file) -> grep(pattern)
Improvements with MCP:
Token reduction: 91.5%
Time reduction: 73.7%
Fewer steps: 25.0%
4. Scenario: Add caching to the search functionality
----------------------------------------------------------------------
MCP Approach:
Steps: 4
Tokens: 1,220
Time: 0.6s
Workflow: search_code(query) -> symbol_lookup(symbol) -> read_partial(file) -> multi_edit(edits)
Native Approach:
Steps: 4
Tokens: 18,000
Time: 2.8s
Workflow: find(name) -> read_full(file) -> read_full(file) -> write(file)
Improvements with MCP:
Token reduction: 93.2%
Time reduction: 78.6%
Fewer steps: 0.0%
================================================================================
OVERALL SUMMARY
================================================================================
Total Token Usage:
- With MCP: 3,070 tokens
- Without MCP: 44,450 tokens
- Reduction: 93.1%
Total Time:
- With MCP: 1.7s
- Without MCP: 7.6s
- Reduction: 77.6%
Key Benefits of MCP:
1. Targeted retrieval: Get exactly what you need (symbol definitions, specific code)
2. Semantic understanding: Natural language queries that understand intent
3. Reduced context usage: Snippets instead of full files
4. Faster operations: Direct lookups instead of pattern matching
5. Better accuracy: Type-aware symbol resolution
Current Status:
- MCP tools have [MCP-FIRST] instructions
- But Claude Code is not using them (0% adoption)
- Patched dispatcher now returns results via BM25 fallback
- Ready for real-world testing
================================================================================
