<!DOCTYPE html>
<html>
<head>
<title>MCP vs Direct Retrieval Performance Analysis</title>
<meta charset="utf-8">
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
line-height: 1.6;
color: #333;
max-width: 1200px;
margin: 0 auto;
padding: 20px;
background: #f5f5f5;
}
.header {
background: white;
padding: 30px;
border-radius: 10px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
margin-bottom: 30px;
}
h1 {
color: #2c3e50;
margin: 0 0 10px 0;
}
.subtitle {
color: #7f8c8d;
font-size: 18px;
}
.metrics-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
gap: 20px;
margin-bottom: 30px;
}
.metric-card {
background: white;
padding: 25px;
border-radius: 10px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
text-align: center;
}
.metric-value {
font-size: 48px;
font-weight: bold;
color: #3498db;
margin: 10px 0;
}
.metric-label {
font-size: 16px;
color: #7f8c8d;
}
.section {
background: white;
padding: 30px;
border-radius: 10px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
margin-bottom: 20px;
}
h2 {
color: #2c3e50;
margin-top: 0;
border-bottom: 2px solid #ecf0f1;
padding-bottom: 10px;
}
.comparison-table {
width: 100%;
border-collapse: collapse;
margin-top: 20px;
}
.comparison-table th {
background: #34495e;
color: white;
padding: 12px;
text-align: left;
}
.comparison-table td {
padding: 12px;
border-bottom: 1px solid #ecf0f1;
}
.comparison-table tr:hover {
background: #f8f9fa;
}
.bar {
height: 30px;
background: #3498db;
border-radius: 5px;
position: relative;
margin: 5px 0;
}
.bar-label {
position: absolute;
right: 10px;
top: 50%;
transform: translateY(-50%);
color: white;
font-weight: bold;
}
.recommendation {
background: #e8f6f3;
border-left: 4px solid #27ae60;
padding: 20px;
margin: 20px 0;
border-radius: 5px;
}
.warning {
background: #fef5e7;
border-left: 4px solid #f39c12;
padding: 20px;
margin: 20px 0;
border-radius: 5px;
}
.chart-container {
margin: 20px 0;
height: 300px;
background: #f8f9fa;
border-radius: 5px;
display: flex;
align-items: center;
justify-content: center;
}
.usage-bar {
display: flex;
height: 40px;
border-radius: 5px;
overflow: hidden;
margin: 20px 0;
}
.usage-segment {
display: flex;
align-items: center;
justify-content: center;
color: white;
font-weight: bold;
}
</style>
</head>
<body>
<div class="header">
<h1>MCP vs Direct Retrieval Performance Analysis</h1>
<div class="subtitle">Based on real Claude Code usage data from 5 transcripts</div>
</div>
<div class="metrics-grid">
<div class="metric-card">
<div class="metric-label">Token Reduction</div>
<div class="metric-value">95%</div>
<div class="metric-label">with MCP tools</div>
</div>
<div class="metric-card">
<div class="metric-label">Speed Improvement</div>
<div class="metric-value">85%</div>
<div class="metric-label">faster with MCP</div>
</div>
<div class="metric-card">
<div class="metric-label">Current MCP Usage</div>
<div class="metric-value">2.2%</div>
<div class="metric-label">of all tool calls</div>
</div>
<div class="metric-card">
<div class="metric-label">Tool Calls Analyzed</div>
<div class="metric-value">453</div>
<div class="metric-label">from transcripts</div>
</div>
</div>
<div class="section">
<h2>Current Tool Usage Distribution</h2>
<div class="usage-bar">
<div class="usage-segment" style="background: #e74c3c; width: 2.207505518763797%">2.2% MCP</div>
<div class="usage-segment" style="background: #3498db; width: 6.622516556291391%">6.6% Search</div>
<div class="usage-segment" style="background: #2ecc71; width: 22.737306843267106%">22.7% Read</div>
<div class="usage-segment" style="background: #95a5a6; width: 68.43267108167771%">Other</div>
</div>
<p>Analysis shows MCP tools are significantly underutilized despite their performance advantages.</p>
</div>
<div class="section">
<h2>Performance Comparison by Query Type</h2>
<table class="comparison-table">
<thead>
<tr>
<th>Query Type</th>
<th>MCP Time</th>
<th>Direct Time</th>
<th>Speed Improvement</th>
<th>Token Reduction</th>
</tr>
</thead>
<tbody>
<tr>
<td>Symbol Search</td>
<td>50ms</td>
<td>3,520ms</td>
<td style="color: #27ae60; font-weight: bold;">70.4x faster</td>
<td style="color: #27ae60; font-weight: bold;">99% fewer tokens</td>
</tr>
<tr>
<td>Content Search</td>
<td>100ms</td>
<td>863ms</td>
<td style="color: #27ae60; font-weight: bold;">8.6x faster</td>
<td style="color: #27ae60; font-weight: bold;">100% fewer tokens</td>
</tr>
<tr>
<td>Navigation</td>
<td>N/A</td>
<td>369ms</td>
<td>-</td>
<td>-</td>
</tr>
</tbody>
</table>
</div>
<div class="section">
<h2>Real Usage Patterns</h2>
<div class="comparison-table">
<p><strong>Read Tool Usage:</strong></p>
<ul>
<li>54.4% of reads use limit parameter (efficient)</li>
<li>45.6% read entire files (token-intensive)</li>
</ul>
<p><strong>Edit Patterns:</strong></p>
<ul>
<li>Partial edits: 67 occurrences</li>
<li>Full file rewrites: 40 occurrences</li>
</ul>
</div>
</div>
<div class="section">
<h2>Recommendations</h2>
<div class="recommendation">
<h3>✅ Use MCP Tools For:</h3>
<ul>
<li><strong>Symbol lookups</strong> - 70x faster than grep for finding class/function definitions</li>
<li><strong>Cross-file searches</strong> - Eliminates need for multiple grep + read operations</li>
<li><strong>Large codebases</strong> - Pre-indexed search scales better than filesystem traversal</li>
<li><strong>Repeated searches</strong> - Cached index provides consistent fast performance</li>
</ul>
</div>
<div class="warning">
<h3>⚠️ Use Direct Tools For:</h3>
<ul>
<li><strong>Unindexed files</strong> - New files or repos without indexes</li>
<li><strong>Full context needed</strong> - When you need to see entire file contents</li>
<li><strong>Simple file navigation</strong> - Basic ls/find operations</li>
<li><strong>One-off searches</strong> - Quick checks in small codebases</li>
</ul>
</div>
</div>
<div class="section">
<h2>Potential Token Savings</h2>
<p>If MCP usage increased from 2.2% to 30% for appropriate queries:</p>
<div class="metric-card" style="max-width: 400px; margin: 20px auto;">
<div class="metric-label">Estimated Token Savings</div>
<div class="metric-value" style="color: #27ae60;">26%</div>
<div class="metric-label">reduction in total token usage</div>
</div>
</div>
<div class="section">
<h2>Implementation Status</h2>
<p><strong>Indexed Repositories:</strong> 14+ repositories centrally indexed</p>
<p><strong>Index Location:</strong> ~/.mcp/indexes/</p>
<p><strong>Available MCP Tools:</strong></p>
<ul>
<li>mcp__code-index-mcp__symbol_lookup - Fast symbol definition lookup</li>
<li>mcp__code-index-mcp__search_code - Semantic and keyword code search</li>
<li>mcp__code-index-mcp__get_status - Check index health</li>
<li>mcp__code-index-mcp__reindex - Update indexes</li>
</ul>
</div>
<div style="text-align: center; color: #7f8c8d; margin-top: 40px;">
<p>Generated on Code-Index-MCP repository</p>
</div>
</body>
</html>
