# Octocode Research Skill - Research Document
**Generated:** 2026-01-18
**Version:** 2.0.0
---
## Table of Contents
- [Overview](#overview)
- [Architecture](#architecture)
- [Core Components](#core-components)
- [API Reference](#api-reference)
- [Tool Registry](#tool-registry)
- [Resilience Patterns](#resilience-patterns)
- [Request Flow](#request-flow)
- [Response Formats](#response-formats)
- [File Structure](#file-structure)
---
## Overview
The **octocode-research** skill is an Express.js HTTP server that wraps `octocode-mcp` tools as REST endpoints for code research. It runs on port **1987** and provides:
| Feature | Description |
|---------|-------------|
| **Unified Tool API** | All tools via `POST /tools/call/:toolName` |
| **MCP-compatible** | Prompts, tools, and system context following MCP patterns |
| **Resilience** | Circuit breaker + retry with exponential backoff |
| **Throttling** | Gradual slowdown for high-frequency requests |
| **Session Telemetry** | Tool call tracking for analytics |
### Key Design Principle
> **All tools are executed via a unified `POST /tools/call/:toolName` endpoint, NOT individual GET routes.**
---
## Architecture
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ OCTOCODE RESEARCH SKILL │
├─────────────────────────────────────────────────────────────────────────────┤
│ HTTP Clients (curl/fetch) ←──────→ Express Server (port 1987) │
│ │ │
│ ┌──────────────────────┼───────────────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ Local Tools LSP Tools GitHub Tools │
│ (ripgrep, fs) (semantic analysis) (API) │
│ │
│ localSearchCode lspGotoDefinition githubSearchCode │
│ localGetFileContent lspFindReferences githubGetFile... │
│ localFindFiles lspCallHierarchy githubViewRepo... │
│ localViewStructure githubSearchPR... │
│ packageSearch │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ EXTERNAL SYSTEMS │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Filesystem │ │ GitHub API │ │ NPM/PyPI │ │ LSP Server │ │
│ │ (ripgrep) │ │ (Octokit) │ │ APIs │ │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Core Components
### 1. Server (`src/server.ts`)
The Express HTTP server that:
- Initializes MCP content cache at startup via `initializeMcpContent()`
- Initializes providers for GitHub token resolution via `initializeProviders()`
- Mounts `/tools` and `/prompts` route handlers
- Exposes `/health` endpoint for monitoring with circuit breaker states
- Handles graceful shutdown with `SIGTERM`/`SIGINT` handlers
**Key functions:**
- `createServer()` - Creates and configures the Express app
- `startServer()` - Starts the HTTP server on port 1987
- `gracefulShutdown()` - Cleans up contexts and circuits on shutdown
### 2. MCP Cache (`src/mcpCache.ts`)
Singleton cache that loads tool metadata **ONCE** at startup:
```typescript
export async function initializeMcpContent(): Promise<CompleteMetadata> {
if (mcpContent) return mcpContent;
initPromise = (async () => {
await initialize();
const content = await loadToolContent();
mcpContent = content;
return content;
})();
return initPromise;
}
export function getMcpContent(): CompleteMetadata {
if (!mcpContent) {
throw new Error('mcpContent not initialized');
}
return mcpContent;
}
```
### 3. Index (`src/index.ts`)
Re-exports layer that maps `octocode-mcp` functions to skill-friendly names:
| Export | Source |
|--------|--------|
| `githubSearchCode` | `searchMultipleGitHubCode` |
| `githubGetFileContent` | `fetchMultipleGitHubFileContents` |
| `localSearchCode` | `executeRipgrepSearch` |
| `localGetFileContent` | `executeFetchContent` |
| `lspGotoDefinition` | `executeGotoDefinition` |
| `lspFindReferences` | `executeFindReferences` |
| `lspCallHierarchy` | `executeCallHierarchy` |
| `packageSearch` | `searchPackages` |
### 4. Routes (`src/routes/`)
| File | Mounted | Purpose |
|------|---------|---------|
| `tools.ts` | ✅ `/tools/*` | **Main API** - Tool discovery & execution |
| `prompts.ts` | ✅ `/prompts/*` | Prompt discovery |
| `local.ts` | ❌ | Handler logic (used in tests) |
| `lsp.ts` | ❌ | Handler logic (used in tests) |
| `github.ts` | ❌ | Handler logic (used in tests) |
| `package.ts` | ❌ | Handler logic (used in tests) |
---
## API Reference
### Discovery Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Server health, memory usage, circuit breaker states |
| `/tools/list` | GET | List all tools (concise - name + description) |
| `/tools/info/:toolName` | GET | Get tool JSON schema + hints |
| `/tools/info` | GET | List all tools with details |
| `/tools/system` | GET | Get system prompt (load FIRST) |
| `/tools/metadata` | GET | Get raw metadata (advanced) |
| `/prompts/list` | GET | List all prompts (MCP-compatible format) |
| `/prompts/info/:promptName` | GET | Get prompt content and arguments |
### Execution Endpoint
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/tools/call/:toolName` | **POST** | **Execute any tool** |
**Request format:**
```json
{
"queries": [{
"mainResearchGoal": "Overall research objective",
"researchGoal": "This specific query's goal",
"reasoning": "Why this approach helps",
// ... tool-specific parameters
}]
}
```
**Constraints:**
- `queries` must be an array with 1-3 items
- Each query must include research context parameters
---
## Tool Registry
The `TOOL_REGISTRY` in `src/routes/tools.ts` maps tool names to their functions and resilience wrappers:
```typescript
const TOOL_REGISTRY: Record<string, ToolEntry> = {
// GitHub tools
githubSearchCode: { fn: githubSearchCode, resilience: withGitHubResilience, category: 'github' },
githubGetFileContent: { fn: githubGetFileContent, resilience: withGitHubResilience, category: 'github' },
githubViewRepoStructure: { fn: githubViewRepoStructure, resilience: withGitHubResilience, category: 'github' },
githubSearchRepositories: { fn: githubSearchRepositories, resilience: withGitHubResilience, category: 'github' },
githubSearchPullRequests: { fn: githubSearchPullRequests, resilience: withGitHubResilience, category: 'github' },
// Local tools
localSearchCode: { fn: localSearchCode, resilience: withLocalResilience, category: 'local' },
localGetFileContent: { fn: localGetFileContent, resilience: withLocalResilience, category: 'local' },
localFindFiles: { fn: localFindFiles, resilience: withLocalResilience, category: 'local' },
localViewStructure: { fn: localViewStructure, resilience: withLocalResilience, category: 'local' },
// LSP tools
lspGotoDefinition: { fn: lspGotoDefinition, resilience: withLspResilience, category: 'lsp' },
lspFindReferences: { fn: lspFindReferences, resilience: withLspResilience, category: 'lsp' },
lspCallHierarchy: { fn: lspCallHierarchy, resilience: withLspResilience, category: 'lsp' },
// Package tools
packageSearch: { fn: packageSearch, resilience: withPackageResilience, category: 'package' },
};
```
### Available Tools (13 total)
| Tool | Category | Description |
|------|----------|-------------|
| `localSearchCode` | Local | Search code with ripgrep |
| `localGetFileContent` | Local | Read local file content |
| `localFindFiles` | Local | Find files by pattern/metadata |
| `localViewStructure` | Local | View directory tree |
| `lspGotoDefinition` | LSP | Go to symbol definition |
| `lspFindReferences` | LSP | Find all symbol references |
| `lspCallHierarchy` | LSP | Get call hierarchy (incoming/outgoing) |
| `githubSearchCode` | GitHub | Search code in repos |
| `githubGetFileContent` | GitHub | Read file from repo |
| `githubViewRepoStructure` | GitHub | View repo tree |
| `githubSearchRepositories` | GitHub | Search repositories |
| `githubSearchPullRequests` | GitHub | Search pull requests |
| `packageSearch` | Package | Search npm/PyPI |
---
## Resilience Patterns
### 1. Circuit Breaker (`src/utils/circuitBreaker.ts`)
Prevents cascading failures with three states:
```
CLOSED (normal) ──[failures >= threshold]──► OPEN (reject all)
▲ │
│ [timeout]
│ │
└──[successes >= threshold]─── HALF-OPEN ◄─┘
(probe)
```
**Default configuration:**
```typescript
const DEFAULT_CONFIG: CircuitBreakerConfig = {
failureThreshold: 3, // Failures before opening
successThreshold: 2, // Successes to close from half-open
resetTimeoutMs: 30000, // Time before half-open probe
};
```
**Per-service overrides:**
| Service | Failure Threshold | Success Threshold | Timeout |
|---------|-------------------|-------------------|---------|
| LSP | 3 | 1 | 10s |
| GitHub | 2 | 1 | 60s |
**Key functions:**
- `withCircuitBreaker(name, operation, fallback?)` - Execute with protection
- `getCircuitState(name)` - Monitor circuit health
- `configureCircuit(name, config)` - Customize thresholds
- `getAllCircuitStates()` - Health dashboard (used in `/health`)
### 2. Retry with Exponential Backoff (`src/utils/retry.ts`)
```typescript
const RETRY_CONFIGS = {
lsp: {
maxAttempts: 3,
initialDelayMs: 500,
maxDelayMs: 5000,
backoffMultiplier: 2,
},
github: {
maxAttempts: 3,
initialDelayMs: 1000,
maxDelayMs: 30000,
backoffMultiplier: 3,
},
package: {
maxAttempts: 3,
initialDelayMs: 1000,
maxDelayMs: 15000,
backoffMultiplier: 2,
},
local: {
maxAttempts: 2,
initialDelayMs: 100,
maxDelayMs: 1000,
backoffMultiplier: 2,
},
};
```
### 3. Combined Resilience (`src/utils/resilience.ts`)
Four pre-configured wrappers that combine circuit breaker + retry:
```typescript
export async function withGitHubResilience<T>(operation: () => Promise<T>, toolName: string): Promise<T>
export async function withLspResilience<T>(operation: () => Promise<T>, toolName: string): Promise<T>
export async function withLocalResilience<T>(operation: () => Promise<T>, toolName: string): Promise<T>
export async function withPackageResilience<T>(operation: () => Promise<T>, toolName: string): Promise<T>
```
### 4. Request Throttling (`src/middleware/throttle.ts`)
Uses `express-slow-down` for gradual slowdown:
| Config | Value | Description |
|--------|-------|-------------|
| `windowMs` | 60000 | 1 minute window |
| `delayAfter` | 50 | Requests before slowdown |
| `delayMs` | 500 | Initial delay increment |
| `maxDelayMs` | 10000 | Maximum delay (10s) |
**Backoff formula:** `delay = 500ms × 1.5^(excess-1)` capped at 10s
---
## Request Flow
### Tool Execution Flow
```
POST /tools/call/localSearchCode
Body: { "queries": [{ "pattern": "auth", "path": "src", ... }] }
│
▼
┌─────────────────────────────────────┐
│ requestLogger Middleware │
│ → Log: tool, params, timestamp │
│ → Write to ~/.octocode/logs/ │
└────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ Route Handler (routes/tools.ts) │
│ POST /tools/call/:toolName │
│ │
│ 1. Lookup tool in TOOL_REGISTRY │
│ 2. Validate queries array (1-3) │
│ 3. Get resilience wrapper │
└────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ Resilience Wrapper │
│ │
│ toolEntry.resilience(fn, toolName) │
│ ├─ Circuit Breaker Check │
│ │ └─ OPEN? → Fail fast │
│ │ └─ CLOSED/HALF-OPEN? → Proceed│
│ ├─ Retry with Backoff │
│ └─ Execute Tool Function │
└────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ octocode-mcp Tool Function │
│ │
│ localSearchCode({ queries: [...] })│
│ ├─ Execute ripgrep search │
│ ├─ Parse results │
│ ├─ Apply hints │
│ └─ Return CallToolResult │
└────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ Response Transformation │
│ │
│ parseToolResponse(rawResult) │
│ ├─ Extract data, hints, research │
│ └─ Detect errors │
└────────┬────────────────────────────┘
│
▼
HTTP Response (JSON)
```
---
## Response Formats
### Single Query Response
```json
{
"tool": "localSearchCode",
"success": true,
"data": {
"files": [...],
"totalMatches": 10,
"pagination": { "page": 1, "hasMore": false }
},
"hints": [
"Use lineHint for LSP tools",
"Consider narrowing search with path filter"
],
"research": {
"mainResearchGoal": "Find authentication handlers",
"researchGoal": "Locate auth middleware",
"reasoning": "Understanding auth flow"
}
}
```
### Bulk Query Response (2-3 queries)
```json
{
"tool": "localSearchCode",
"bulk": true,
"success": true,
"instructions": "Review results by status...",
"results": [
{ "id": 1, "status": "hasResults", "data": {...}, "research": {...} },
{ "id": 2, "status": "empty", "data": {}, "research": {...} }
],
"hints": {
"hasResults": ["Follow up with LSP for semantic analysis"],
"empty": ["Try broader search pattern"],
"error": []
},
"counts": {
"total": 2,
"hasResults": 1,
"empty": 1,
"error": 0
}
}
```
### Error Response
```json
{
"tool": "localSearchCode",
"success": false,
"data": {},
"hints": [
"Circuit breaker is open - retry in 30s",
"Consider using fallback tool"
],
"research": {
"mainResearchGoal": "...",
"researchGoal": "...",
"reasoning": "..."
}
}
```
---
## File Structure
```
octocode-research/
├── src/
│ ├── server.ts # Express server entry point
│ ├── index.ts # Re-exports from octocode-mcp
│ ├── mcpCache.ts # MCP client caching
│ ├── routes/
│ │ ├── tools.ts # /tools/* - MAIN tool API
│ │ ├── prompts.ts # /prompts/* - prompt discovery
│ │ ├── local.ts # Handler logic (tests)
│ │ ├── lsp.ts # Handler logic (tests)
│ │ ├── github.ts # Handler logic (tests)
│ │ └── package.ts # Handler logic (tests)
│ ├── middleware/
│ │ ├── contextPropagation.ts # Shutdown cleanup
│ │ ├── errorHandler.ts # Error response formatting
│ │ ├── logger.ts # Request/response logging
│ │ ├── queryParser.ts # Zod validation
│ │ └── throttle.ts # Rate limiting
│ ├── validation/
│ │ ├── index.ts # Schema exports
│ │ ├── schemas.ts # HTTP schemas (from octocode-mcp)
│ │ └── httpPreprocess.ts # Query string conversion
│ ├── utils/
│ │ ├── circuitBreaker.ts # Circuit breaker pattern
│ │ ├── colors.ts # Console color functions
│ │ ├── logger.ts # File-based logging
│ │ ├── resilience.ts # Combined resilience wrappers
│ │ ├── responseBuilder.ts # Research response formatting
│ │ ├── responseFactory.ts # Safe data extraction
│ │ ├── responseParser.ts # MCP response parsing
│ │ ├── retry.ts # Retry with backoff
│ │ └── routeFactory.ts # Route handler factory
│ ├── types/
│ │ ├── express.d.ts # Express type extensions
│ │ ├── guards.ts # Type guard functions
│ │ ├── mcp.ts # MCP protocol types
│ │ ├── responses.ts # Response type definitions
│ │ └── toolTypes.ts # Tool parameter types
│ └── __tests__/
│ ├── integration/ # Integration tests
│ └── unit/ # Unit tests
├── docs/
│ ├── API_REFERENCE.md # HTTP API reference
│ ├── ARCHITECTURE.md # Architecture documentation
│ ├── FLOWS.md # Flow diagrams
│ └── SKILL_RESEARCH.md # This document
├── references/
│ ├── GUARDRAILS.md # Safety guardrails
│ └── QUICK_DECISION_GUIDE.md # Tool selection guide
├── scripts/
│ ├── server.js # Bundled server (tsdown)
│ └── server.d.ts # Type declarations
├── SKILL.md # Skill definition for agents
├── AGENTS.md # Development guide
├── README.md # Quick start
├── package.json
├── tsconfig.json
├── tsdown.config.ts
└── vitest.config.ts
```
---
## Quick Start
```bash
# Build
npm run build
# Start server
npm start
# Development with hot reload
npm run dev
# Health check
curl http://localhost:1987/health
# List tools
curl http://localhost:1987/tools/list
# Get tool schema
curl http://localhost:1987/tools/info/localSearchCode
# Execute tool
curl -X POST http://localhost:1987/tools/call/localSearchCode \
-H "Content-Type: application/json" \
-d '{"queries": [{
"mainResearchGoal": "Find auth",
"researchGoal": "Search handlers",
"reasoning": "Testing",
"pattern": "authenticate",
"path": "/src"
}]}'
```
---
## Related Documentation
| Document | Purpose |
|----------|---------|
| [SKILL.md](../SKILL.md) | Agent workflow guide |
| [AGENTS.md](../AGENTS.md) | Development guidelines |
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Full architecture details |
| [FLOWS.md](./FLOWS.md) | Request flow diagrams |
| [API_REFERENCE.md](./API_REFERENCE.md) | HTTP API reference |
| [QUICK_DECISION_GUIDE.md](../references/QUICK_DECISION_GUIDE.md) | Tool selection |
| [GUARDRAILS.md](../references/GUARDRAILS.md) | Safety rules |
---
*Generated by Octocode Research Analysis*
