# Octocode Research - Main Flows & Architecture
> Understanding how the octocode-research skill works from server startup to tool execution.
> **v2.1.0** - Process managed by PM2
## Table of Contents
- [Overview](#overview)
- [Architecture Diagram](#architecture-diagram)
- [Main Components](#main-components)
- [Core Flows](#core-flows)
- [Server Startup Flow](#1-server-startup-flow)
- [Tool Execution Flow](#2-tool-execution-flow)
- [Discovery Flow](#3-discovery-flow)
- [PM2 Process Management](#pm2-process-management)
- [Component Connections](#component-connections)
- [Quick Reference](#quick-reference)
---
## Overview
The **octocode-research** skill is a lightweight HTTP API server that wraps `octocode-mcp` tools for code research. It provides:
- **HTTP Interface**: REST API on `localhost:1987`
- **HTTP Clients**: curl, fetch, or any HTTP client
- **Unified Tool API**: All tools via `POST /tools/call/:toolName`
- **Resilience**: Circuit breaker + retry patterns for reliability
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ OCTOCODE RESEARCH SKILL │
├─────────────────────────────────────────────────────────────────────────────┤
│ HTTP Clients (curl/fetch) ←──────→ HTTP Server (port 1987) ←→ octocode-mcp │
│ │ │
│ ├─→ Local Tools (ripgrep, fs) │
│ ├─→ LSP Tools (semantic analysis)│
│ ├─→ GitHub Tools (API) │
│ └─→ Package Tools (npm/PyPI) │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Architecture Diagram
```
┌───────────────────────────┐
│ AI Agent / User │
└─────────────┬─────────────┘
│
┌─────────────────┼─────────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ curl │ │ fetch │ │ HTTP │
│ │ │ │ │ Client │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────────┼─────────────────┘
│ HTTP Request
▼
┌──────────────────────────────────────────────────────────────────────────────┐
│ EXPRESS SERVER (port 1987) │
├──────────────────────────────────────────────────────────────────────────────┤
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │ MIDDLEWARE LAYER │ │
│ │ ┌─────────────┐ ┌───────────────┐ ┌─────────────┐ ┌───────────────┐ │ │
│ │ │requestLogger│→│ express.json │→│contextPropag│→│ errorHandler │ │ │
│ │ └─────────────┘ └───────────────┘ └─────────────┘ └───────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │ ROUTE HANDLERS │ │
│ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │
│ │ │ /tools/* │ │ │
│ │ │ • GET /tools/list - List all tools │ │ │
│ │ │ • GET /tools/info/:name - Get tool schema │ │ │
│ │ │ • GET /tools/system - Get system prompt │ │ │
│ │ │ • POST /tools/call/:name - Execute any tool │ │ │
│ │ └─────────────────────────────────────────────────────────────────────┘ │ │
│ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │
│ │ │ /prompts/* │ │ │
│ │ │ • GET /prompts/list - List all prompts │ │ │
│ │ │ • GET /prompts/info/:name - Get prompt details │ │ │
│ │ └─────────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │ │
│ │ ┌─────┴─────────────────────────────────────────────────────┐ │ │
│ │ │ RESILIENCE LAYER │ │ │
│ │ │ (Circuit Breaker + Retry + Rate Limit Handling) │ │ │
│ │ └───────────────────────────┬────────────────────────────────┘ │ │
│ └──────────────────────────────┼────────────────────────────────────────────┘ │
└────────────────────────────────┼────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────────┐
│ OCTOCODE-MCP PACKAGE │
├──────────────────────────────────────────────────────────────────────────────┤
│ src/index.ts (re-exports) → Tool Execution Functions │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ localSearchCode, localGetFileContent, localFindFiles, localViewStructure│ │
│ │ lspGotoDefinition, lspFindReferences, lspCallHierarchy │ │
│ │ githubSearchCode, githubGetFileContent, githubViewRepoStructure, etc. │ │
│ │ packageSearch │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────────┐
│ EXTERNAL SYSTEMS │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Filesystem │ │ GitHub API │ │ NPM/PyPI │ │ LSP Server │ │
│ │ (ripgrep) │ │ (Octokit) │ │ APIs │ │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
```
---
## Main Components
### 1. **Server (\`src/server.ts\`)**
The Express HTTP server that:
- Initializes MCP content cache at startup
- Mounts \`/tools\` and \`/prompts\` route handlers
- Handles graceful shutdown
- Exposes \`/health\` endpoint for monitoring
### 2. **MCP Cache (\`src/mcpCache.ts\`)**
Singleton cache that:
- Loads tool metadata ONCE at startup
- Provides fast access to tool schemas
- Avoids repeated initialization costs
### 3. **Index (\`src/index.ts\`)**
Re-exports layer that:
- Maps \`octocode-mcp\` functions to skill-friendly names
- Provides type exports for TypeScript consumers
- Centralizes all tool imports
### 4. **Routes (\`src/routes/\`)**
| File | Endpoints | Purpose |
|------|-----------|---------|
| \`tools.ts\` | \`/tools/list\`, \`/tools/info/:name\`, \`/tools/call/:name\`, \`/tools/system\` | **Main API** - Tool discovery & execution |
| \`prompts.ts\` | \`/prompts/list\`, \`/prompts/info/:name\` | Prompt discovery |
| \`local.ts\` | *(Not mounted - used in tests only)* | Filesystem operations handlers |
| \`lsp.ts\` | *(Not mounted - used in tests only)* | Semantic analysis handlers |
| \`github.ts\` | *(Not mounted - used in tests only)* | GitHub API handlers |
| \`package.ts\` | *(Not mounted - used in tests only)* | npm/PyPI search handlers |
> **Note**: Only \`/tools/*\` and \`/prompts/*\` are mounted in production. The individual route files contain handler logic used by the unified \`/tools/call/:toolName\` endpoint.
### 5. **Build Output (\`scripts/\`)**
The bundled server output from tsdown:
| File | Purpose |
|------|---------|
| \`server.js\` | Bundled server (standalone, all deps included) |
| \`server.d.ts\` | TypeScript type declarations |
---
## Core Flows
### 1. Server Startup Flow
```
┌────────────────────────────────────────────────────────────────────────────┐
│ SERVER STARTUP FLOW (PM2 Managed) │
└────────────────────────────────────────────────────────────────────────────┘
npm run pm2:start (or pm2 start ecosystem.config.cjs)
│
▼
┌──────────────────────────────────────┐
│ PM2 Process Manager │
│ - Reads ecosystem.config.cjs │
│ - Spawns node scripts/server.js │
│ - Sets wait_ready: true │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ src/server.ts │
│ startServer() │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ 1. createServer() │
│ ├─ initializeLogger() │
│ ├─ initializeSession() │
│ ├─ express() │
│ ├─ use(requestLogger) │
│ ├─ mount(/health) │
│ ├─ mount(/tools, toolsRoutes) │
│ ├─ mount(/prompts, promptsRoutes) │
│ └─ use(errorHandler) │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ 2. app.listen(1987) │
│ └─ HTTP server listening │
│ └─ Status: "initializing" │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ 3. Background initialization (async) │
│ ├─ initializeMcpContent() │
│ │ ├─ initialize() (octocode-mcp) │
│ │ └─ loadToolContent() → cache │
│ ├─ initializeProviders() │
│ │ └─ Resolve GitHub token │
│ └─ process.send('ready') ◄─────────│── PM2 ready signal
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ 4. PM2 marks process "online" │
│ └─ Status: "ok" │
│ └─ Server Ready! │
└──────────────────────────────────────┘
```
### 2. Tool Execution Flow
```
┌────────────────────────────────────────────────────────────────────────────┐
│ TOOL EXECUTION FLOW │
└────────────────────────────────────────────────────────────────────────────┘
POST /tools/call/localSearchCode
Body: { "queries": [{ "pattern": "auth", "path": "src", ... }] }
│
▼
┌──────────────────────────────────────┐
│ requestLogger Middleware │
│ → Log: tool, params, timestamp │
│ → Write to ~/.octocode/logs/ │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Route Handler (routes/tools.ts) │
│ POST /tools/call/:toolName │
│ │
│ 1. Lookup tool in TOOL_REGISTRY │
│ 2. Validate queries array │
│ 3. Get resilience wrapper │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Body Validation │
│ │
│ Input: { queries: [...] } │
│ Checks: │
│ - queries is array │
│ - 1-3 queries per request │
│ - Tool exists in registry │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Resilience Wrapper │
│ │
│ toolEntry.resilience(fn, toolName) │
│ ├─ Circuit Breaker Check │
│ │ └─ OPEN? → Fail fast │
│ │ └─ CLOSED/HALF-OPEN? → Proceed │
│ ├─ Retry with Backoff │
│ │ └─ Config per tool category │
│ └─ Execute Tool Function │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ octocode-mcp Tool Function │
│ │
│ localSearchCode({ queries: [...] }) │
│ ├─ Execute ripgrep search │
│ ├─ Parse results │
│ ├─ Apply hints │
│ └─ Return CallToolResult │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Response Transformation │
│ │
│ parseToolResponse(rawResult) │
│ ├─ Extract data, hints, research │
│ └─ Detect errors │
└────────┬─────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ HTTP Response │
│ │
│ { │
│ tool: "localSearchCode", │
│ success: true, │
│ data: { ... }, │
│ hints: [...], │
│ research: { ... } │
│ } │
└──────────────────────────────────────┘
```
### 3. Discovery Flow
```
┌────────────────────────────────────────────────────────────────────────────┐
│ DISCOVERY FLOW │
└────────────────────────────────────────────────────────────────────────────┘
GET /tools/list GET /tools/info/localSearchCode
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ Static tool list │ │ getMcpContent() │
│ (hardcoded in │ │ └─ Return cached │
│ tools.ts) │ │ metadata │
└────────┬─────────────┘ └────────┬─────────────┘
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ Return tool list: │ │ Find tool by name: │
│ - name │ │ - name │
│ - description │ │ - description │
│ - _hint: use │ │ - inputSchema (Zod)│
│ /tools/info │ │ - hints │
│ │ │ │
│ (concise discovery) │ │ (full schema) │
└──────────────────────┘ └──────────────────────┘
```
---
## Component Connections
```
┌────────────────────────────────────────────────────────────────────────────┐
│ COMPONENT DEPENDENCY GRAPH │
└────────────────────────────────────────────────────────────────────────────┘
┌─────────────────┐
│ package.json │
│ (scripts) │
└────────┬────────┘
│
┌───────────────────┼───────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ scripts/ │ │ src/server.ts │ │ src/cli.ts │
│ server.ts │──│ (entry) │ │ (entry) │
└───────┬───────┘ └───────┬───────┘ └───────────────┘
│ │
│ ├─────────────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ octocode- │ │ src/mcpCache │ │ src/routes/ │
│ shared │ │ .ts │◄───────────│ tools.ts │
│ (session) │ └───────┬───────┘ │ prompts.ts │
└───────────────┘ │ └───────┬───────┘
│ │
▼ ▼
┌───────────────┐ ┌───────────────┐
│ src/index.ts │◄───────────│ src/utils/ │
│ (re-exports) │ │ resilience │
└───────┬───────┘ └───────┬───────┘
│ │
│ │
▼ ▼
┌───────────────┐ ┌───────────────┐
│ octocode-mcp │ │ src/utils/ │
│ (tools) │ │ circuitBreaker│
└───────────────┘ │ retry.ts │
└───────────────┘
```
---
## PM2 Process Management
The server is managed by PM2 for automatic restarts, monitoring, and log management.
### PM2 Features
| Feature | Configuration | Description |
|---------|---------------|-------------|
| **Cron Restart** | `0 * * * *` | Restarts every hour for memory hygiene |
| **Memory Guard** | `500M` | Auto-restart if memory exceeds threshold |
| **Ready Signal** | `wait_ready: true` | PM2 waits for `process.send('ready')` |
| **Kill Timeout** | `10000ms` | Graceful shutdown timeout |
| **Auto Restart** | `max_restarts: 10` | Restart on crash with backoff |
### PM2 Commands
```bash
# NPM Scripts
npm run pm2:start # Start with PM2
npm run pm2:stop # Stop gracefully
npm run pm2:restart # Restart
npm run pm2:reload # Zero-downtime reload
npm run pm2:delete # Remove from PM2
npm run pm2:logs # View logs
npm run pm2:monit # Dashboard
# Direct PM2 Commands
pm2 status # List processes
pm2 logs octocode-research # Tail logs
pm2 describe octocode-research # Process details
```
### Lifecycle Diagram
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ PM2 LIFECYCLE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ pm2 start ──▶ [STARTING] ──▶ [WAITING READY] ──▶ [ONLINE] │
│ │ │ │
│ process.send('ready') │ │
│ │ │
│ ┌──────────────────────────────────────────────────┴──────────────────┐ │
│ │ RUNNING │ │
│ │ - Cron restart every hour │ │
│ │ - Memory check (restart if > 500MB) │ │
│ │ - Auto-restart on crash (with backoff) │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ SIGINT/SIGTERM ──────────────────▶│ │
│ │ │
│ ┌───────────────┴───────────────┐ │
│ │ gracefulShutdown() │ │
│ │ 1. stopCircuitCleanup() │ │
│ │ 2. clearAllCircuits() │ │
│ │ 3. server.close() │ │
│ │ 4. process.exit(0) │ │
│ └───────────────────────────────┘ │
│ │ │
│ ▼ │
│ PM2 Auto-Restart │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Quick Reference
### Start Server
```bash
# Development (with hot reload)
npm run dev
# Production (PM2 managed - RECOMMENDED)
npm run pm2:start
npm run pm2:logs # View logs
# Direct (no PM2)
npm start
# Or: node scripts/server.js
```
### HTTP Examples
```bash
# Discovery
curl http://localhost:1987/health # Health check
curl http://localhost:1987/tools/list # List all tools
curl http://localhost:1987/tools/info/localSearchCode # Get tool schema
curl http://localhost:1987/tools/system # Load system prompt
curl http://localhost:1987/prompts/list # List all prompts
# Tool Execution (ALL tools via POST /tools/call/:toolName)
curl -X POST http://localhost:1987/tools/call/localSearchCode \
-H "Content-Type: application/json" \
-d '{"queries": [{"mainResearchGoal": "Find auth", "researchGoal": "Search", "reasoning": "Test", "pattern": "auth", "path": "src"}]}'
curl -X POST http://localhost:1987/tools/call/lspGotoDefinition \
-H "Content-Type: application/json" \
-d '{"queries": [{"mainResearchGoal": "Find def", "researchGoal": "Locate", "reasoning": "Test", "uri": "file:///path/file.ts", "symbolName": "createServer", "lineHint": 20}]}'
```
### HTTP Endpoints
| Category | Endpoint | Method | Description |
|----------|----------|--------|-------------|
| **Health** | \`/health\` | GET | Server health + circuit states |
| **Discovery** | \`/tools/list\` | GET | List all tools (concise) |
| | \`/tools/info/:name\` | GET | Get tool schema + hints |
| | \`/tools/system\` | GET | Get system prompt |
| | \`/prompts/list\` | GET | List all prompts |
| | \`/prompts/info/:name\` | GET | Get prompt details |
| **Execution** | \`/tools/call/:toolName\` | POST | **Execute any tool** |
### Available Tools (via \`/tools/call/:toolName\`)
| Tool Name | Category | Description |
|-----------|----------|-------------|
| \`localSearchCode\` | Local | Search code with ripgrep |
| \`localGetFileContent\` | Local | Read local file content |
| \`localFindFiles\` | Local | Find files by pattern/metadata |
| \`localViewStructure\` | Local | View local directory tree |
| \`lspGotoDefinition\` | LSP | Go to symbol definition |
| \`lspFindReferences\` | LSP | Find all symbol references |
| \`lspCallHierarchy\` | LSP | Get call hierarchy |
| \`githubSearchCode\` | GitHub | Search code in GitHub repos |
| \`githubGetFileContent\` | GitHub | Read file from GitHub repo |
| \`githubViewRepoStructure\` | GitHub | View GitHub repo tree |
| \`githubSearchRepositories\` | GitHub | Search GitHub repositories |
| \`githubSearchPullRequests\` | GitHub | Search pull requests |
| \`packageSearch\` | Package | Search npm/PyPI packages |
### Resilience Configuration
| Service | Max Attempts | Initial Delay | Max Delay | Backoff |
|---------|--------------|---------------|-----------|---------|
| GitHub | 3 | 1000ms | 30000ms | 3x |
| LSP | 3 | 500ms | 5000ms | 2x |
| Local | 2 | 100ms | 1000ms | 2x |
| Package | 3 | 1000ms | 15000ms | 2x |
### Circuit Breaker States
```
CLOSED (normal) ──[3 failures]──► OPEN (reject all)
▲ │
│ [30s timeout]
│ │
└──[1 success]─── HALF-OPEN ◄──┘
(probe)
```
**Per-service configuration:**
- **LSP**: 3 failures, 1 success, 10s timeout
- **GitHub**: 2 failures, 1 success, 60s timeout
---
## Files Summary
| File | Purpose |
|------|---------|
| `ecosystem.config.cjs` | **PM2 configuration** - restart strategies, memory limits, ready signal |
| `src/server.ts` | Express server, route mounting (`/tools`, `/prompts`), graceful shutdown |
| `src/index.ts` | Re-exports from octocode-mcp |
| `src/mcpCache.ts` | Tool metadata caching |
| `src/routes/tools.ts` | **Main API** - `/tools/call/:toolName` and discovery |
| `src/routes/prompts.ts` | Prompt discovery |
| `src/routes/local.ts` | Handler logic (used by tools.ts registry) |
| `src/routes/lsp.ts` | Handler logic (used by tools.ts registry) |
| `src/routes/github.ts` | Handler logic (used by tools.ts registry) |
| `src/routes/package.ts` | Handler logic (used by tools.ts registry) |
| `src/middleware/*.ts` | Logging, validation, error handling |
| `src/utils/*.ts` | Resilience, formatting, parsing |
| `src/validation/schemas.ts` | Zod validation schemas |
| `scripts/server.js` | Bundled server (tsdown output) |
---
*Created by Octocode Research Skill v2.1.0*
*Last validated: 2026-01-20*
