ISSUE_9_ANALYSIS.md•23.7 kB
# Analysis of GitHub Issue #9: MCP Integration Problems
**Issue:** [Feature]: Github Copilot chat mode or instructions or AGENT.md template
**Reporter:** @gilfrade
**Status:** Reopened
**Date Analyzed:** October 26, 2025
**Versions Affected:** v0.4.6, v0.5.4
## Executive Summary
Users are experiencing multiple critical issues when using In-Memoria with GitHub Copilot, including database initialization failures, path handling problems, and tool schema mismatches. These issues stem from incomplete file system handling, poor path management in MCP server context, and gaps in documentation that lead to incorrect tool usage.
---
## User-Reported Issues
### Issue 1: Database Initialization Failures
**Symptom:**
```
Error: Cannot open database because the directory does not exist
```
**Affected Tools:**
- `learn_codebase_intelligence`
- `get_project_blueprint`
**User Configuration:**
```json
{
"command": "npx",
"args": ["in-memoria", "server", ".mcp/in-memoria"],
"env": {
"IN_MEMORIA_DB_FILENAME": ".mcp/in-memoria/memory.db"
}
}
```
**Root Cause Analysis:**
1. **No Parent Directory Creation**
The `SQLiteDatabase` constructor (line 110-114 in `src/storage/sqlite-db.ts`) directly calls `new Database(dbPath)` without checking if parent directories exist:
```typescript
constructor(dbPath: string = ':memory:') {
this.db = new Database(dbPath); // Fails if parent directory doesn't exist
this.migrator = new DatabaseMigrator(this.db);
this.initializeDatabase();
}
```
2. **Path Construction Logic**
The `config.getDatabasePath()` method (line 48 in `src/config/config.ts`) constructs paths but doesn't validate them:
```typescript
getDatabasePath(projectPath?: string): string {
const basePath = projectPath || process.cwd();
return join(basePath, this.config.database.filename);
}
```
When `IN_MEMORIA_DB_FILENAME=".mcp/in-memoria/memory.db"`, the resulting path includes subdirectories that may not exist.
3. **better-sqlite3 Behavior**
The underlying `better-sqlite3` library does not create parent directories automatically and fails immediately if they don't exist.
**Why It Worked in v0.4.6:**
Earlier versions may have had the database in the root directory or the user's environment had the directories pre-created.
**Why It Fails in v0.5.4:**
More tools were refactored to use project-specific databases (lines 254-280 in `intelligence-tools.ts`), exposing the missing directory creation logic across more code paths.
---
### Issue 2: Path Ambiguity & Wrong Directory Learning
**Symptom:**
The user reports that tools try to read from `/home` instead of the project directory.
**Root Cause Analysis:**
1. **Optional Path Parameters with Unsafe Defaults**
Most tools make the `path` parameter optional and fall back to `process.cwd()`:
```typescript
async getProjectBlueprint(args: { path?: string; includeFeatureMap?: boolean }) {
const projectPath = args.path || process.cwd(); // Unsafe default
// ...
}
```
2. **Unpredictable Working Directory in MCP Context**
When the MCP server is launched by a client (like VS Code with Copilot), `process.cwd()` depends on:
- Where the MCP client process was started
- The shell's current working directory
- The MCP client's configuration
This can result in `process.cwd()` returning unexpected paths like `/home/username` instead of the project root.
3. **Server Initialization Issue**
The MCP server command accepts a path argument but doesn't use it effectively:
```typescript
case 'server':
console.log('Starting In Memoria MCP Server...');
await runServer(); // Ignores args[1] which could contain the path
break;
```
4. **Ambiguous Copilot Instructions**
The `.github/copilot-instructions.md` file states:
> `path` is always relative to the project root.
But this doesn't enforce absolute paths or clarify what happens when `path` is omitted.
**Impact:**
- Tools may analyze the wrong directory
- Database files get created in unexpected locations
- Learning operations process incorrect or excessive files
---
### Issue 3: `contribute_insights` Missing Required Field
**Symptom:**
```
Error: MPC -32602: Invalid input for contribute_insights: content: Required
```
**GitHub Copilot's Actual Input:**
```json
{
"confidence": 0.9,
"sourceAgent": "github-copilot",
"type": "best_practice",
"sessionUpdate": { ... }
}
```
**Expected Input (per schema):**
```json
{
"type": "best_practice",
"content": {
// THIS IS REQUIRED
"key": "value"
},
"confidence": 0.9,
"sourceAgent": "github-copilot"
}
```
**Root Cause Analysis:**
1. **Schema Definition**
The tool schema correctly marks `content` as required (line 152-191 in `intelligence-tools.ts`):
```typescript
{
name: 'contribute_insights',
inputSchema: {
type: 'object',
properties: {
type: { ... },
content: {
type: 'object',
description: 'The insight content and details'
},
// ...
},
required: ['type', 'content', 'confidence', 'sourceAgent']
}
}
```
2. **Inadequate Documentation**
The Copilot instructions file doesn't provide a complete, valid example. The example shown is:
```typescript
await use_mcp_tool("in-memoria", "contribute_insights", {
type: "architectural_decision",
content: {
// Example shows this, but not clearly enough
decision: "Use React Query for all API calls",
reasoning: "Provides better caching and error handling",
affectedFiles: ["src/hooks/", "src/api/"],
},
confidence: 0.95,
sourceAgent: "github-copilot",
});
```
3. **AI Model Inference**
GitHub Copilot is inferring that `sessionUpdate` might be sufficient context, possibly conflating it with `content`. The schema's description for `content` is generic ("The insight content and details"), which doesn't help the AI understand what structure is expected.
**Why This Happens:**
- The `content` field accepts `type: 'object'` with no further schema definition
- The AI agent tries to be efficient and may think `sessionUpdate` serves the same purpose
- No examples in the instructions show what happens when `content` is missing
---
### Issue 4: Version Regression (v0.4.6 → v0.5.4)
**Symptom:**
Tools that worked in v0.4.6 now fail in v0.5.4 with database-related errors.
**Root Cause Analysis:**
1. **Increased Use of Project-Specific Databases**
In v0.5.4, the `learnCodebaseIntelligence` method creates project-specific database instances:
```typescript
async learnCodebaseIntelligence(args: { path: string; force?: boolean }) {
// ...
try {
const projectDbPath = config.getDatabasePath(args.path);
projectDatabase = new SQLiteDatabase(projectDbPath); // NEW in v0.5.4
projectVectorDB = new SemanticVectorDB(process.env.OPENAI_API_KEY);
// ...
}
}
```
This exposes the directory creation bug that may have been hidden in v0.4.6 where a shared database was used.
2. **Refactored Tool Architecture**
The Phase 4 implementation merged several tools (`get_learning_status` into `get_project_blueprint`, etc.), changing the code paths and revealing latent bugs.
3. **No Migration Guide**
Users upgrading from v0.4.6 to v0.5.4 received no guidance about:
- Changed configuration requirements
- New directory structure expectations
- Breaking changes in tool behavior
---
## Root Causes Summary
| Root Cause | Impact | Severity |
| -------------------------------------------- | -------------------------------------- | -------- |
| **Missing Directory Creation Logic** | Database initialization fails | Critical |
| **Inconsistent Path Handling** | Tools operate on wrong directories | Critical |
| **Schema-Documentation Mismatch** | AI agents call tools incorrectly | High |
| **No Configuration Validation** | Invalid configs accepted silently | High |
| **Lack of Defensive Programming** | Errors not caught early | Medium |
| **Breaking Changes Without Migration Guide** | Version upgrades break existing setups | High |
---
## Recommended Fixes
### Fix 1: Create Parent Directories for Database (Critical)
**File:** `src/storage/sqlite-db.ts`
**Change:**
```typescript
import Database from "better-sqlite3";
import { mkdirSync, existsSync } from "fs";
import { dirname } from "path";
import { DatabaseMigrator } from "./migrations.js";
export class SQLiteDatabase {
private db: Database.Database;
private migrator: DatabaseMigrator;
constructor(dbPath: string = ":memory:") {
// Ensure parent directory exists for file-based databases
if (dbPath !== ":memory:") {
const dir = dirname(dbPath);
if (!existsSync(dir)) {
console.error(`Creating database directory: ${dir}`);
mkdirSync(dir, { recursive: true });
}
}
this.db = new Database(dbPath);
this.migrator = new DatabaseMigrator(this.db);
this.initializeDatabase();
}
// ... rest of the class
}
```
**Impact:** Eliminates all "directory does not exist" errors.
---
### Fix 2: Validate Database Filename Configuration (High Priority)
**File:** `src/config/config.ts`
**Change:**
```typescript
getDatabasePath(projectPath?: string): string {
const basePath = projectPath || process.cwd();
const filename = this.config.database.filename;
// Warn if filename contains path separators
if (filename.includes('/') || filename.includes('\\')) {
console.warn(
'⚠️ Warning: IN_MEMORIA_DB_FILENAME contains path separators.\n' +
` Current: "${filename}"\n` +
' This may cause issues. Consider using a simple filename.\n' +
` The database directory is determined by the project path: ${basePath}`
);
}
const dbPath = join(basePath, filename);
console.error(`📁 Database path: ${dbPath}`);
return dbPath;
}
```
**Impact:** Users get clear warnings about configuration issues.
---
### Fix 3: Server Command Line Path Handling (High Priority)
**File:** `src/index.ts`
**Change:**
```typescript
case 'server':
// Accept optional path argument to set working directory
const serverPath = args[1];
if (serverPath) {
const { resolve } = await import('path');
const { existsSync } = await import('fs');
const resolvedPath = resolve(serverPath);
if (!existsSync(resolvedPath)) {
console.error(`❌ Error: Path does not exist: ${resolvedPath}`);
process.exit(1);
}
console.log(`Setting working directory to: ${resolvedPath}`);
process.chdir(resolvedPath);
}
console.log(`Starting In Memoria MCP Server from: ${process.cwd()}`);
await runServer();
break;
```
**Impact:** Server uses the correct project directory.
---
### Fix 4: Improve Copilot Instructions - Path Handling (Critical)
**File:** `.github/copilot-instructions.md`
**Add this section at the beginning:**
````markdown
## ⚠️ CRITICAL: Path Parameter Usage
**ALWAYS provide absolute paths to In-Memoria tools. NEVER rely on default values.**
### Why This Matters
In MCP server context, `process.cwd()` is unpredictable. Always specify the path explicitly:
```typescript
// ❌ WRONG - May use incorrect directory
await use_mcp_tool("in-memoria", "get_project_blueprint", {
includeFeatureMap: true,
});
// ✅ CORRECT - Always specify path
await use_mcp_tool("in-memoria", "get_project_blueprint", {
path: "/absolute/path/to/project", // or use workspace root
includeFeatureMap: true,
});
```
````
### Getting the Project Path
Use the workspace root in VS Code context:
- Access via `workspaceFolder` or similar API
- Ensure it's an absolute path
- Verify the path exists before calling tools
### Path Convention
All paths should be:
- **Absolute paths** (starting with `/` on Unix, `C:\` on Windows)
- **Pointing to the project root directory**
- **Consistent across all tool calls in a session**
````
---
### Fix 5: Improve `contribute_insights` Schema and Documentation (High Priority)
**File:** `src/mcp-server/tools/intelligence-tools.ts`
**Improve the schema description:**
```typescript
{
name: 'contribute_insights',
description: 'Allow AI agents to contribute insights back to the knowledge base. The `content` field must contain the actual insight details as a structured object.',
inputSchema: {
type: 'object',
properties: {
type: { ... },
content: {
type: 'object',
description: 'The insight content and details. REQUIRED. Must be a structured object describing the insight. Examples: {practice: "...", reasoning: "..."} or {bugPattern: "...", fix: "..."}',
additionalProperties: true
},
// ...
},
required: ['type', 'content', 'confidence', 'sourceAgent']
}
}
````
**File:** `.github/copilot-instructions.md`
**Update the example:**
````markdown
#### 10. **contribute_insights** - Record Learnings
Use for: Teaching In Memoria about the codebase
**⚠️ IMPORTANT:** The `content` field is REQUIRED and must contain the actual insight data.
```typescript
// ✅ CORRECT - content field with insight details
await use_mcp_tool("in-memoria", "contribute_insights", {
type: "best_practice",
content: {
// REQUIRED FIELD
practice: "Use React Query for all API calls",
reasoning: "Provides better caching and error handling",
affectedFiles: ["src/hooks/", "src/api/"],
},
confidence: 0.95,
sourceAgent: "github-copilot",
sessionUpdate: {
// OPTIONAL
feature: "API integration",
files: ["src/api/users.ts"],
},
});
// ❌ WRONG - missing content field
await use_mcp_tool("in-memoria", "contribute_insights", {
type: "best_practice",
confidence: 0.9,
sourceAgent: "github-copilot",
// ERROR: content is required!
});
```
````
````
---
### Fix 6: Add Path Validation Utility (Medium Priority)
**File:** `src/utils/path-validator.ts` (new file)
```typescript
import { existsSync } from 'fs';
import { resolve, isAbsolute } from 'path';
export class PathValidator {
/**
* Validate and resolve a path for tool usage
* @param path - The path to validate
* @param context - Context for error messages (e.g., tool name)
* @returns Resolved absolute path
* @throws Error if path is invalid
*/
static validateProjectPath(path: string | undefined, context: string): string {
// If no path provided, use cwd but warn
if (!path) {
const cwd = process.cwd();
console.warn(
`⚠️ Warning: No path provided to ${context}. ` +
`Using current directory: ${cwd}\n` +
' This may not be the intended project directory in MCP context.'
);
return cwd;
}
// Resolve to absolute path
const absolutePath = isAbsolute(path) ? path : resolve(process.cwd(), path);
// Check if path exists
if (!existsSync(absolutePath)) {
throw new Error(
`Invalid path for ${context}: ${absolutePath}\n` +
'Path does not exist. Please provide a valid project directory path.'
);
}
return absolutePath;
}
/**
* Check if a path looks like a project root
* (contains package.json, .git, or other project markers)
*/
static looksLikeProjectRoot(path: string): boolean {
const markers = [
'package.json',
'Cargo.toml',
'go.mod',
'requirements.txt',
'pyproject.toml',
'.git',
'pom.xml',
'build.gradle'
];
return markers.some(marker =>
existsSync(resolve(path, marker))
);
}
}
````
**Usage in tools:**
```typescript
async getProjectBlueprint(args: { path?: string; includeFeatureMap?: boolean }) {
const projectPath = PathValidator.validateProjectPath(args.path, 'get_project_blueprint');
if (!PathValidator.looksLikeProjectRoot(projectPath)) {
console.warn(
`⚠️ Warning: ${projectPath} doesn't look like a project root.\n` +
' No package.json, Cargo.toml, or other project markers found.'
);
}
// ... rest of the implementation
}
```
---
### Fix 7: Add Health Check Endpoint (Low Priority)
**File:** `src/mcp-server/tools/monitoring-tools.ts`
**Add a new tool:**
```typescript
{
name: 'health_check',
description: 'Verify In-Memoria setup and configuration',
inputSchema: {
type: 'object',
properties: {
path: {
type: 'string',
description: 'Project path to check'
}
}
}
}
```
**Implementation:**
```typescript
async healthCheck(args: { path?: string }): Promise<{
status: 'healthy' | 'warning' | 'error';
checks: {
name: string;
status: 'pass' | 'fail' | 'warning';
message: string;
}[];
}> {
const checks = [];
const projectPath = args.path || process.cwd();
// Check 1: Project path exists
checks.push({
name: 'Project Path',
status: existsSync(projectPath) ? 'pass' : 'fail',
message: existsSync(projectPath)
? `Path exists: ${projectPath}`
: `Path does not exist: ${projectPath}`
});
// Check 2: Database path is writable
try {
const dbPath = config.getDatabasePath(projectPath);
const dbDir = dirname(dbPath);
checks.push({
name: 'Database Directory',
status: existsSync(dbDir) || dbPath.includes(':memory:') ? 'pass' : 'warning',
message: existsSync(dbDir)
? `Directory exists: ${dbDir}`
: `Directory will be created: ${dbDir}`
});
} catch (error) {
checks.push({
name: 'Database Directory',
status: 'fail',
message: `Error: ${error}`
});
}
// Check 3: OpenAI API Key (if needed)
checks.push({
name: 'OpenAI API Key',
status: process.env.OPENAI_API_KEY ? 'pass' : 'warning',
message: process.env.OPENAI_API_KEY
? 'API key configured'
: 'No API key - vector embeddings disabled'
});
// Check 4: Project looks like a valid codebase
const looksValid = PathValidator.looksLikeProjectRoot(projectPath);
checks.push({
name: 'Project Structure',
status: looksValid ? 'pass' : 'warning',
message: looksValid
? 'Project markers found (package.json, .git, etc.)'
: 'No common project markers found - verify path is correct'
});
const hasFailures = checks.some(c => c.status === 'fail');
const hasWarnings = checks.some(c => c.status === 'warning');
return {
status: hasFailures ? 'error' : hasWarnings ? 'warning' : 'healthy',
checks
};
}
```
---
## Testing Plan
### Test Case 1: Database Directory Creation
```bash
# Setup
export IN_MEMORIA_DB_FILENAME=".mcp/in-memoria/memory.db"
mkdir -p /tmp/test-project
cd /tmp/test-project
# Test
in-memoria learn .
# Expected: Creates .mcp/in-memoria/ directory and memory.db file
# Verify
ls -la .mcp/in-memoria/memory.db
```
### Test Case 2: Path Parameter Validation
```typescript
// Test with Copilot
// 1. Call tool without path - should warn
await use_mcp_tool("in-memoria", "get_project_blueprint", {
includeFeatureMap: true,
});
// 2. Call tool with path - should succeed
await use_mcp_tool("in-memoria", "get_project_blueprint", {
path: "/absolute/path/to/project",
includeFeatureMap: true,
});
```
### Test Case 3: contribute_insights Validation
```typescript
// Test 1: Missing content - should fail with clear error
await use_mcp_tool("in-memoria", "contribute_insights", {
type: "best_practice",
confidence: 0.9,
sourceAgent: "test",
});
// Expected: Error message mentions 'content' field is required
// Test 2: Valid call - should succeed
await use_mcp_tool("in-memoria", "contribute_insights", {
type: "best_practice",
content: { practice: "test", reasoning: "test" },
confidence: 0.9,
sourceAgent: "test",
});
// Expected: Success
```
---
## Migration Guide for Users
### For Users Upgrading from v0.4.6 to v0.5.4+
1. **Update Configuration**
```json
// Before (may cause issues)
{
"env": {
"IN_MEMORIA_DB_FILENAME": ".mcp/in-memoria/memory.db"
}
}
// After (recommended)
{
"args": ["in-memoria", "server", "/absolute/path/to/project"],
"env": {
"IN_MEMORIA_DB_FILENAME": "in-memoria.db"
}
}
```
2. **Clean Existing Databases**
```bash
# Remove old database files if you had issues
find . -name "in-memoria.db" -delete
find . -name "*.db" -path "*/.mcp/*" -delete
```
3. **Re-learn Codebase**
```bash
cd /path/to/your/project
in-memoria learn .
```
### For New Users
1. **Configuration Template**
```json
{
"command": "npx",
"args": ["in-memoria", "server", "${workspaceFolder}"],
"env": {
"IN_MEMORIA_DB_FILENAME": "in-memoria.db",
"OPENAI_API_KEY": "your-key-here-optional"
}
}
```
2. **Initial Setup**
```bash
cd /path/to/your/project
in-memoria init
in-memoria learn .
```
---
## Monitoring and Diagnostics
### Symptoms to Watch For
1. **"Cannot open database"** errors → Fix 1 needed
2. **Learning processes /home or wrong directory** → Fix 2 or 3 needed
3. **"content: Required"** errors → Fix 5 needed
4. **Inconsistent behavior between runs** → Path validation issue
### Debug Commands
```bash
# Check database location
in-memoria check /path/to/project --verbose
# Verify configuration
node -e "console.log(require('path').join(process.cwd(), process.env.IN_MEMORIA_DB_FILENAME || 'in-memoria.db'))"
# Test learning with explicit path
in-memoria learn /absolute/path/to/project
```
---
## Future Improvements
1. **Configuration Validator CLI Command**
```bash
in-memoria validate-config
```
2. **Better Error Messages**
- Include resolution steps in error messages
- Link to documentation
- Show example of correct usage
3. **Automatic Path Detection**
- Detect workspace root from MCP client
- Fall back to git root detection
- Warn when detection is uncertain
4. **Database Migration Tool**
```bash
in-memoria migrate-database --from v0.4.6 --to v0.5.4
```
---
## Related Issues and References
- **Issue #9:** https://github.com/pi22by7/In-Memoria/issues/9
- **better-sqlite3 docs:** https://github.com/WiseLibs/better-sqlite3/wiki
- **MCP Protocol:** https://github.com/modelcontextprotocol
- **GitHub Copilot Custom Instructions:** https://docs.github.com/en/copilot/customizing-copilot/adding-custom-instructions-for-github-copilot
---
## Appendix: Code Locations
| Issue | File | Line(s) | Priority |
| --------------------------- | -------------------------------------------- | ---------- | -------- |
| Database directory creation | `src/storage/sqlite-db.ts` | 110-114 | Critical |
| Path validation | `src/config/config.ts` | 48-52 | High |
| Server path handling | `src/index.ts` | 50-52 | High |
| contribute_insights schema | `src/mcp-server/tools/intelligence-tools.ts` | 152-191 | High |
| Path defaults in tools | `src/mcp-server/tools/intelligence-tools.ts` | Multiple | High |
| Copilot instructions | `.github/copilot-instructions.md` | Throughout | Critical |
---
**Document Version:** 1.0
**Last Updated:** October 26, 2025
**Author:** GitHub Copilot Analysis
**Status:** Active - Awaiting Implementation