metacognitivemonitoring
Assess and improve reasoning quality by systematically monitoring knowledge boundaries, claim certainty, and biases. Supports metacognitive evaluation across planning, execution, and reflection stages.
Instructions
A detailed tool for systematic self-monitoring of knowledge and reasoning quality. This tool helps models track knowledge boundaries, claim certainty, and reasoning biases. It provides a framework for metacognitive assessment across various domains and reasoning tasks.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| claims | No | ||
| iteration | Yes | Current iteration of the monitoring process | |
| knowledgeAssessment | No | ||
| monitoringId | Yes | Unique identifier for this monitoring session | |
| nextAssessmentNeeded | Yes | Whether further assessment is needed | |
| overallConfidence | Yes | ||
| reasoningSteps | No | ||
| recommendedApproach | Yes | ||
| stage | Yes | ||
| suggestedAssessments | No | ||
| task | Yes | ||
| uncertaintyAreas | Yes |
Implementation Reference
- Core handler function that validates input, formats detailed monitoring report, logs it, and returns structured JSON response with success/error status.public processMetacognitiveMonitoring(input: unknown): { content: Array<{ type: string; text: string }>; isError?: boolean } { try { const validatedData = this.validateInputData(input); const formattedOutput = this.formatOutput(validatedData); console.error(formattedOutput); return { content: [{ type: "text", text: JSON.stringify({ task: validatedData.task, stage: validatedData.stage, monitoringId: validatedData.monitoringId, iteration: validatedData.iteration, overallConfidence: validatedData.overallConfidence, nextAssessmentNeeded: validatedData.nextAssessmentNeeded, status: 'success' }, null, 2) }] }; } catch (error) { return { content: [{ type: "text", text: JSON.stringify({ error: error instanceof Error ? error.message : String(error), status: 'failed' }, null, 2) }], isError: true }; } }
- src/models/interfaces.ts:179-230 (schema)TypeScript interfaces defining the complete data structure for metacognitive monitoring input, used for validation in the handler.export interface KnowledgeAssessment { domain: string; knowledgeLevel: | "expert" | "proficient" | "familiar" | "basic" | "minimal" | "none"; confidenceScore: number; supportingEvidence: string; knownLimitations: string[]; relevantTrainingCutoff?: string; } export interface ClaimAssessment { claim: string; status: "fact" | "inference" | "speculation" | "uncertain"; confidenceScore: number; evidenceBasis: string; falsifiabilityCriteria?: string; alternativeInterpretations?: string[]; } export interface ReasoningAssessment { step: string; potentialBiases: string[]; assumptions: string[]; logicalValidity: number; inferenceStrength: number; } export interface MetacognitiveMonitoringData { task: string; stage: | "knowledge-assessment" | "planning" | "execution" | "monitoring" | "evaluation" | "reflection"; knowledgeAssessment?: KnowledgeAssessment; claims?: ClaimAssessment[]; reasoningSteps?: ReasoningAssessment[]; overallConfidence: number; uncertaintyAreas: string[]; recommendedApproach: string; monitoringId: string; iteration: number; suggestedAssessments?: ("knowledge" | "claim" | "reasoning" | "overall")[]; nextAssessmentNeeded: boolean; }
- src/index.ts:532-687 (registration)MCP Tool definition including name, description, and detailed inputSchema matching the TS interfaces for registration.const METACOGNITIVE_MONITORING_TOOL: Tool = { name: "metacognitivemonitoring", description: `A detailed tool for systematic self-monitoring of knowledge and reasoning quality. This tool helps models track knowledge boundaries, claim certainty, and reasoning biases. It provides a framework for metacognitive assessment across various domains and reasoning tasks.`, inputSchema: { type: "object", properties: { task: { type: "string" }, stage: { type: "string", enum: [ "knowledge-assessment", "planning", "execution", "monitoring", "evaluation", "reflection", ], }, knowledgeAssessment: { type: "object", properties: { domain: { type: "string" }, knowledgeLevel: { type: "string", enum: [ "expert", "proficient", "familiar", "basic", "minimal", "none", ], }, confidenceScore: { type: "number", minimum: 0, maximum: 1 }, supportingEvidence: { type: "string" }, knownLimitations: { type: "array", items: { type: "string" }, }, relevantTrainingCutoff: { type: "string" }, }, required: [ "domain", "knowledgeLevel", "confidenceScore", "supportingEvidence", "knownLimitations", ], }, claims: { type: "array", items: { type: "object", properties: { claim: { type: "string" }, status: { type: "string", enum: [ "fact", "inference", "speculation", "uncertain", ], }, confidenceScore: { type: "number", minimum: 0, maximum: 1, }, evidenceBasis: { type: "string" }, falsifiabilityCriteria: { type: "string" }, alternativeInterpretations: { type: "array", items: { type: "string" }, }, }, required: [ "claim", "status", "confidenceScore", "evidenceBasis", ], }, }, reasoningSteps: { type: "array", items: { type: "object", properties: { step: { type: "string" }, potentialBiases: { type: "array", items: { type: "string" }, }, assumptions: { type: "array", items: { type: "string" }, }, logicalValidity: { type: "number", minimum: 0, maximum: 1, }, inferenceStrength: { type: "number", minimum: 0, maximum: 1, }, }, required: [ "step", "potentialBiases", "assumptions", "logicalValidity", "inferenceStrength", ], }, }, overallConfidence: { type: "number", minimum: 0, maximum: 1 }, uncertaintyAreas: { type: "array", items: { type: "string" } }, recommendedApproach: { type: "string" }, monitoringId: { type: "string", description: "Unique identifier for this monitoring session", }, iteration: { type: "number", minimum: 0, description: "Current iteration of the monitoring process", }, suggestedAssessments: { type: "array", items: { type: "string", enum: ["knowledge", "claim", "reasoning", "overall"], }, }, nextAssessmentNeeded: { type: "boolean", description: "Whether further assessment is needed", }, }, required: [ "task", "stage", "overallConfidence", "uncertaintyAreas", "recommendedApproach", "monitoringId", "iteration", "nextAssessmentNeeded", ], }, };
- src/index.ts:1123-1136 (registration)Registration of the tool handler in the main CallToolRequestSchema switch statement, dispatching to the server instance.case "metacognitivemonitoring": { const result = metacognitiveMonitoringServer.processMetacognitiveMonitoring( request.params.arguments ); return { content: [ { type: "text", text: JSON.stringify(result, null, 2), }, ], }; }
- Helper method for input validation, checking required fields and type/range constraints before processing.private validateInputData(input: unknown): MetacognitiveMonitoringData { const data = input as MetacognitiveMonitoringData; if (!data.task || !data.stage || !data.monitoringId) { throw new Error("Invalid input for MetacognitiveMonitoring: Missing required fields."); } if (typeof data.overallConfidence !== 'number' || data.overallConfidence < 0 || data.overallConfidence > 1) { throw new Error("Invalid overallConfidence value for MetacognitiveMonitoringData."); } if (typeof data.iteration !== 'number' || data.iteration < 0) { throw new Error("Invalid iteration value for MetacognitiveMonitoringData."); } if (typeof data.nextAssessmentNeeded !== 'boolean') { throw new Error("Invalid nextAssessmentNeeded value for MetacognitiveMonitoringData."); } return data; }