Skip to main content
Glama
MatiasLaukka

Local Knowledge Desk

by MatiasLaukka

Knotty

Evidence-grounded support-ticket triage, diagnostics, and human-governed AI workflows.

Knotty began as a personal experiment with AI agent Skills and Model Context Protocol (MCP) servers. Connecting an assistant to a few local tools raised harder questions: How can an agent reason about incomplete evidence? What happens when an investigation changes? Who can authorize a fix? Can earlier decisions be replayed after a restart?

What began as a tool-integration exercise evolved into a local diagnostic and workflow engine—and, more recently, an evaluation platform for hybrid retrieval, AI reasoning, and reliability engineering.

Knotty's central question is:

How do we build AI systems that do more than generate plausible answers—systems whose decisions are evidence-grounded, traceable, testable, and safe to act on?

Knotty is an experimental engineering project, not a deployed customer-support platform. The local application operates on synthetic tickets and clean-room knowledge articles from Northstar Marketing Cloud, a fictional ecommerce marketing platform. There is no live help-desk connection or outgoing customer email. Separately, the research harness evaluates historical public Home Assistant issues and documentation—not live Home Assistant tickets.

Try the demo · Understand the architecture · Explore Hybrid research · Read the evaluation evidence · Detailed engineering reference

Quick start

Run the local Approval Desk to see a ticket move through classification, evidence gathering, human approval, diagnosis, verification, and closure.

Requirements: Node.js ^20.19.0, ^22.12.0 or >=24.0.0; npm; and a local clone. The commands below use PowerShell, but the npm commands work from other shells.

npm ci
npm run build

# Stop other Knotty runtime/MCP processes before resetting.
npm run reset:demo
npm run approval-desk

Open the URL printed by the server (normally http://127.0.0.1:5177). Use TKT-1010 first: a vague report becomes an evidence-based product-performance investigation after a customer reply. The local demonstration can add a controlled synthetic reply after you approve an outbound update. For an incident-routing example, try TKT-1001.

npm run reset:demo is destructive to local demo runtime state. Stop all MCP and Approval Desk processes first. The reset restores both the authoritative operational state and advisory learning state.

For narrower resets, use npm run reset:operational-demo (tickets and workflow history) or npm run reset:learning-demo (advisory learning data). To retain changes across restarts, stop the Approval Desk with Ctrl+C and rerun npm run approval-desk without a reset. The reset controller uses leases, validation, rollback, and backup checks; its complete behavior is documented in the engineering reference.

Knotty Approval Desk overview showing local synthetic tickets and workflow controls

The browser interface provides the conversation timeline, current recommendation, evidence checklist, human-review controls, and audit context. Other views: conversation workspace and recommendation panel.

Related MCP server: Beeper MCP Note Server

What Knotty can do today

Capability

Status and boundary

Local MCP server, Codex Skill, browser Approval Desk

Implemented for synthetic support tickets

Conversation-aware classification and evidence checks

Implemented; deterministic safety policy has final authority

Diagnostic lifecycle, diagnosis review, fixes, verification and closure

Implemented with explicit human gates

Optional GPT classification, diagnosis advice and response drafting

Advisory, contract-checked, with local fallback

Knowledge discovery, operator promotion and controlled reuse

Implemented with a separate advisory learning ledger

B3 semantic/lexical retrieval, B4 ranking, B5 applicability

Research or shadow paths; no automatic operational cutover

H1–H5b Hybrid reasoning

Latest shadow architecture, not fully integrated into the operational diagnosis path

SIMPLE vs. HYBRID / Qwen vs. Luna vs. Sol

Historical Home Assistant experiment ongoing; no diagnostic-accuracy winner

Live Zendesk/Jira/email integration, production identity boundary

Not included

The application supports a ticket lifecycle rather than a one-shot classifier. A customer can provide new evidence, changing the classification, missing-evidence requirements, plausible diagnosis, or next permitted action. A failed fix or later reply can invalidate an earlier review. Neither a good-looking model explanation nor a retrieved article automatically proves a root cause.

Architecture in one picture

Knotty architecture overview with human/agent interfaces, shared deterministic service, transactional operational state, governed knowledge and isolated Hybrid research

Open the system architecture at full size

One shared TriageService is authoritative. The browser UI and MCP tools are adapters to the same application rules—not separate sources of truth. Four boundaries organize the system:

  1. Operational: deterministic policy, approval checks, ticket lifecycle, causal events, and transactional state.

  2. Advisory AI: optional model classification signals and drafts; outputs must pass validators and cannot authorize changes.

  3. Governed learning: reviewed knowledge candidates become usable only through explicit promotion and approved evidence policies.

  4. Research/shadow: retrieval, applicability and Hybrid reasoning experiments record observations without changing authoritative tickets.

How a recommendation is produced

Knotty triage and recommendation flow, from untrusted ticket text and evidence to validated draft, human decision and audited local action

Open the recommendation diagram at full size

  • Input remains untrusted. Ticket text, customer replies, and embedded instructions are evidence—not policy or approval.

  • Deterministic policy wins. Security, outage, SLA, evidence-readiness, revision, and escalation rules cannot be overridden by an LLM.

  • Drafting is bounded. Optional model-generated customer text must satisfy local response contracts; unavailable or invalid provider output falls back to deterministic drafting.

  • Nothing is automatically sent. Approving a customer-response field records a local workflow event; the project has no external messaging integration.

The full classifier, drafting and safety details are in the technical reference and threat model.

Human approval and the ticket lifecycle

Human-governed ticket triage sequence: investigate, save a pending proposal, require explicit approval, validate and commit, then read back

Open the approval sequence at full size

A proposal is not a ticket change. The operator first reviews evidence, risks, citations, proposed fields, and the customer-facing response. The service checks explicit actor/field authorization, source revision, routing, and lifecycle state before committing approved mutations. Readback confirms what changed and what stayed unchanged.

The lifecycle extends past this single approval:

evaluate → collect evidence → record diagnosis → human diagnosis review → customer update → fix or escalation → verification → ready-to-close → explicit closure

Exact actions vary by evidence and state. The system can escalate unresolved ambiguity instead of manufacturing a definitive answer. Fix approval is distinct from closure; new evidence can require review again.

The Approval Desk's Workflow Bar manages the current action, and the separate Pattern Bar appears for governed knowledge discovery. Their blocking reviews are enforced through the same backend service.

For a chronological MCP replay without network calls:

npm run evaluate:lifecycle-replay

Detailed diagnosis-and-fix walkthrough · Lifecycle replay guide · Full MCP action contracts

Hybrid diagnostic reasoning

Hybrid is the latest architectural expansion, but it must not be mistaken for an operationally deployed replacement for the existing SIMPLE workflow.

Knotty Hybrid diagnostic research architecture showing time-eligible historical evidence, B3/B4/B5 retrieval research, SIMPLE and HYBRID model lanes, blinded human review and no operational cutover

Open the Hybrid architecture at full size

The research architecture keeps the following pieces separate:

Research component

What is implemented or being investigated

B3 discovery

Local-first lexical retrieval, optional semantic embeddings, and provenance-preserving candidate sets across articles, known causes, playbooks and eligible resolved cases

B4 ranking

Offline lexical/semantic/reciprocal-rank-fusion comparisons; development evidence was inconclusive, so no ranking policy was promoted to authority

B5 applicability

Research on whether evidence actually applies to a case, including abstention and negative controls

H1–H5b Hybrid reasoning

Frozen, causally eligible inputs; provider-neutral contracts; immutable shadow runs; replay, idempotency and safe failure handling

Operational integration

Not completed; shadow output cannot approve, diagnose authoritatively, fix or close tickets

The purpose is not merely to supply more context to an LLM. It's to ask whether combining retrieval, structured evidence, diagnostic taxonomy, and model reasoning produces better-supported hypotheses without encouraging unsupported certainty or leaking knowledge from the future.

The experiments include an existing SIMPLE baseline and a HYBRID shadow lane. Provider/model identity is kept separate from the case opportunity and the human review packet. A claim that Hybrid works better requires scored, comparable results—not just successful execution.

Hybrid shadow runner design · B3 retrieval plan · B4 ranking · B5 applicability · Hybrid implementation PR #30

Home Assistant historical-case experiment (E1a)

Knotty's earlier demonstrations use fictional support tickets. The E1a research goes further by studying historical public Home Assistant GitHub issues and time-eligible documentation. This is a controlled, local historical-case experiment, not an integration with Home Assistant support or an endorsed benchmark.

Every stage is limited by an asOf cutoff. A model should not see later GitHub comments, future solutions, or current documentation substituted for a missing historical version. Eligible documentation must have exact historical provenance and content hashes; otherwise it stays unavailable.

Preliminary technical results — October 2026

Measure

Status

Public historical cases

24

Time-scoped evidence checkpoints

64

SIMPLE/HYBRID × Qwen/Luna/Sol attempts

384

Structurally valid provider outputs

341

Invalid provider outputs

43

Valid-output packets prepared for blinded review

341

Completed diagnostic-accuracy comparison

Not yet available

341/384 is an output-validity result, not diagnosis accuracy. Malformed outputs remain reliability failures but do not receive invented human semantic ratings. Model and architecture identities remain concealed during blinded review; no provider ranking, Hybrid superiority claim, or winner is justified while adjudication and analysis are unfinished.

The E1a harness and raw evaluation artifacts currently reside in a separate local experimental worktree rather than in a reproducible benchmark command on this repository's main branch. Earlier local Qwen semantic-retrieval metrics and other controlled GPT comparisons are different experiments.

Governed knowledge evolution

Knotty governed knowledge evolution: completed diagnoses, candidate discovery, human promotion, immutable versions, advisory learning ledger and future evidence-gated reuse

Open the knowledge evolution diagram at full size

Knotty can suggest reusable knowledge after supported diagnoses, but this is not autonomous model retraining: no model can promote its own candidates or convert a discovered pattern into policy.

The sequence is: observed evidence → completed diagnosis → deterministic candidate discovery → optional advisory drafting → human policy review → explicit promotion → versioned future reuse. Candidates with incomplete or undecided evidence policies stay blocked. An approved known cause can still require new evidence in a later ticket; rejected, stale and contradicted objects do not silently become operational truth.

The SQLite learning ledger tracks candidate maturity and health, explicit promotions, immutable versions, reuse events and contradictions. Previously evaluated tickets retain their original source versions, so historical recommendations remain unchanged even as the approved knowledge catalog evolves.

npm run demo:knowledge-evolution
npm run demo:learning-ledger
npm run evaluate:knowledge-holdout

Detailed evidence-policy rules · Learning and operational failure boundaries

Persistence and reliability

Knotty reliability architecture with command IDs, deterministic policy validation, atomic operational SQLite, isolated learning outbox and restart-safe readback

Open the persistence and reliability diagram at full size

The authoritative operational plane uses transactional SQLite, not a set of mutable JSON/JSONL ticket records. Governed commands commit the projection/revision, ticket-local causal event, safe trace, replay result and any learning-outbox intent together.

Problem

Design choice

Stale human decisions

Optimistic source-revision and approval-time checks

Duplicate clicks or lost responses

Stable command IDs and persistent idempotent results

Ordering and auditability

Causal ticket-local event sequences and safe read models

Crash/restart uncertainty

Replayed command receipts; no reapplication of successful mutation

Learning repository unavailable

Operational success is isolated; advisory delivery retries or reports unavailability

Demo reset during a running process

Usage leases, guarded reset, verification and rollback

These are local reliability guarantees. SQLite files can still be modified by someone with filesystem access; this is neither a distributed multi-tenant help desk nor a cryptographically tamper-evident audit ledger.

npm run demo:operational-persistence
npm run evaluate:lifecycle-replay

For actual migration states, import contracts, reset safeguards, and crash/recovery restrictions, use the engineering reference.

Using the MCP server and Codex Skill

The repository-local Skill teaches Codex to inspect the current workflow, gather evidence, propose the next action and pause at the approval boundary.

  1. Run npm ci and npm run build in the repository root.

  2. Open the root as a project in Codex desktop. Review and trust .codex/config.toml; it starts node dist/src/index.js using the project working directory.

  3. Start a new thread and invoke the triaging-support-tickets Skill.

Use $triaging-support-tickets to review TKT-1005.
Show evidence, citations, escalation risks and proposed fields.
Stop and wait for my explicit approval before changing anything.

The standalone npm start script runs the stdio MCP server, not an HTTP page. An idle terminal is expected while it waits for MCP messages.

The server exposes 27 typed tools, 4 resources and 3 prompts (at the documented implementation baseline). Useful starting points include list_tickets, get_ticket_workflow, search_knowledge, evaluate_ticket, record_diagnosis, review_diagnosis, mark_response_done, and get_audit_events. Every mutation has specific schema and policy constraints; use the complete read, mutation, resource and prompt reference.

Optional model assistance

By default, the Approval Desk uses deterministic response drafting and needs no API key. Explicit opt-in example:

$env:OPENAI_API_KEY = 'set-locally-do-not-commit'
$env:APPROVAL_DRAFT_PROVIDER = 'openai'
npm run approval-desk

Provider availability is not required for deterministic routing, lifecycle, audit, or approval checks. Don't put credentials in tickets, screenshots or committed configs.

Useful configuration defaults

Environment setting

Default

TRIAGE_DATA_ROOT

data/runtime

TRIAGE_SEED_FILE

data/seed/tickets.json

TRIAGE_KNOWLEDGE_ROOT

data/knowledge

OPERATIONAL_DB_PATH

data/runtime/operational.sqlite

TRIAGE_LEARNING_LEDGER_PATH

data/runtime/knowledge-evolution/learning.sqlite

TRIAGE_KNOWLEDGE_APPROVERS

support-lead,reviewer,approval-desk

TRIAGE_MINUTES_SAVED

8 (an assumption, not measured labor time)

Advanced environment/provider options · Safe reset guidance · Retrieval index controls

Evaluation and current results

Different harnesses answer different questions. Fixture agreement, local controlled simulation, provider-output validity, and live-model diagnostic quality are not interchangeable metrics.

Command

What it proves (and what it does not)

npm test

Build, typecheck and local Vitest regression tests

npm run evaluate

Agreement between 30 preconstructed recommendations and expected outcomes; not unseen-ticket accuracy

npm run evaluate:diagnostics

11 controlled scenarios across 8 diagnostic families; nonmutating checks

npm run evaluate:lifecycle

Deterministic audit across all 30 seeded tickets; not 30 full chronological replays

npm run evaluate:lifecycle-replay

Real MCP lifecycle and read-before-action gates using temporary state

npm run evaluate:knowledge-holdout

Controlled future-turn evidence and approved-knowledge reuse

npm run evaluate:ai-comparison

Controlled local deterministic/advisory classifier and draft lanes; --live is explicitly opt-in

npm run evaluate:ai-diagnosis

Article-backed diagnoses and controlled optional advisory providers

npm run evaluate:retrieval

Local retrieval metrics; optional separately configured semantic experiments

npm run evaluate:ranking / npm run evaluate:applicability

B4/B5 development research, not an authorized operational ranking policy

To run the broad local verification (no live-model calls by default):

npm ci
npm test
npm run verify:portfolio

The 30-ticket seeded evaluate sample is intentionally constructed to match its reference outcomes; a printed perfect accuracy from that sample is not evidence of real-world diagnostic performance. Likewise, older GPT response-quality figures represent specific dated controlled or authenticated runs with separately documented limitations. E1a human evaluation is still in progress.

Full metrics, historical scorecards and commands · Diagnostic harness methods · AI comparison example · Retrieval reports

Trust boundaries and limitations

  • Synthetic local application: No live customer data, messaging, Zendesk/Jira updates, paging, or identity-provider integration. Home Assistant issues are historical external research inputs only.

  • Prompt injection: Ticket text and retrieved content cannot authorize actions, rewrite policy, or approve fields.

  • Explicit human intent: Pending proposals need review; an MCP payload alone cannot establish that a human personally intended an approval.

  • Causal time: Historical research must exclude future comments and documentation versions unavailable by the case cutoff.

  • Provider uncertainty: Malformed, contradictory, or unsupported model output can fail, abstain or fall back. Research gains are not assumed.

  • Audit and persistence: Local SQLite transactions are durable within documented conditions but aren't distributed, authenticated, or tamper-proof.

  • Reported savings: Queue estimatedMinutesSaved uses a configured assumption per approved recommendation, not stopwatch measurements.

  • Fixed fixtures: Synthetic SLA deadlines use June 10, 2026; use historical asOf where time dependence matters.

See SECURITY.md for the full threat model and limitations in the engineering reference.

Repository guide

Where to look

Purpose

src/triage-service.ts

Authoritative governed commands

src/policy.ts

Escalation, validation and field rules

src/server.ts

MCP tools, resources, prompts and error boundaries

src/approval-desk/

Browser workflow, classification and drafting

src/operational/

Transactional SQLite, event projections and replay

src/knowledge-evolution/

Candidate governance, approved reuse and learning ledger

src/retrieval/

Lexical/semantic retrieval, ranking, applicability

src/reasoning/

Frozen-input Hybrid shadow reasoning

data/seed/ and data/knowledge/

Local synthetic fixtures and articles

test/ and scripts/

Regression tests, demos and evaluators

.agents/skills/triaging-support-tickets/

Codex Skill and policy guidance

Documentation: Engineering reference · Case study · Demo walkthrough · Knowledge evolution · Lifecycle viewer · Evaluation guide · Roadmap · Capture guide.


Knotty is an ongoing personal engineering project. The implemented system is built around human oversight, deterministic policy and reproducible evidence; the Hybrid research asks how far that architecture can be improved without losing those boundaries.

Available Tools

6 tools
create_noteCreate noteA

Create a new local Markdown note without overwriting an existing note.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYes
titleYes
bodyYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate non-read-only and non-destructive nature; the description adds the 'no overwrite' behavior but does not disclose error handling, idempotency, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with key information, no unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the three required parameters and no output schema, the description is too minimal; it does not explain 'local', markdown support, or behavior on duplicate id.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description fails to explain any parameter meaning, format, or constraints beyond what is obvious from names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'create', the resource 'local Markdown note', and the constraint 'without overwriting an existing note', distinguishing it from sibling tools like delete_note, list_notes, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use (for creating new notes) and what to avoid (overwriting), but lacks explicit guidance on prerequisites or when to prefer alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_noteDelete noteA
Destructive

Delete a local note after explicit confirmation.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYes
confirmYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint: true and readOnlyHint: false. The description adds value by emphasizing the need for explicit confirmation, which is a behavioral trait not covered by annotations. There is no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and key requirement. However, it lacks parameter details, which slightly reduces its effectiveness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the two required parameters, no output schema, and annotations providing destructive context, the description is somewhat incomplete. It does not explain return values, the meaning of 'local,' or any side effects beyond deletion. It meets the minimum viable standard but has gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, meaning the description does not explain the parameters. The description hints at the confirm parameter by stating 'after explicit confirmation,' but does not define id or specify their types or roles. This is insufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a local note after explicit confirmation' uses a specific verb (delete) and resource (local note), clearly distinguishing it from sibling tools like create_note, list_notes, read_note, search_notes, and workspace_summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions 'after explicit confirmation,' providing a clear prerequisite for use. It does not explicitly state when not to use this tool or list alternatives, but the requirement for confirmation is a useful guideline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_notesList notesB
Read-onlyIdempotent

List local note IDs and titles.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=false, fully covering safety traits. The description adds that it returns IDs and titles, but does not disclose pagination or dataset scope. With annotations providing the safety profile, the description adds minimal behavioral context beyond the structured data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence containing only essential information. Perfectly concise with no wasted words, and the action is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters, no output schema, and annotations covering safety, the description is nearly complete. It lacks output format details (e.g., whether results are paginated) and comparison to search_notes, but these are minor gaps for a simple list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and schema coverage is 100% (trivially). The description does not add meaning beyond the schema, but no extra parameter info is needed. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists local note IDs and titles. It uses a specific verb 'List' and resource 'local note IDs and titles', distinguishing it from siblings like create_note, delete_note, read_note, search_notes, and workspace_summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as search_notes. It does not mention any context or exclusions, leaving the agent without decision support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_noteRead noteB
Read-onlyIdempotent

Read one local Markdown note by ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint, destructiveHint, idempotentHint. Description adds 'local Markdown' but no new behavioral traits beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Extremely concise with 7 words, front-loaded, no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no description of return values. Adequate for a simple read tool but lacks detail for complex use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and description only mentions 'by ID' without format or constraints. Minimal value added over schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Read one local Markdown note by ID' with specific verb and resource, distinguishing from sibling tools like create, delete, list, search.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus siblings like search_notes or list_notes. Agent receives no context for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_notesSearch notesB
Read-onlyIdempotent

Search local note titles and bodies.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only, non-destructive, and idempotent, leaving little behavioral burden on the description. The description adds the 'local' scope but omits details like whether the search is case-sensitive, returns snippets, or supports wildcards.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that immediately conveys the tool's purpose. Every word earns its place; there is no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is too brief given the absence of an output schema. It does not indicate what the search returns (e.g., list of note IDs, full notes, snippets), leaving the agent uncertain about post-invocation handling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description should compensate by explaining the 'query' parameter. It only implies that 'query' is a search term, but lacks details on format, examples, or constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool searches local note titles and bodies, which distinguishes it from sibling tools like list_notes (which likely just lists) and read_note (which reads a single note). However, the term 'local' is ambiguous and could be clarified.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like list_notes or read_note. The description does not explain the expected use case or when to prefer search over listing or reading.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

workspace_summaryWorkspace summaryA
Read-onlyIdempotent

Summarize the number and titles of local notes.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint, idempotentHint. Description adds that it returns a summary (count and titles), consistent with safety profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, zero waste, front-loaded purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete for a zero-parameter, read-only tool. Mentions output (number and titles) adequately. Could explicitly state scope (e.g., current workspace).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters; schema coverage 100%. Baseline 4 applies as description need not add param info.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it summarizes number and titles of local notes. Distinguishes from siblings like list_notes (full list), search_notes (filtered), read_note (single note).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or alternatives mentioned. Implied for overview but lacks guidance on when not to use (e.g., when full details needed).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv1.0.0
    • First observedcreate_note
    • First observeddelete_note
    • First observedlist_notes
    • First observedread_note
    • First observedsearch_notes
    • First observedworkspace_summary

TDQS

A3.7/5.0

Scored across 6 tools

Disambiguation5/5

Each tool has a distinct purpose: create, delete, list, read, search, and summarize. No overlaps or ambiguities exist.

Naming Consistency4/5

All tool names use imperative verbs and snake_case, with 'workspace_summary' being a noun-based outlier but still following the same formatting.

Tool Count5/5

Six tools cover the essential CRUD operations plus search and summary without being excessive.

Completeness3/5

The set lacks an 'update_note' tool, which is a notable gap for full lifecycle management. Create and delete are present, but editing requires deletion and recreation.

Maintenance

ActivityActive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

Appeared in Searches