Skip to main content
Glama
miguelgarglez

ghost-audit-mcp

Ghost Audit - proving that "deleted" really means deleted

npm MIT licensed · Python 3.9+ standard library only · zero LLM calls at runtime

Ghost Audit is a deterministic deletion-audit agent for privacy / DPO teams. It does not trust workflow statuses: it reconstructs what personal data is actually still reachable across systems at the audit cut-off, and returns an ordered, evidence-cited remediation and verification plan. It ships as a CLI (ghost-audit) and a read-only stdio MCP server (ghost-audit-mcp).

Built for HackerRank Orchestrate October'26 - Ghost Data Still Exists. The submission window closed before the entry was uploaded, so this repository is published as a case study. The challenge dataset, the generated output.csv and the OCR transcriptions of challenge images are not redistributed here; a synthetic demo dataset ships instead.

Videos

Product overview (97.6 s, music + on-screen English text)

ghostaudit-launch-v5.mp4

Technical walkthrough (219 s, music + on-screen English text)

ghostaudit-tech-v4-en.mp4

Both are attached to the v1.0.1 release.

Related MCP server: eu-audit-mcp

Case study

The problem

A deletion request marked Complete in a workflow tool is a claim, not proof. Copies survive in search indexes, support tickets, analytics profiles, replicas and backups; sync connectors can re-create deleted records; and the person may be recorded under an e-mail or phone they no longer use. A privacy team answering a regulator needs to know, per request:

  1. Did the person really ask for deletion? (NOT_APPLICABLE otherwise - never delete a customer who only asked for a copy of their data.)

  2. What of the person is still reachable at audit_at? (records, identifiers still exposed in records that must be kept, leaky tombstones)

  3. What can bring it back? (active sync connectors, restorable backups)

  4. What must be done, in which order, and which checks prove it afterwards?

On the challenge's 518 evaluation requests, the workflow reported COMPLETE for 254 deletion requests; Ghost Audit found personal data still reachable in 163 of them (64%).

How it works

  • 0 LLM at runtime. Every answer is computed deterministically from the evidence files (JSON, SQL dumps, seven log dialects, TXT, and text from PDFs/screenshots). Same input, same output, about 1 s per full run, no API keys, no network.

  • Identity across time. A directory of every e-mail, phone and hash a person ever had (from the audit log) resolves records that only carry an old identifier; matching is exact on normalised identifiers, so look-alike decoys never match.

  • State at the audit cut-off. Only evidence captured and timestamped up to audit_at counts; if any replica still exposes a record, the record is exposed ("worst copy wins"). Reported statuses are never used as state.

  • Closed-world elimination + shared event clock. Resource ids that appear with no identifier (e.g. a deleted account's 404 lookup) are linked by eliminating every candidate already explained by other requests on the same day and claim, then using the constant seconds-of-minute offsets between one request's worker events as a shared clock. Remaining ties are flagged ambiguous, not guessed.

  • Policy table. deletion_rules.csv maps each data category to delete / redact / anonymize / retain / preserve-tombstone / block-restore; policy is a pure function (kind, state) -> finding / remediation / verification. Retained audit and legal records are verified present, never deleted.

  • Ordered plan. Findings, then remediation (stop flows and restores before touching copies), then verification, sorted deterministically.

  • confidence_v2 orders the human review queue. It is a priority signal, not a calibrated probability (only 32 labelled samples exist); output.csv never depends on it (CONFIDENCE.md).

Results and robustness

  • Public samples: deletion_status accuracy 100%, step F1 99.4% (cases 11-32) / 99.2% (all 32); details in evaluation_report.md and the provable ceiling in limits_and_ceiling.md. The sample set is small, so treat these as optimistic.

  • Stress harness: 29/29 adversarial scenarios (corrupt, empty, truncated or missing files, unknown dialects, BOM/CRLF/reordered CSVs, odd dates, 10x volume) finish without a crash and with a valid output (ROBUSTNESS.md). It needs the challenge dataset.

  • Tests: 54 in total. With the challenge dataset present all 54 pass. In this public repository the dataset-dependent modules skip with an explicit message and the demo/robustness-independent tests pass (python3 -m unittest discover -s code/tests).

Install

npx -y ghost-audit-mcp                                         # MCP server on stdio (needs python3 >= 3.9 on PATH)
npx -y -p ghost-audit-mcp ghost-audit demo                     # CLI on the bundled synthetic dataset ("Lumen Lane Books")
pipx install git+https://github.com/miguelgarglez/ghost-audit   # Python package from GitHub (also pip / uvx)
uvx --from git+https://github.com/miguelgarglez/ghost-audit ghost-audit demo

npm package: https://www.npmjs.com/package/ghost-audit-mcp. Not published on PyPI. Point the tool at your own data with --dataset DIR (or GHOST_DATASET). Full guide, uninstalling and from-source use: docs/INSTALL.md.

Use from an MCP client

{ "mcpServers": { "ghost-audit": { "command": "npx", "args": ["-y", "ghost-audit-mcp", "--dataset", "/path/to/dataset"] } } }

The server is read-only and exposes the same capabilities as the CLI (describe, summary, cases, case, explain, evidence, queue, ingest-report, whatif, verify). An agent builds whatever artefact the conversation needs (risk dashboard, review queue, case card, DPO brief): see AGENT_GUIDE.md.

From source

python3 code/ghost_cli.py demo                # synthetic demo, no dataset needed
python3 -m unittest discover -s code/tests   # tests (dataset-dependent ones skip without dataset/)
# with the challenge data in ./dataset/:
python3 code/main.py                          # writes output.csv
python3 code/tools/stress.py                  # 29 adversarial scenarios

Screenshots and PDFs are read from an OCR cache (--ocr-cache DIR) when present, otherwise through pdftotext / tesseract if installed; media that yields no text is reported and its cases go to manual review. code/tools/build_ocr_cache.py rebuilds a cache.

Architecture

flowchart LR
    subgraph IN[dataset/ - participant-facing only]
      R[deletion_requests.csv]
      I[user_identities + user_audit_log]
      A[system_activity.csv]
      P[deletion_rules.csv]
      E[evidence/: JSON, SQL, logs, TXT, PDF, PNG]
    end
    R --> INT[intent.py<br/>deletion vs access/export/opt-out]
    I --> DIR[dataset.py Directory<br/>current + historical emails, hashes, phones]
    E --> PAR[evidence.py + logparse.py + media.py<br/>7 log dialects, SQL replicas, bundles, OCR cache]
    PAR --> OBS[(Obs: system, resource, kind,<br/>state, time, identifiers, file)]
    DIR --> LNK[world.py<br/>identifier -> resource links<br/>+ data-catalog links]
    OBS --> LNK
    A --> ORPH[audit.py closed-world matching + shared event clock<br/>ids with no identifier]
    LNK --> ST[audit.py state at audit_at<br/>latest eligible observation,<br/>replica drift = worst copy wins]
    ORPH --> ST
    P --> POL[policy.py<br/>category -> required action]
    ST --> POL
    INT --> POL
    POL --> PLAN[plan: FINDING -> REMEDIATION -> VERIFICATION<br/>deterministic ordering]
    PLAN --> OUT[output.csv]
    PLAN --> TRAIL[audit_trail.jsonl + audit_summary.md]

Domain model in one paragraph

Everything is normalised into an observation Obs(file, captured_at, t, system, resource, kind, state, identifiers). A kind is a data category from deletion_rules.csv (profile, search entry, support message/attachment, analytics profile, export, audit log, legal hold, backup, tombstone...) plus DATA_FLOW for connectors. A state is what the evidence shows (PRESENT, ABSENT, IDENTIFIABLE, ANONYMIZED, REDACTED, ACTIVE, DISABLED, BLOCKED). Policy is a pure function (kind, state) -> violation? -> finding / remediation / verification operation.

Investigation approach

  • Identity - the request e-mail may be historical. The directory knows every e-mail/phone/hash a person ever had (audit log), so a record that only carries an old e-mail hash still resolves. Look-alike decoys (g.silva76.jr@...) never match because matching is exact on normalised identifiers.

  • State at audit_at - only observations with file captured_at <= audit_at and record time <= audit_at count. Later snapshots (re-synced data, restores) are ignored for state and never cited. Several copies of one record (primary, regional, reporting, read, failover replica): any exposed copy means exposed.

  • Claims are not proof - reported_status and activity reported_result are never used as state. The activity stream is used only to find resource ids (e.g. a deleted account whose 404 lookup carries no identifier) via closed-world matching on report day and claim.

  • Policy - delete / redact / anonymize / retain / preserve-tombstone / block-restore come from deletion_rules.csv; retained audit and legal records are verified present, never deleted.

  • Plan order - findings (remaining record, exposed PII, active flow, restore risk), then remediation (disable flows, block restores, delete, redact, anonymize), then verification (absent, present, redacted, anonymized, flow disabled, restore blocked); inside a phase sorted by system_id, resource_id. So dependent copies are never modified before the process that recreates them is stopped.

Metrics (full details: code/evaluation/evaluation_report.md)

public samples case_11-32 (same generator as the 518)

all 32 samples

deletion_status accuracy

100%

100%

Step F1 (exact tuple)

99.4%

99.2%

Row exact match

90.9%

90.6%

Output validation: all structural checks pass. Seven extra synthetic cases (historical identifiers, time-travel evidence, tombstone, non-deletion, restore + flow ordering, replica drift, all-compliant) pass 7/7; 54 tests (27 regression + 13 agent CLI/MCP + 4 robustness + 7 confidence + 3 demo) all pass with the challenge dataset. A per-case confidence_v2 orders the human review queue; it is a priority signal, not a calibrated probability (CONFIDENCE.md). 29 adversarial scenarios (python3 code/tools/stress.py) never crash the run and always yield a valid output.csv (ROBUSTNESS.md). Deterministic, ~1 s, zero LLM tokens (usage_report.md). The three remaining sample mismatches are provably not fixable from the data; the achievable ceiling is derived in limits_and_ceiling.md (about 98% step F1). Every requirement of the three spec documents is traced to code and tests in compliance_matrix.md; the ten key design decisions are in code/docs/DECISIONS.md.

Limitations (honest)

  • A resource id that never appears with an identifier in any evidence file cannot be recovered: ~17% of requests miss their retained-audit-record verification step. This is proven from the data (no unlinked audit ids exist, archival events are unrelated noise, ids are not derivable) in limits_and_ceiling.md; we omit rather than invent ids.

  • Account / anonymised-analytics ids without an identifier are linked by closed-world elimination on the activity stream plus a shared event clock (the seconds-of-minute offsets between a request's worker events are constant per event type). Hiding the identifier of random accounts and recovering them gives ~88% right; 18 requests remain a coin-flip and are flagged ambiguous in the audit trail.

  • Screenshots are read with OCR + page-type rules, not a vision model; unseen console layouts need a mapping entry.

  • Public samples are small (22 representative cases) - treat the accuracy numbers as optimistic.

Roadmap

  1. Connect to live systems (account, search, support, analytics, backup APIs) instead of snapshots; schedule continuous deletion audits.

  2. Execute the plan behind approvals: disable flow, block restore, delete/redact/anonymize, then run the verification steps and attach the signed audit trail.

  3. Vision-language model fallback for unseen screenshot layouts and multilingual free-text requests, kept behind the deterministic policy core.

  4. Probabilistic linking with confidence scores and a reviewer queue for ambiguous matches; regulator-ready PDF report per request.

Repository layout

code/main.py              entry point (writes ../output.csv)
code/ghost_cli.py         read-only agent CLI (JSON); code/ghost_mcp.py: stdio MCP server; logic in code/ghost/agent.py
code/ghost/               model, dataset, evidence parsers, logparse, media (OCR), world (links), audit (state + plan), policy, intent, io,
                          diag (ingest report), timeutil (tolerant timestamps), confidence (confidence_v2)
code/evaluation/          main.py (scoring + reports), validate_output.py, custom_cases.py, masking_test.py, ceiling_analysis.py,
                          evaluation_report.md, usage_report.md, audit_summary.md, limits_and_ceiling.md, compliance_matrix.md
code/tests/               regression + agent CLI/MCP + robustness + confidence tests (python3 -m unittest discover -s code/tests)
code/docs/DECISIONS.md    the 12 key technical decisions (mirrored to docs/); AGENT_GUIDE.md: CLI/MCP for agents;
                          ROBUSTNESS.md: behaviour on unseen/malformed input, limits, how to extend;
                          CONFIDENCE.md: confidence_v2 method, reliability tables, limits
code/tools/               build_ocr_cache.py, package.py (portable packager), stress.py (adversarial harness)
Makefile                  make all | run | validate | test | eval | stress | package
package.sh                builds code.zip
SUBMISSION.md             what was prepared for the HackerRank upload slots

Available Tools

10 tools
ghost_describeDescribe Ghost AuditA
Read-onlyIdempotent

Call first. Capabilities, glossary of statuses/phases/operations, field meanings and known limits.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare read-only, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description adds useful context by saying it exposes capabilities, glossary definitions, field meanings, and known limits, which tells the agent what kind of reference information to expect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short, front-loads the key instruction 'Call first', and then lists the useful content categories without filler. Every element earns its place for a zero-parameter orientation tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only describe tool with no output schema, the description is complete enough to invoke correctly and tells the agent the categories of information returned. It could be slightly more explicit about what the output looks like, but it covers the essential onboarding role.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and the schema fully covers them, so parameter semantics are not a meaningful differentiator here. The baseline for a zero-parameter tool is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies this as a first-call orientation tool that provides capabilities, status/phase/operation glossary, field meanings, and known limits. That is clear enough to distinguish it from operational siblings like ghost_get_case or ghost_summary, though it does not explicitly say it 'returns documentation' and relies partly on the title.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Call first' gives explicit usage timing relative to the sibling tools. It does not state when not to use it or name a specific alternative, so it falls short of a full when/when-not/alternatives guideline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_explain_caseExplain caseA
Read-onlyIdempotent

Deterministic 8-15 line markdown narrative of one decision with [evidence/activity] citations. Quote it, do not paraphrase facts.

NameRequiredDescriptionDefault
maskNomask e-mails, phones and names
case_idYese.g. case_33

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, closed-world behavior, so the description is free to add value — and it does, disclosing determinism, an 8-15 line bound, markdown format, and that citations are embedded. With no output schema, this is the only place the return shape is described; only error behavior (e.g., unknown case_id) is unstated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler. The output specification comes first and the citing directive last, both of which earn their place for a tool whose whole value is a quotable narrative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only tool with full schema coverage and no output schema, the description covers format, length, determinism, and citation content — enough to call it and use the result correctly. The remaining gap is positioning against the very similar ghost_summary/ghost_get_case siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including the mask flag's effect ('mask e-mails, phones and names') and an example value for case_id. The description adds no parameter meaning beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (one decision/case) and precisely characterizes the output: a deterministic 8-15 line markdown narrative with evidence/activity citations. That is far more specific than a restatement of the title, but it never distinguishes this from ghost_summary or ghost_get_case, so an agent must guess how the narrative differs from a summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The only guidance is a downstream instruction ('Quote it, do not paraphrase facts'), which governs how to consume the output, not when to call this tool. There is no statement of when to choose ghost_explain_case over ghost_summary or ghost_get_case, and no prerequisites or exclusion cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_get_caseGet caseA
Read-onlyIdempotent

Full context of one case: request, subject and identifier history, resources with rule/proof/link/activity ids, planned steps, review signals and why claimed differs from verified.

NameRequiredDescriptionDefault
maskNomask e-mails, phones and names
case_idYese.g. case_33

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. With no output schema, the description carries the burden of describing the return payload, and it does so concretely by naming the sections returned, including the claimed-vs-verified divergence. It still omits error behavior for an unknown case_id and any size/cost characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with the resource and its contents front-loaded; every clause names a distinct part of the payload rather than padding. It is a fragment rather than a complete sentence, which slightly reduces readability but not information density.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read tool with full schema coverage and complete annotations, the only genuine gap is the return payload, and the description enumerates it. An agent knows what it gets back and that the call is safe and idempotent; only edge-case behavior (missing case, payload size) is unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: case_id has a format example ('case_33') and mask is documented as masking e-mails, phones and names. The description adds no further semantics about either parameter, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource (one case) and enumerates exactly what the payload contains: request, subject/identifier history, resources with rule/proof/link/activity ids, planned steps, review signals, and claimed-vs-verified delta. That is far more specific than a tautology. It does not, however, distinguish itself from close siblings like ghost_explain_case or ghost_get_evidence, leaving the agent to infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use statement, and no sibling is named as an alternative. Usage is only implied by the breadth of the enumerated contents ('full context'), which suggests it fits when the agent needs everything about a case at once.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_get_evidenceGet evidenceA
Read-onlyIdempotent

Metadata of an evidence file (path, origin, captured_at, resources seen) and a text snippet from the file or OCR cache; binaries are never returned.

NameRequiredDescriptionDefault
grepNoonly lines containing this text (e.g. a resource id)
linesNo
evidence_idYese.g. evidence_18

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, and closed-world behavior. The description adds meaningful behavioral context beyond those annotations by stating that a text snippet may come from the file or OCR cache and that binaries are never returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is a single compact sentence with the return contents front-loaded and no wasted wording. The parenthetical metadata list and the binaries exclusion are both useful and efficiently placed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only retrieval tool with no output schema, the description covers the main returned data and the no-binaries constraint. However, it does not explain the output shape, how grep/lines govern the snippet, or what happens when OCR cache is unavailable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%, with evidence_id and grep documented but lines lacking a schema description. The description adds context by saying a text snippet is returned, but it does not explicitly explain how lines or grep shape that snippet, so it only partially compensates for the remaining gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states exactly what is returned for an evidence file: metadata fields plus a text snippet, with binaries explicitly excluded. It is specific about the resource and return scope, but does not explicitly distinguish this tool from siblings such as ghost_summary or ghost_describe.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for retrieving evidence details, but it gives no explicit when-to-use guidance, exclusions, or alternatives. With several sibling tools available, the agent receives no routing help for choosing ghost_get_evidence over them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_ingest_reportIngest reportA
Read-onlyIdempotent

Ingestion diagnostics: evidence files that could not be parsed, ignored rows/log lines, CSV issues and cases degraded to INCOMPLETE + needs_manual_review. Empty on the shipped dataset.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and closed-world, so the safety profile is covered. The description adds real value beyond that: it discloses the report's scope (parse failures, ignored rows, degraded cases) and the important behavioral fact that it is empty on the shipped dataset, preventing a wasted call or a false-negative interpretation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that names the report, lists its contents, and closes with the most decision-relevant fact (empty on shipped data). No filler or restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters, no output schema, and full annotation coverage, the description carries its share by enumerating what the report returns. A minor gap remains: it doesn't indicate the output format or explicitly position itself against ghost_review_queue, which covers the similarly-named needs_manual_review cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline of 4 applies. The description correctly implies a no-argument, no-filtering report rather than suggesting options that don't exist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific resource (ingestion diagnostics) and enumerates exactly what the report contains: unparsed evidence files, ignored rows/log lines, CSV issues, and cases degraded to INCOMPLETE + needs_manual_review. That content list distinguishes it from generic siblings. It stops short of a clean verb framing ('returns/reads'), but the intent is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The note 'Empty on the shipped dataset' is a genuine usage signal that tells an agent not to expect output in the common case. However, it does not state when to call this versus the overlapping siblings ghost_review_queue or ghost_verify_output, nor does it say whether to call it proactively or only after an ingest failure.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_list_casesList casesA
Read-onlyIdempotent

Filter cases; returns compact rows (status, risks, steps, confidence, needs_manual_review). Use before ghost_get_case.

NameRequiredDescriptionDefault
qNosubstring of case_id, e-mail or user_id
riskNodata category, e.g. USER_EXPORT, SUPPORT_MESSAGE
limitNo
statusNo
claimedNo
needs_reviewNo
include_samplesNoalso include the public sample cases

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description contributes the row contents returned (status, risks, steps, confidence, needs_manual_review), but says nothing about ordering, pagination, or whether filters combine with AND or OR — behavior an agent would want before paging through 600 rows.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence pair with zero boilerplate; the return shape and the follow-up pointer are both front-loaded, and every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-param filter tool with no output schema the description does name the returned columns, which is helpful. But it leaves the status/claimed distinction, filter composition, and limit default (50, max 600) unexplained, which are exactly the ambiguities the structured fields cannot resolve.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 43%, and the description adds no parameter meaning at all; it only names the return fields. Most critically, the two enum params status (COMPLETE/INCOMPLETE/NOT_APPLICABLE) and claimed (COMPLETE/PARTIALLY_COMPLETE/NOT_STARTED) are near-identical in surface form, and only the description could explain how an assessed status differs from a claimed status — it does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

A specific verb (filter) plus resource (cases), with the returned row shape spelled out so the agent knows what it gets back. It also positions itself relative to ghost_get_case, letting an agent separate listing from fetching without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use before ghost_get_case" gives a clear sequencing rule covering the main workflow. It does not address alternatives such as ghost_review_queue for needs_manual_review=true, nor whether this is the right tool for bulk exports, so it stops short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_review_queueReview queueA
Read-onlyIdempotent

Prioritised human review queue (withheld/ambiguous links, inferred states, false COMPLETE with active risk) with reasons, a score and confidence_v2 (a review-priority signal, not a probability).

NameRequiredDescriptionDefault
sortNoconfidence_v2 = least certain firstpriority
limitNo
include_samplesNoalso include the public sample cases

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive and closed-world, so the bar is low; the description adds real semantic context by defining what is queued and by warning that confidence_v2 is a review-priority signal, not a probability — a genuine misreading risk it pre-empts. It does not discuss volume, truncation, or empty-queue behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with the headline concept front-loaded, then item classes, then output fields, then the confidence_v2 caveat. The parenthetical list is long but every element carries information, and nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-required-param read-only list tool with full annotations and no output schema, the description conveys both what is returned and the key interpretation caveat. The remaining gaps — result shape, ordering guarantee, limit ceiling semantics — are minor and partly covered by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67% and the two documented params (sort, include_samples) already carry descriptions, so the schema does most of the work. The description's confidence_v2 gloss does clarify what sort='confidence_v2' means, but it says nothing about limit or how samples affect results — a baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete artifact — a prioritised human review queue — and enumerates the item classes it contains (withheld/ambiguous links, inferred states, false COMPLETE with active risk) plus the payload fields (reasons, score, confidence_v2). An agent can distinguish it from ghost_summary or ghost_list_cases on content, though no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The word 'human review' implies a triage/audit use case, but there is no explicit statement of when to prefer this over ghost_list_cases or ghost_summary, and no exclusions or prerequisites. Usage is inferable rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_summaryAudit summaryA
Read-onlyIdempotent

Portfolio view: verified/claimed status counts, claimed->verified transitions (false COMPLETE), top risk categories, violations by system, manual-review counts. Good for dashboards and executive briefs.

NameRequiredDescriptionDefault
include_samplesNoalso include the public sample cases

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered by structured data. The description still adds real value by disclosing the returned analytical content, including the notable 'claimed->verified transitions (false COMPLETE)' concept an agent could not guess. It does not mention cost, aggregation window, or freshness, which keeps it below 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense sentence plus a short usage clause, front-loaded with 'Portfolio view', with zero filler. The telegraphic noun-list style ('claimed->verified transitions (false COMPLETE)') is efficient but slightly cryptic without domain context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must carry the return-value burden, and it does so by naming the five categories of aggregate data returned. For a read-only, zero-required-parameter aggregate tool this is close to sufficient; only the response format and the time scope of the aggregates are left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter include_samples is already documented with its default in the schema. The description adds nothing about that parameter, so the baseline 3 for schema-covered parameters applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description enumerates concretely what the tool returns: verified/claimed status counts, claimed->verified transitions flagged as false COMPLETE, top risk categories, violations by system, and manual-review counts. That makes the resource unmistakable and differentiates it from case-level siblings like ghost_get_case and ghost_list_cases. It falls short of 5 because there is no explicit verb and it never names the sibling it replaces for aggregate questions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Good for dashboards and executive briefs' implies the intended consumption context, so an agent can infer this is the roll-up entry point. But there is no explicit when-not guidance and no named alternative (e.g. ghost_list_cases for per-case detail), leaving routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_verify_outputVerify output.csvB
Read-onlyIdempotent

Structural validation of output.csv (contract, vocabulary, ids, phase order) plus sha256 and status counts.

NameRequiredDescriptionDefault
pathNodefaults to the repository output.csv

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the safety profile is covered. The description usefully adds that it performs four named structural checks and computes a sha256 and status counts, but it does not disclose whether a failed validation returns a report or raises an error, which is the critical behavioral detail for a validation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with zero filler; the resource and the checks are front-loaded and the auxiliary outputs (sha256, status counts) follow compactly. Nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must convey what comes back; it hints at sha256 and status counts but leaves the shape of validation results and failure behavior unspecified. Adequate for a simple read-only check but with a clear gap around return semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single optional parameter with 100% schema description coverage, including the default ('defaults to the repository output.csv'), so the schema does the heavy lifting. The description adds no path semantics beyond what the schema already states, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Structural validation') plus the exact resource (output.csv) and enumerates the checks performed (contract, vocabulary, ids, phase order) along with sha256 and status counts. It does not explicitly distinguish itself from siblings, but none of the listed siblings overlap with this artifact validation task.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this tool versus alternatives, nor any prerequisite or exclusion guidance. The usage is only implied by the artifact name; an agent must infer that this is the pre-flight check on output.csv.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ghost_whatifWhat-if thresholdA
Read-onlyIdempotent

In-memory re-run with another MIN_PICK_CONFIDENCE (default 0.30): which cases would change or become manual (~1 s), and/or which cases a confidence_v2 review threshold would route to a human. Never writes output.csv.

NameRequiredDescriptionDefault
min_confidence_v2No
min_pick_confidenceNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false. The description adds useful behavioral context beyond that: the re-run is in-memory, takes roughly one second, and never writes output.csv, which is consistent with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is a single dense but front-loaded sentence that leads with the core in-memory re-run purpose and follows with outcomes and the no-write guarantee. It is appropriately sized, with only slight awkwardness from the 'and/or' phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only what-if tool with two optional numeric parameters and no output schema, the description gives enough context about what results to expect: changed/manual cases and confidence_v2 human routing. It could be more complete by stating defaults or behavior for the confidence_v2 threshold, but the core call context is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry parameter meaning. It clarifies MIN_PICK_CONFIDENCE with a default of 0.30 and references the confidence_v2 review threshold, but it does not explicitly map the latter to min_confidence_v2 or explain its default, leaving a gap for the second parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description defines a specific what-if operation: an in-memory re-run with a different MIN_PICK_CONFIDENCE and/or confidence_v2 threshold to see which cases would change, become manual, or route to a human. It distinguishes this from tools that write outputs by stating it never writes output.csv, though it does not explicitly compare itself to sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the what-if framing: use it to simulate threshold changes and review routing without committing results. However, there is no explicit when-to-use/when-not-to-use guidance or named alternative among the sibling tools, leaving the agent to infer boundaries.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 10 tool updatesv1.0.1
    • First observedghost_describe
    • First observedghost_explain_case
    • First observedghost_get_case
    • First observedghost_get_evidence
    • First observedghost_ingest_report
    • First observedghost_list_cases
    • First observedghost_review_queue
    • First observedghost_summary
    • First observedghost_verify_output
    • First observedghost_whatif

TDQS

A3.8/5.0

Scored across 10 tools

Disambiguation5/5

Each tool targets a distinct task: describe (glossary), summary (portfolio), list_cases (filter rows), get_case (full case context), explain_case (narrative with citations), get_evidence (file metadata/snippet), review_queue (human queue), whatif (re-run simulation), ingest_report (ingestion diagnostics), verify_output (contract validation). The read-side case tools (list/get/explain) operate at clearly different granularity and the descriptions even guide sequencing ('Use before ghost_get_case').

Naming Consistency4/5

All names share the consistent 'ghost_' prefix and snake_case convention, which is good. Most are verb_noun (get_case, list_cases, explain_case, verify_output, ingest_report), but a few are noun/other-style (summary, review_queue, whatif), a minor deviation from a strict pattern.

Tool Count5/5

Ten tools is well within the ideal range and each earns its place: reporting, per-case inspection, evidence access, review, simulation, ingestion, and validation are all non-redundant functions. No filler tools present.

Completeness4/5

The audit/verification lifecycle is well covered: ingestion diagnostics, portfolio summary, case listing/inspection/explanations, evidence, review queue, what-if simulation, and output validation. Minor gaps exist—no tool to retrieve full evidence content (binaries intentionally withheld) and no write/resolve action for review items, but these reflect deliberate read-only design.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Provides read-only MCP tools for market snapshots, position risk, order reconciliation, and daily report previews with deterministic financial calculations, evidence chains, and audit trails.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Tamper-evident audit trail MCP server for EU AI Act and GDPR compliance. It enables logging events with automatic PII scanning, compliance checks, and right-to-erasure support.
    Apache 2.0
  • F
    license
    Not graded
    quality
    C
    maintenance
    Enables querying enterprise records and retention policies from any MCP client over stdio, with read-only tools for searching records, fetching retention verdicts, identifying archival candidates, summarizing departments, forecasting retentions, and viewing audit history.
    -
  • F
    license
    A
    quality
    A
    maintenance
    Provides coding agents read-only access to Depot's CI failure diagnoses, container build forensics, run history, cache effectiveness, registry contents, and usage data through MCP tools.
    28
    47 npm
    1
    -