Skip to main content
Glama

GroundLens: the answer and its passages go through GroundLens first, then an LLM judge, then human review. A policy decides, and every check leaves a signed record.

Guardrails for LLM answers

PyPI License OpenSSF Scorecard SLSA

              

   

Quickstart

# pip install groundlens
import groundlens as gl

verifier = gl.Verifier(sink=gl.sinks.jsonl("records/"))
record = verifier.check(answer="Revenue was €42.1 million.", passages=["Revenue reached €41.2 million."])
print(record.decision, record.reasons)
FAIL ['c2: groundlens.numeric contradicted: €42.1 million not in sources; nearest 41200000 EUR in ctx-0 (relative distance 0.021)']

Runs locally: no account, no API key, no model, no network. The signed record is in records/ before check returns.

Related MCP server: Sentry MCP

Integration

The same check, wired into your stack:

from langfuse import get_client
verifier = gl.Verifier(sink=gl.sinks.jsonl("records/"), observers=[gl.integrations.langfuse(get_client())])

Call check inside the observation that produced the answer, and its trace gets:

  • Score groundlens.decision: categorical, PASS, REVIEW or FAIL, on the check and on its trace. Filter traces by it, chart it, export it.

  • Observation groundlens.check: type guardrail. The reasons are its status message, one per line. Its level follows Langfuse's own scale: PASS is DEFAULT, REVIEW is WARNING, FAIL is ERROR.

  • Metadata of the check: decision, reason codes (groundlens.scope.contradicted), policy, and the id and hash of the signed record behind it.

  • Judge: a generation observation with the model name, next to your own model calls.

  • Spans: one per verifier, plus the store, each with its duration. Log View shows them in milliseconds.

  • SDK: Python v3 and v4. Tested on Langfuse Cloud and self-hosted Langfuse 4.58.

verifier = gl.Verifier(sink=gl.sinks.jsonl("records/"), observers=[gl.integrations.otel()])

Spans and metrics go through the providers your application already configured: Tempo, Prometheus, or any OpenTelemetry backend.

  • Metrics: groundlens_decisions_total{decision, policy}, groundlens_reasons_total{reason, decision}, groundlens_check_duration_seconds{stage} with stage total, engine, judge, sign, store.

  • Spans: groundlens.check with groundlens.decision, groundlens.reasons, record id and hash; children per verifier, the judge (gen_ai.*) and the store.

  • Dashboard: the one in the image. gl.integrations.grafana_dashboard() returns the path of its JSON; import it in Dashboards → New → Import.

  • PromQL, share of answers that failed:

    sum(rate(groundlens_decisions_total{decision="FAIL"}[15m])) / sum(rate(groundlens_decisions_total[15m]))
  • Alert when more than 5 % fail:

    - alert: GroundLensFailRateHigh
      expr: sum(rate(groundlens_decisions_total{decision="FAIL"}[15m])) / sum(rate(groundlens_decisions_total[15m])) > 0.05
      for: 10m
  • Try it: docker run -p 3000:3000 -p 4318:4318 grafana/otel-lgtm, then python examples/observability/credit_copilot.py grafana --minutes 5. Tested on Grafana 12 with Tempo and Prometheus.

Your assistant checks its own draft before it answers. Three read-only tools over stdio: verify_answer, verify_run, verify_records. Needs uv.

Add to Cursor Install in VS Code

  • Claude Code (plugin: the MCP server plus a skill that tells Claude when to check):

    claude plugin marketplace add groundlens-dev/groundlens
    claude plugin install groundlens@groundlens
  • Codex:

    codex mcp add groundlens -- uvx --from "groundlens[mcp]" groundlens mcp
  • Any MCP client (Claude Desktop, Windsurf, Zed...):

    { "mcpServers": { "groundlens": { "command": "uvx", "args": ["--from", "groundlens[mcp]", "groundlens", "mcp"] } } }

    With the package installed (pip install "groundlens[mcp]"), "command": "groundlens-mcp" is enough.

  • Nothing to install: the Hugging Face Space is also a remote MCP server, with verify_answer, at https://groundlens-groundlens.hf.space/gradio_api/mcp/.

A skill on ClawHub, the skill registry of OpenClaw. Before your assistant sends an answer with figures taken from your documents, it runs groundlens verify on its draft:

  • PASS: the answer goes out.

  • FAIL: it fixes the figure from the source and checks again.

  • REVIEW: the answer goes out, saying which figure it could not confirm.

Runs on your machine, with no API key and no second model. The skill is in plugins/groundlens/skills/groundlens, the same one the Claude Code plugin uses. OpenClaw can also use the MCP server directly, with openclaw mcp add.

check() returns only after the signed record is stored. If it cannot be stored, check() raises and no decision comes back. Each sink keeps a chain: every record carries the hash of the one before it.

Sink

What you pass

How it stays append-only

gl.sinks.jsonl("records/")

a directory or a .jsonl path

one JSON line per record, flushed to disk and locked on every write. For development and single machines

gl.sinks.s3_object_lock(boto3.client("s3"), "bucket")

your S3 client and a bucket created with Object Lock

one object per record, written only if absent; retain_days= adds a retention date (COMPLIANCE mode by default)

gl.sinks.azure_blob(container_client)

an azure.storage.blob.ContainerClient

one blob per record, never overwritten; give the container a time-based immutability policy

gl.sinks.postgres(psycopg.connect(...))

any DB-API connection

one row per record; the table is created if missing, and its keys stop a position from being written twice or a chain from forking

import boto3
verifier = gl.Verifier(
    signer=gl.signers.aws_kms(boto3.client("kms"), key_id),           # the key never leaves KMS
    sink=gl.sinks.s3_object_lock(boto3.client("s3"), "audit-records", retain_days=3650),
)

Several processes: give each its own chain (chain="worker-3") and verify them together with groundlens record verify. GroundLens depends on none of these SDKs: you pass the client you already use. Postgres, S3 Object Lock, AWS KMS, Azure Blob and PKCS#11 are tested in CI against real software (Postgres, MinIO, LocalStack, Azurite, SoftHSM); Azure Key Vault has no emulator and is experimental.

LLM-as-a-judge

The exact checks catch a figure that is wrong, miscalculated, from another company or of another kind than asked. They cannot catch a real figure taken from the wrong row of the right table, or a sentence the passages never say. For that you connect a model that reads: the judge. GroundLens ships no model; you pass the client you already use.

from openai import OpenAI

verifier = gl.Verifier(
    "credit_desk.yaml",                                       # a policy that allows a judge, below
    judge=gl.judges.openai_compatible(OpenAI(), "gpt-4.1"),
    sink=gl.sinks.jsonl("records/"),
)

Adapter

Client

gl.judges.openai_compatible(client, model)

openai.OpenAI, or any server with the same API: vLLM, Ollama, your gateway (base_url=)

gl.judges.azure_openai(client, deployment)

openai.AzureOpenAI

gl.judges.bedrock(client, model_id)

boto3.client("bedrock-runtime")

The shipped policies forbid a judge, so the policy that uses one says so:

id: credit_desk
version: 1.0.0
extends: eu_ai_act_high_risk_v1
verifiers:
  forbidden: []          # allow llm_judge.*
judge:
  required: true         # no judge, or a judge that fails: REVIEW
  contradicted: REVIEW   # or FAIL
  unsupported: REVIEW

The judge can escalate, never pass. It labels each sentence supported, unsupported, contradicted or uncertain. supported resolves nothing on its own, so a judge that is wrong, or that a passage has talked into agreeing, cannot let through an answer the exact checks fail. A timeout, an error or a malformed reply is REVIEW.

You see exactly what it is sent. One prompt per answer, from a fixed template (groundlens.judge/1): what PASS means for you, the scope, the question, the answer and every passage with its provenance. Passages sit between markers they cannot contain, so text inside them is data, not instructions. groundlens judge show-template prints the exact prompt before anything is sent anywhere.

The record keeps the trail. The template id, the hash of the prompt, the model the provider says answered and the hash of its reply, with each sentence's label, the passages it cited and its one-line reason. In Langfuse the judge appears as a generation, next to your own model calls.

Architecture

GroundLens splits verification into five parts that never mix: claims, verifiers, evidence, policy and record.

  1. The answer is cut into claims: each figure and each sentence, with its character offsets.

  2. Verifiers examine each claim against the passages and return evidence (supported, contradicted, unsupported), never a verdict. The shipped ones are exact: numeric (exact fractions, locales, units, recomputed sums and ratios), scope, intent, and your rules. The judge, if you connect one, is the model you already use, asked through a fixed template.

  3. The evidence graph keeps what each verifier said about each claim, and where they disagree.

  4. The policy, a YAML file you own, turns evidence into a decision: PASS, REVIEW or FAIL, with reasons.

  5. The record holds the input hash, the evidence, the policy hash and the decision, chained to the previous record and signed. It is stored before check returns; observers are told after.

The engine is Rust, shipped as a Python wheel (abi3, Python 3.10+), with a standalone glv binary that checks records without Python. It runs in your process: no server, no network, no runtime dependencies. From Rust, cargo add groundlens; for the binary, cargo install groundlens-cli.

Parameters

Parameter of Verifier

Options

policy

groundlens_default_v1 (default), eu_ai_act_high_risk_v1, or your YAML, which can extends either

signer

local_key (development), aws_kms, azure_key_vault, pkcs11

sink

jsonl, s3_object_lock, azure_blob, postgres

judge

openai_compatible (OpenAI, vLLM, Ollama…), azure_openai, bedrock; optional

observers

integrations.langfuse, integrations.otel, or your own

locale, rounding

how numbers are read and compared

Parameter of check

answer, passages

required; passages as strings or (id, text, provenance)

question

enables the intent check: the answer gives the kind and unit asked for

scope

what the request is about ({"entity": "ACME"}): a figure from another entity fails

risk_class

a class your policy can send to human review whatever the checks say

Policy

The checks produce evidence. The policy decides what that evidence means for you: what fails, what goes to a person, and what PASS allows. It is a short YAML file that you own and version with your code. The same answer under two policies can get two different decisions, and both are correct; that difference is your risk appetite, written down.

Two policies ship with GroundLens:

Policy

For

groundlens_default_v1 (default)

any application. A contradicted figure or a figure from outside the scope fails; an answer of the wrong kind or unit goes to review. No model decides.

eu_ai_act_high_risk_v1

systems under Annex III of the EU AI Act. The same rules, with each decision mapped to Art. 15(1) accuracy, Art. 14(4)(a) human oversight and Art. 12(1) record keeping.

Write your own policy

Extend a shipped policy and write only what differs:

id: credit_desk
version: 1.2.0
extends: eu_ai_act_high_risk_v1
pass_means: "may be shown to the analyst as a summary of the filing"
locale: es-ES                     # how the documents write numbers: 1.234,56
rounding: half_even               # how a recomputed figure may round
scope:
  required: [entity]              # every request must name the borrower
  missing_scope: REVIEW           # or FAIL; never PASS
risk_classes:
  default: credit_information
  mandatory_review: [credit_limit_change]   # always seen by a person
verifier = gl.Verifier("policies/credit_desk.yaml")
verifier.check(answer=a, passages=p, scope={"entity": "Talleres Norte"}, risk_class="credit_limit_change")

Key

What it sets

extends

a shipped policy to start from. Maps merge key by key, lists replace. Your policy must set its own id and version.

pass_means

what PASS allows in your application, in one sentence. A verifier's PASS is not a decision to publish; this says which one it is.

locale, rounding

how figures are read and compared: half_up, half_even or within_last_digit. Written here, they win over the request.

scope

the keys every request must carry, and what a missing one gets.

risk_classes

classes that always go to review. A class the policy does not list is an error, so a typo cannot route around review.

verifiers

which checks are required, recommended, optional, fallback or forbidden.

decision

which verifiers can fail an answer, which send it to review, and what unresolved or conflicting claims get.

judge

whether a judge is required and what its labels may do. See LLM-as-a-judge.

regulatory_mapping

which control and article each decision evidences, in your own control ids.

Unknown keys are errors, not ignored: a misspelt rule never silently does nothing.

Failure is not a PASS

Situation

Decision

a figure contradicts the passages, under the rounding rule

FAIL

a figure comes only from a passage outside the scope

FAIL

a required scope key is missing

REVIEW (or FAIL, if the policy says so)

a risk class that requires review

REVIEW, whatever the checks say

a check could not run, or the judge timed out or failed

REVIEW

the record cannot be stored

an exception, and no decision

The final decision is the worst outcome of all checks.

Check your policy

groundlens policy lint policies/credit_desk.yaml                # errors before you deploy
groundlens policy show --resolved policies/credit_desk.yaml     # every default filled in

The hash of the resolved policy goes into every record it decides, so each decision can be traced to the exact rules that produced it.

Before you change a policy,you have to measure it:

  • groundlens measure estimates the share of wrong answers among those that passed, from a small sample your team labels without seeing the verdicts, with a 95 % confidence interval, per risk class, and the cost per decision.

  • groundlens replay runs past traffic under the new policy and lists the decisions that would change.

Both reports are signed. Full reference: Writing policies.

Security

GroundLens sits on the path between a model and a decision, and its records are meant to be evidence. Both make it a target: an attacker may try to get a wrong answer through, or to alter the trail afterwards.

Threat

Control

A wrong answer passes

Exact checks with no model in the default path; errors and timeouts are REVIEW, never PASS; no stored record, no decision

Instructions hidden in passages steer the judge

Passages are delimited data in a fixed, versioned template; the judge can only escalate; adversarial tests

A record is edited, deleted or reordered

SHA-256 content hash, hash chain, signature; groundlens record verify checks offline with the public key alone

The signing key leaks

Keys stay in AWS KMS, Azure Key Vault or a PKCS#11 HSM (ECDSA P-256); GroundLens never holds them

Two writers fork a chain

Conditional writes in every sink; a lost race is retried, never overwritten

Data leaves through the verifier

No network code in the engine (cargo-deny bans HTTP and TLS crates); other systems are reached only through clients you pass in

Hostile input exhausts the parser

Bounded numeral grammar against regular-expression DoS; fuzzing of every parser

A tampered release

Pinned and hashed CI dependencies, SLSA Build Level 3 provenance, signed release binaries, OpenSSF Scorecard

Report vulnerabilities privately through GitHub security advisories. See SECURITY.md.

Contributing

If GroundLens got one of your answers wrong, that case is the most valuable contribution there is. Checks, connectors and documentation are welcome too; CONTRIBUTING.md explains how to start.

Open source

Apache-2.0, all of it: engine, checks, policies, connectors, command line and MCP server. Verification evidence has to be checkable by anyone, without our software or our permission, and that only holds if the verifier is open.

Contact

You can contact the developer at javier@groundlens.dev

1. GroundLens 6.0.5, default policy, no judge. Financial answers: 189 real answers by Claude Haiku from a RAG over FinQA 10-K pages (e5 embeddings, top 4 passages), on multi-step questions never used while developing GroundLens. Each answer labelled twice by a language model (Claude Opus 5.5, claude-opus-5-5) against fixed written criteria, blind to GroundLens, with a third pass on the 3 disagreements; labels were frozen before GroundLens ran. 22 answers were wrong (1 in 9): 17 got FAIL or REVIEW (77 %, 95 % CI 57–90 %). Of the 129 answers that passed, 5 were wrong (3.9 %, 95 % CI 1.7–8.8 %). The price: 60 of 189 answers (32 %) went to FAIL or REVIEW, including 38 of 110 correct ones. Median 10.6 ms per check in a fresh process. Public claims: LLM-AggreFact test split (29,320 claim–document pairs from 11 sets, human labels), never used while developing GroundLens. Of the 1,616 unsupported claims that contain a number, 796 got FAIL or REVIEW (49 %, 95 % CI 47–52 %); among the claims with a number that passed, 16.3 % were unsupported, against 25.4 % before the check. Claims without a number all passed: GroundLens caught none of their errors. Median 4.0 ms per check. All results, including the sets where it does not help, in BENCHMARK.md.

Copyright Groundlens 2026

Available Tools

3 tools
verify_answerA

Check an answer against the passages it was written from, before you send it.

    Every figure must be in the passages or follow from figures that are; no model
    is used. Returns PASS, REVIEW or FAIL with the reasons, and the signed record.

    sources: (id, text) pairs, {"id","text","provenance"} dicts, or bare strings.
    policy: a built-in name (e.g. "eu_ai_act_high_risk_v1"), a path, or YAML.
    scope: what the request is about, e.g. {"entity": "UPS"}; a figure taken
    from a source whose provenance is another entity fails.
    risk_class: the request's risk class, when the policy defines them.
    Returns the decision (PASS/REVIEW/FAIL), the evidence, the regulatory
    mapping and the record with its content hash.
    
NameRequiredDescriptionDefault
scopeNo
answerYes
localeNound
policyNo
sourcesYes
questionNo
risk_classNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does real work: it discloses that 'no model is used' (deterministic, rule-based), the three-outcome decision space, that reasons/evidence/regulatory mapping are produced, and that a signed record with a content hash is emitted. It does not mention auth requirements, rate limits, or cost, which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded purpose, then behavioral summary, then parameter notes – a sensible order. The only waste is the duplicated return description ('Returns PASS, REVIEW or FAIL with the reasons, and the signed record' vs 'Returns the decision ... the evidence, the regulatory mapping and the record with its content hash').

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with an output schema, the description covers the semantically hard inputs (sources, policy, scope, risk_class) and the decision model. Because an output schema exists it need not re-explain returns, and it partially does anyway. The gaps are minor: question and locale are never described, and no sibling disambiguation is offered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it largely does: sources is given three concrete accepted shapes, policy lists built-in name/path/YAML forms, scope is explained with an example plus the cross-entity failure rule, and risk_class is tied to policy-defined classes. The question and locale parameters remain unexplained in both schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource pair (check an answer against its source passages) with the timing constraint 'before you send it'. It is clear on its own, but it never names or distinguishes itself from the siblings verify_run and verify_records, so an agent cannot route between the three from the description alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'before you send it' implies the workflow position (pre-publication gate), which is useful. However, there is no statement of when NOT to use it, no prerequisites (e.g. whether sources must be pre-chunked), and no routing to verify_run or verify_records, so the guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_recordsA

Verify a log of records offline: every hash, every link, every signature.

    records: the JSON Lines text of an answer-record or run-record log.
    Returns {"ok", "verified", "kind"}; fails if any record or link was altered.
    
NameRequiredDescriptionDefault
recordsYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and does meaningful work: it discloses the return shape ('Returns {"ok", "verified", "kind"}'), the failure mode ('fails if any record or link was altered'), and that the operation happens offline. It stops short of explicitly stating verification is non-destructive, a minor gap given 'verify' implies it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact — purpose is front-loaded in the first sentence, followed by the parameter and then the return/failure behavior. Every clause carries information an agent needs; there is no filler or restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter verification tool with an output schema present, the description covers purpose, input format, return shape, and failure behavior — nearly everything needed to call it correctly. Minor gaps like the possible values of 'kind' are left to the output schema, which is acceptable per the rubric.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does: it documents 'records' as 'the JSON Lines text of an answer-record or run-record log,' adding format and content meaning the schema lacks. It doesn't specify the exact structure of a valid record, but for a single string parameter the added semantics are substantial.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Verify a log of records offline') with concrete scope ('every hash, every link, every signature'), so an agent can tell exactly what operation this performs. It also distinguishes this from the siblings verify_run and verify_answer by clarifying that it accepts both answer-record and run-record logs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by noting the tool accepts 'an answer-record or run-record log,' which hints it covers the domains of both siblings. However, it never names verify_run or verify_answer or gives an explicit when-to-use vs. when-not-to-use rule, leaving the routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_runA

Verify an MCP execution trace under an execution policy and return the run record.

    trace: the MCP session as JSON-RPC messages (JSON Lines).
    policy: the execution policy, as YAML/JSON text or a path.
    Returns the gate (ALLOW/REVIEW/DENY), any breaches, and the signed run record.
    
NameRequiredDescriptionDefault
traceYes
policyYes
run_idYes
systemYes
started_atNo
system_versionNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the return values (gate, breaches, signed run record) but does not mention potential side effects (e.g., whether it writes or stores anything), permission requirements, or error behavior. This is some behavioral context but incomplete for a tool with no annotation safety net.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is reasonably concise, with the purpose front-loaded and parameters broken into clear lines. It avoids redundant wording and communicates the key return values efficiently, though it could be tightened slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema, so return format details are not strictly required, and the description already provides a high-level return summary. However, given the six-parameter complexity and lack of annotations, the description should explain all parameters and ideally differentiate usage from siblings. It covers the core purpose but leaves several parameters and usage guidance gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains trace (format: JSON-RPC messages as JSON Lines) and policy (format: YAML/JSON text or path), which is useful. However, it does not explain run_id, system, started_at, or system_version, leaving 4 of 6 parameters undocumented in both schema and description. This is a significant gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool verifies an MCP execution trace against an execution policy and returns the run record with gate, breaches, and signed record. This specific verb+resource distinguishes it from sibling tools verify_answer and verify_records, which target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by specifying it is for verifying execution traces, which gives clear context. However, it does not explicitly mention when not to use it or point to alternatives like verify_answer or verify_records, so it lacks explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool updatev6.0.2
    • Changedverify_answer2 fields changed
      • addedInput schema / properties / risk_class
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Risk Class"
        +}
      • addedInput schema / properties / scope
        Added value: +{
        +  "anyOf": [
        +    {
        +      "additionalProperties": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Scope"
        +}
  2. 4 tool updatesv3.0.6
    • Removedfind_unsupported_words
    • Addedverify_answer
    • Addedverify_records
    • Addedverify_run
  3. 1 tool updatev0.1.0
    • First observedfind_unsupported_words

TDQS

A3.9/5.0

Scored across 3 tools

Disambiguation4/5

Each tool takes a clearly different input shape: a JSON-RPC session trace, answer passages, and a JSON Lines record log. There is mild overlap because verify_records can validate run records that verify_run also produces/validates, which could cause a brief hesitation, but the described inputs keep them separable.

Naming Consistency5/5

All three tools follow a strict verb_noun pattern (verify_run, verify_answer, verify_records), with the same verb used consistently for the same conceptual action.

Tool Count4/5

Three tools is on the lean side but each covers a distinct verification target (run trace, answer grounding, record log), so nothing feels redundant or padded.

Completeness4/5

The core verification lifecycle (verify runs, verify answers, verify logs offline) is covered, and records are produced by the verify calls themselves. A minor gap is that no tool enumerates available built-in policies (e.g. 'eu_ai_act_high_risk_v1'), though agents can work around this by passing policy text or a path.

Maintenance

ActivityActive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers