groundcheck

by offquestxo

Overview Schema Related Servers Score Discussions

Python

Local

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.1.0
Disambiguation4/5
The tools are mostly distinct, covering comparison, hallucination detection, faithfulness evaluation, retrieval evaluation, batch runs, and report retrieval. However, there is overlap between groundcheck_detect_hallucinations and groundcheck_evaluate_faithfulness as both deal with unsupported claims, though descriptions differentiate by use case.
Naming Consistency5/5
All tools follow a consistent pattern: 'groundcheck_' prefix followed by an imperative verb and noun (e.g., compare, detect_hallucinations, evaluate_faithfulness, get_report). Snake_case and verb-noun structure are uniform across the set.
Tool Count5/5
Six tools is well-scoped for a server focused on RAG evaluation. Each tool has a clear role without redundancy, and the count is appropriate for the domain.
Completeness4/5
The tools cover the main evaluation needs: comparing answers, detecting hallucinations, scoring faithfulness, evaluating retrieval, batch execution, and report retrieval. Minor gaps exist, such as no dedicated tool for answer relevance beyond the compare function, but the surface is largely complete.
Average 4.6/5 across 6 of 6 tools scored.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 10 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark as readOnly. Description adds cost info ('~2 model calls per case'), report persistence path, and how to retrieve the full report. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with first sentence stating purpose, then usage note, then parameter descriptions. Slightly verbose with the Args block but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, behavior, cost, and return structure. References related tool for full report. Lacks detail on output schema but given it's provided separately, it's sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description fully explains each parameter: cases as inline list format, dataset_path with directory restriction, k_values with defaults. Adds critical meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs faithfulness (+ retrieval) over a batch of cases, distinguishing it from siblings like groundcheck_evaluate_faithfulness which likely handles single cases. Explicitly says 'evaluate a whole RAG pipeline run rather than one answer at a time.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this to evaluate a whole RAG pipeline run' and 'Supply exactly one of cases or dataset_path.' Doesn't explicitly mention when not to use or alternatives, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond annotations by explaining the tie mechanism (double call to mitigate position bias), the return structure (winner, verdict per criterion, rationale), and resource usage (costs 2 model calls, no API key needed if sampling supported). Annotations already indicate read-only behavior, so no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a concise one-line summary followed by usage context and an Args list. Every sentence adds value, and it is appropriately sized without redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers most aspects: purpose, usage, parameter semantics, behavioral details, and return. However, it could mention potential error cases or edge scenarios (e.g., empty answers). Given the presence of an output schema, return values are adequately described. The tool's role among siblings is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description provides clear explanations for all parameters in the Args section: query, answer_a, answer_b, sources, and criteria (with default). This compensates fully for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool judges which of two candidate answers is better, using specific verbs ('judge', 'compare') and a defined resource (candidate answers to query). It distinguishes from sibling tools like groundcheck_detect_hallucinations by focusing on A/B comparison rather than detection of single issues.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly describes when to use this tool: 'Use this to A/B two RAG configurations (prompts, retrievers, models) on the same query.' It also explains position bias mitigation. However, it does not explicitly state when not to use it or mention alternatives, though sibling tools are available for other tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as read-only and non-destructive. The description adds valuable context: Mode A is instant with no model calls, Mode B uses one sampling call without requiring an API key. This helps the agent understand performance and dependency implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is approximately 150 words, well-organized into purpose, modes, and argument list. Every sentence serves a distinct purpose—no redundancy or fluff. Front-loading the core action and mode selection makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (two modes, multiple parameters, output schema exists), the description covers behavioral aspects, parameter semantics, and output indication ('states which mode ran'). It does not detail the output metrics, but that is delegated to the output schema. Minor gap: it could explicitly mention the metric names (precision, recall, MRR, NDCG) are computed, but they are listed earlier in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite low schema description coverage (0%), the description fully explains all four parameters (query, retrieved, relevant_ids, k_values) with their roles and constraints. It clarifies the structure of 'retrieved' as a list of {id, text} objects and emphasizes that rank matters. This compensates entirely for the sparse schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear action ('Score retrieval quality') and specifies the resources ('retrieved' chunks, 'query'). It distinguishes itself from sibling tools (e.g., detect_hallucinations, evaluate_faithfulness) by focusing solely on retrieval quality. The two-mode breakdown further clarifies the tool's scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly guides when to use each mode: Mode A when relevant_ids are available ('free and exact'), Mode B when not. It does not directly contrast with sibling tools, but the purpose is distinct enough that an agent can infer usage. The guidance is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=false, destructiveHint=false. The description adds valuable context: costs 2 model calls via sampling, no API key needed if client supports sampling, and explains response format behavior (concise vs detailed). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose sentence, usage guidance, then args list. It is slightly long but every part adds value. Front-loading the key purpose and sibling differentiation is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, an output schema is referenced (not shown), and description covers return structure (score, counts, claims) and cost. It lacks details like max sources limit or async behavior, but overall is quite complete for a metadata evaluation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description carries full burden. It explains 'answer' as RAG-generated answer text, 'sources' as list of {id, text} chunks, and 'response_format' with values 'concise' (default) and 'detailed' plus their return behavior. This adds significant meaning beyond bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The tool description explicitly states it scores faithfulness of answer vs sources claim-by-claim. It clearly distinguishes from the sibling tool groundcheck_detect_hallucinations by noting it shows every claim's verdict instead of just problems.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use (need overall faithfulness score with all verdicts) and when-not-to-use (use groundcheck_detect_hallucinations for tighter/cheaper problem-only detection). This is excellent guidance with named alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable context: it will raise an error listing available report IDs if an unknown ID is provided, and it clarifies that it reads from a local store without making API calls. This goes beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is composed of four concise sentences, front-loaded with the main purpose. Every sentence adds valuable information with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (fetching a report by ID), the description covers all necessary aspects: purpose, parameter usage, error behavior, and performance characteristics (no model calls). The presence of an output schema further complements the documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description fully explains both parameters. For `report_id`, it specifies it is the id returned by `groundcheck_run_suite`. For `response_format`, it details the two enum values ('concise' for summary, 'detailed' for full results). Since the schema description coverage is 0%, the description carries the full burden and does so excellently.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Fetch a previously persisted evaluation report by id.' It clearly identifies the verb (fetch) and the resource (evaluation report). The distinction from sibling tools like `groundcheck_run_suite` (which runs suites) and `groundcheck_evaluate_*` (which perform evaluations) is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage after running a suite by referencing `groundcheck_run_suite` for the report_id origin. It also mentions that no model calls are made, which suggests it is safe to use without cost. However, it does not explicitly state when to use this tool over alternatives or provide exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=true and destructiveHint=false, but the description goes beyond by explaining return behavior (empty list if clean), entry structure, and cost (2 model calls, no API key needed if client supports sampling). No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: brief purpose, usage distinction, parameter explanations, and return value description. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, the description sufficiently explains the return format (empty list or entries with span, closest source, reason) and usage context. Complete for the intended purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage, but the description's Args section provides clear definitions for both parameters: answer as 'RAG-generated answer text' and sources as 'list of {id, text} chunks'. This compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool finds unsupported or contradicted claims in an answer. It distinguishes itself from the sibling tool groundcheck_evaluate_faithfulness by specifying this tool is for a fix-it loop focusing only on wrong claims.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool ('fix-it loop where you only care about what's wrong') and when to use the alternative ('for the full picture with a score and every claim's verdict, use groundcheck_evaluate_faithfulness').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![groundcheck MCP server](https://glama.ai/mcp/servers/offquestxo/groundcheck/badges/card.svg)](https://glama.ai/mcp/servers/offquestxo/groundcheck)

Score Badge

Copy to your README.md:

[![groundcheck MCP server](https://glama.ai/mcp/servers/offquestxo/groundcheck/badges/score.svg)](https://glama.ai/mcp/servers/offquestxo/groundcheck)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/offquestxo/groundcheck'

If you have feedback or need assistance with the MCP directory API, please join our Discord server