Skip to main content
Glama
phillipkaraya

rageval-mcp

by phillipkaraya
OverviewSchemaRelated ServersScoreDiscussions
Python
Local

Server Quality Checklist

67%
Profile completionA complete profile improves this server's visibility in search results.

  • Latest release: v0.1.0

  • Disambiguation5/5

    Each tool has a clear and distinct purpose: retrieve is for fetching passages, evaluate_retrieval scores a single method, and compare_methods benchmarks all methods side by side. No overlap in functionality.

    Naming Consistency4/5

    All tool names use snake_case. 'compare_methods' and 'evaluate_retrieval' follow a verb_noun pattern, but 'retrieve' is a single verb. The slight inconsistency is minor and does not hinder readability.

    Tool Count3/5

    Three tools is borderline small for a retrieval evaluation server. It covers basic operations (retrieve, evaluate, compare) but feels thin; additional tools for managing data or methods would improve scope. The count is acceptable for a focused demo.

    Completeness3/5

    The tools cover core retrieval and evaluation workflows but lack functionality for managing the question set or corpus. Users cannot add custom data, which limits the server's utility beyond the bundled sample.

  • Average 4.3/5 across 3 of 3 tools scored.

    See the Tool Scores section below for per-tool breakdowns.

    • No community issues in the last 6 months
    • 3 commits in the last 12 weeks
    • No stable releases found
    • No critical vulnerability alerts
    • No high-severity vulnerability alerts
    • No code scanning findings
    • CI status not available

  • This repository is licensed under MIT License.

  • This repository includes a README.md file.

  • No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.

    Tip: use the "Try in Browser" feature on the server page to seed initial usage.

  • Add a glama.json file to provide metadata about your server.

  • If you are the author, simply .

    If the server belongs to an organization, first add glama.json to the root of your repository:

    {
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}

    Then . Browse examples.

  • Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

  • Behavior4/5

    Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

    Annotations already declare readOnlyHint, destructiveHint, and idempotentHint. The description adds behavioral context by noting that methods with missing dependencies are reported under 'skipped' with a reason rather than failing, which goes beyond what annotations provide.

    Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

    Conciseness4/5

    Is the description appropriately sized, front-loaded, and free of redundancy?

    The description is well-structured with Args and Returns sections, and is mostly concise. However, the first sentence includes somewhat marketing-like phrasing that could be trimmed without loss of clarity.

    Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

    Completeness5/5

    Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

    Given the presence of a comprehensive output schema and annotations, the description fully explains input, behavior, and output. It covers how skipped methods are handled and lists all return fields, making it complete for an agent to use effectively.

    Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

    Parameters3/5

    Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

    Schema description coverage is 100% and the only parameter 'k' is fully documented in the schema. The description repeats the schema's description and default but adds no new semantic meaning, so baseline score of 3 is appropriate.

    Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

    Purpose5/5

    Does the description clearly state what the tool does and how it differs from similar tools?

    The description clearly states the tool benchmarks every available retrieval method side by side at cutoff k, using a specific verb and resource. It distinguishes from siblings like evaluate_retrieval and retrieve by focusing on comparative benchmarking.

    Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

    Usage Guidelines4/5

    Does the description explain when to use this tool, when not to, or what alternatives exist?

    The description explicitly frames the tool as 'the honest, defensible answer to which retrieval strategy should we use', indicating when to use it. It also handles missing dependencies gracefully, but does not explicitly mention when not to use it versus alternatives.

    Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

  • Behavior4/5

    Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

    Annotations already indicate readOnlyHint=true and idempotentHint=true. The description adds that it raises a tool error if the dense extra is missing, which is a behavioral disclosure beyond annotations.

    Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

    Conciseness5/5

    Is the description appropriately sized, front-loaded, and free of redundancy?

    The description is concise with no unnecessary words, front-loaded with the main purpose, and includes Args and Returns sections for clarity.

    Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

    Completeness5/5

    Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

    The tool has only 2 parameters, complete schema coverage, and the description explains the return fields, error conditions, and metrics. It is fully complete for its complexity.

    Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

    Parameters3/5

    Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

    Schema coverage is 100% and the description repeats the parameter defaults and ranges without adding new semantics. Baseline 3 is appropriate because the schema already provides full details.

    Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

    Purpose5/5

    Does the description clearly state what the tool does and how it differs from similar tools?

    The description starts with 'Score one retrieval method against the labeled question set,' which uses a specific verb and resource, clearly distinguishing it from siblings 'compare_methods' and 'retrieve'.

    Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

    Usage Guidelines4/5

    Does the description explain when to use this tool, when not to, or what alternatives exist?

    The description explains that the tool evaluates a single method and provides default parameters, and it warns about a potential error when method='dense' without the optional extra. However, it does not explicitly contrast with siblings or state when not to use.

    Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

  • Behavior4/5

    Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

    Annotations declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds beyond annotations by explaining the bundled corpus (no setup) and error behavior for missing dense extra. This provides useful behavioral context not available from annotations alone.

    Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

    Conciseness5/5

    Is the description appropriately sized, front-loaded, and free of redundancy?

    The description is concise with clear sections (overview, Args, Returns, Raises). Every sentence adds value; no fluff. It is well-structured and front-loaded with the core purpose.

    Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

    Completeness5/5

    Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

    Given the tool's complexity (3 parameters, output schema implied by Returns section), the description is complete. It covers return fields, error cases, and usage context. Annotations are rich, and the description provides adequate supplementary information.

    Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

    Parameters3/5

    Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

    Schema description coverage is 100%, so baseline is 3. The description's 'Args' section restates schema info concisely but adds no new meaning beyond what the schema already provides. The error message hint is helpful but not parameter-specific.

    Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

    Purpose5/5

    Does the description clearly state what the tool does and how it differs from similar tools?

    The description clearly states the action ('Retrieve the top-k passages for a query') and the resource ('bundled sample corpus'). It distinguishes from sibling tools (compare_methods, evaluate_retrieval) by specifying that it is for simple retrieval to see context surfaced by a RAG system.

    Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

    Usage Guidelines4/5

    Does the description explain when to use this tool, when not to, or what alternatives exist?

    The description explicitly says 'Use this to see what context a RAG system would surface' and notes that no setup is required. It implies when to use but does not explicitly state alternatives or when not to use. The context signal includes sibling tools, but no direct comparison is provided.

    Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

  • Confirm that the MCP server is working as expected.
  • Confirm that there are no obvious security issues.
  • Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

rageval-mcp MCP server

Copy to your README.md:

Score Badge

rageval-mcp MCP server

Copy to your README.md:

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/phillipkaraya/rageval-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server