Skip to main content
Glama

Bankstatemently

Evaluate Benchmark

evaluate_benchmark
Read-only

Score parsed bank statement transactions against the Bankstatemently benchmark ground truth. Accepts a statement_id (e.g. "bsb-001") or content_hash, plus your parsed transactions. Returns extraction accuracy, integrity score, and an overall score. Only statements marked published: true in the catalog can be evaluated — held-out statements return an error. transactions[].originalData is optional but strongly recommended: fetch it via get_statement with data_mode: "original" and pass it through verbatim — an absent originalData scores that transaction's raw-fidelity (parsed) dimension 0; never fabricate a value. Free to use — no credits consumed. Read the benchmark://catalog resource first to see available statements and their published status.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
accountsNo
content_hashNo
statement_idNo
transactionsYes

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
idYes
challengesYes
difficultyYes
parsedScoreYes
normalizedScoreYes

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint: true, destructiveHint: false, and openWorldHint: false, which already signal a safe read operation. The description adds significant behavioral context beyond this: it states the tool is free to use (no credits consumed), that only published statements can be evaluated, that held-out statements error, and that missing originalData scores dimension 0 while fabricating a value is prohibited. These details enrich the safety and operational profile without contradicting any annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense paragraph covering purpose, constraints, parameter guidance, and usage tips. All sentences are informative and non-redundant, and the most critical information (purpose and required inputs) is front-loaded. It is somewhat long but not wasteful; each clause adds value. A slightly clearer separation of concerns might improve scannability, but it remains concise enough.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (benchmark evaluation, multiple inputs, an output schema), the description is thorough: it specifies what returns are expected, the published-only constraint, behavior on held-out statements, how to handle originalData, and resource reading guidance. It also covers edge cases like missing originalData. The presence of an output schema reduces the need to explain return structure, and the description complements it well, making the tool fully comprehensible.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for parameter meaning. It explains the role of statement_id and content_hash (though content_hash is only mentioned), examples the statement_id format, and thoroughly explains the originalData property within transactions, including how to obtain it (via get_statement) and consequences of omission. It does not detail accounts or other parameters, but the core parameters are well-covered, making effective use of the description to add meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: scoring parsed bank statement transactions against a benchmark ground truth. It specifies the inputs (statement_id or content_hash, plus transactions) and outputs (extraction accuracy, integrity score, overall score), and distinguishes it from general scoring by referencing the specific benchmark. This is a specific verb-resource-action statement that is not redundant with the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context: it states that only statements with published: true can be evaluated, held-out statements return an error, and it instructs to read the benchmark://catalog resource first. It also gives guidance on originalData being optional but recommended. However, it does not explicitly mention when not to use this tool versus alternatives like rate_statement, so it lacks explicit exclusion/alternative comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4/5.0
Disambiguation4/5

Core tools are cleanly separated by resource: statements, transactions, transfers, credits, and benchmark all have dedicated entry points. The five analytics tools (aggregate, group_by, compare, time_series, top_n) share metric/filter language, but their distinct output shapes and careful descriptions prevent major confusion, with only group_by vs. time_series/top_n occasionally overlapping.

Naming Consistency4/5

Most tools follow a clear snake_case verb_noun pattern such as convert_statement, list_transactions, rate_statement, and dismiss_statement. The analytics tools (aggregate, compare, group_by, time_series, top_n) break that pattern, but they are still recognizable, consistently styled, and form a meaningful cluster.

Tool Count4/5

At 16 tools, this is slightly above the typical 3-15 well-scoped range, but the server covers a broad workflow: upload, conversion, retrieval, categorization, analytics, transfer matching, rating, credits, and benchmarking. Each tool maps to a distinct capability, so the count feels justified rather than bloated.

Completeness4/5

The tool surface covers the full statement lifecycle from upload and conversion through retrieval, categorization, analysis, rating, and dismissal, plus useful side capabilities like credits and benchmark evaluation. Minor gaps exist—no permanent deletion and no way to manually edit category mappings—but dismiss_statement and categorize_statement provide adequate workarounds.