Skip to main content
Glama
thejaredchapman

EvalForge Lite

by thejaredchapman
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

get_report

Generate a base64-encoded PDF report for a model evaluation run, with optional run ID and priority override for the recommendation line.

Instructions

Get a PDF report (base64-encoded) for a run. Defaults to the most recent run.

priority (balanced|quality|fastest|cheapest), if given, overrides the priority the
run was made with for the report's priority/best-pick line; otherwise the run's own
priority (from run_comparison) is used.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
run_idNo
priorityNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.1

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It discloses the return format (base64-encoded PDF) and the latest-run default, but says nothing about permission requirements, behavior on in-progress or missing runs, cost/latency of report generation, or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core action and output format in the first sentence, then explains the priority parameter. Slightly awkward line breaking and a redundant parenthetical reference to run_comparison, but no filler sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, no-output-schema tool, the description covers what is returned, the optional nature of both parameters, and the priority override rule. The remaining gaps (error behavior, run_id format) are modest given the tool's low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does for the more complex parameter: it supplies the enum values for priority (balanced|quality|fastest|cheapest) and explains the override semantics relative to the run's own priority. run_id receives only the implicit "defaults to most recent run" behavior, leaving its format unspecified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (get a PDF report for a run), gives the output encoding (base64), and names the default scope (most recent run). Together with the sibling get_report_csv, which it implicitly contrasts by format, an agent can identify this tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Defaults to the most recent run" implies when the run_id can be omitted, which is useful usage context. However, it never names the alternative (get_report_csv) or states when to prefer PDF over CSV, and gives no preconditions such as requiring a completed run.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools