Skip to main content
Glama
Sign In

Runcrate

by io.github.aeonmindai
OverviewDiscussionsAdmin

Rate a model answer

rate_request
DestructiveIdempotent

Score one answer of a model called through the Runcrate inference API, by its request id (the X-Request-Id response header, or the completion id chatcmpl-). Send exactly one of thumbs ("up" or "down"), rating (1 to 5) or score (-1 to 1), with an optional label such as "helpful" or "wrong_format" and a comment. The rating shows with the request in the dashboard (Models > Observability) and can be filtered on there. Rating the same request again replaces this agent's rating. The request must be from the environment this session acts in and from the last 30 days.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "request_id": "<request_id>"
    },
    "name": "rate_request"
  }
}

Input Schema

NameRequiredDescriptionDefault
labelNoA short tag, e.g. "helpful" or "wrong_format"
scoreNoA score from -1 (bad) to 1 (good)
ratingNoA rating from 1 (bad) to 5 (good)
thumbsNoA good (up) or bad (down) answer
commentNoWhat was right or wrong with the answer
workspaceNoWorkspace ID (optional: defaults to the authenticated workspace)
request_idYesThe X-Request-Id of the call, or its completion id (chatcmpl-...)
environmentNoEnvironment name (optional: defaults to the environment this session acts in)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint=true and idempotentHint=true; the description explains both by stating that re-rating the same request replaces this agent's existing rating, which clarifies the upsert semantics rather than contradicting them. It also discloses where the result surfaces (Models > Observability dashboard) and the 30-day/environment scoping limits, though it says nothing about what error or partial-update behavior to expect if constraints are violated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded: the action and identifier come first, then the value options, then side effects and constraints. The clause about dashboard filtering is slightly peripheral to invocation, but most sentences carry load-bearing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 optional-heavy parameters, full schema coverage, annotations and no output schema, the description supplies the missing behavioral picture: exclusivity of value fields, replacement semantics, and eligibility limits. Minor gaps remain around error handling when zero or multiple value fields are sent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds a real constraint not expressible in the schema: thumbs, rating and score are mutually exclusive and exactly one must be supplied. It also restates the request_id formats (X-Request-Id header vs. chatcmpl- id) which the schema already covers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Score one answer of a model called through the Runcrate inference API") plus the identifier used to address it, which cleanly separates it from read-only siblings like get_request and list_requests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete preconditions for use: the request must belong to this session's environment and be from the last 30 days, and exactly one of thumbs/rating/score must be sent. It does not explicitly name sibling tools to use instead (e.g. get_request to verify the id), but the context for calling is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Runcrate toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources