Skip to main content
Glama
Sign In

agentcheck

by io.github.agentwares
OverviewDiscussionsAdmin

agentcheck_add_check

Add a check to one of your targets. A check runs an input (http path/prompt, mcp tool call, a2a message) on a schedule and judges the answer with a golden: exact, contains, regex and json_schema cost nothing; rubric and baseline use the LLM judge (baseline = same outcome as the last known-good answer). Returns the check id. Requires an API key.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>",
      "kind": "<kind>",
      "golden": "<golden>",
      "targetId": "<targetId>"
    },
    "name": "agentcheck_add_check"
  }
}

Input Schema

NameRequiredDescriptionDefault
kindYeswhat to run: http (GET path or POST prompt), mcp_tools_list, mcp_tool_call (tool + args), a2a_task (message)
nameYesshort name shown on the status page
inputNohttp: { path?, method?, body?, prompt? } · mcp_tool_call: { tool, args } · a2a_task: { message }
goldenYeshow the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer.
targetIdYesid from agentcheck_create_target or GET /api/v1/targets
intervalSecNoseconds between runs; the tier's interval is the floor (Free hourly, Starter+ 5 min)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedInput schema / properties / golden / description
      Added value: +"how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer."
  2. First observed

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is a non-read-only, non-destructive, non-idempotent, closed-world mutation. The description adds real context beyond them: the API-key auth requirement, the returned check id, and the cost split between free deterministic judges and LLM-judge goldens. It stops short of stating duplicate-creation behavior or rate limits, which matter for a non-idempotent create.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and the concept definition, then the return value and auth note. The middle clause listing judge kinds and costs is dense and slightly run-on, but every sentence carries information and nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter mutation with a rich nested schema, full annotation coverage and no output schema, the description supplies the missing pieces an agent needs: what a check is, that a target must pre-exist, the auth requirement, the return value, and the cost implications of golden choice. Nothing essential is left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents kind, input shapes, golden fields, targetId provenance and intervalSec tier floors. The description largely restates the golden cost/judge distinction that the schema itself carries, adding no syntax or format detail beyond it. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Add a check to one of your targets') and then defines the domain concept a check embodies: an input run on a schedule judged by a golden. It also names the sibling that produces the required targetId (agentcheck_create_target), so an agent can distinguish this from create/run/promote siblings without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the prerequisite 'one of your targets' and the explicit API-key requirement, and the golden taxonomy gives selection guidance among judge kinds. However, there is no explicit when-to-use/when-not, and no routing against the closest sibling agentcheck_run_now (immediate execution) or agentcheck_promote_trace.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All agentcheck toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources