agentcheck
Server Details
Synthetic checks, nightly regression replay and model-drift alerts for AI agents
- Status
- Healthy
- Uptime
- 100.0% over 21 days
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- agentwares/servers
- GitHub Stars
- 0
TDQS
Scored across 8 tools
Tools separate target enrollment, check creation, trace recording/promotion, manual runs, incident listing, and public status/pricing. There is minor overlap between add_check and promote_trace (both create checks) and create_target's inline checks, but descriptions make the intended path clear.
All tools use the agentcheck_ prefix and snake_case, with verb_noun names except record which is a bare verb. Consistency is high, with only a minor deviation.
8 tools is well-scoped for a monitoring service: enrollment, checks, traces, runs, incidents, status, pricing. No obvious bloat or thinness.
Core create/add/run/status workflows exist, but there is no tool to list, update, or delete targets and checks. Agents must remember IDs from creation responses and cannot manage or discover existing resources, a notable lifecycle gap.
Available Tools
8 toolsagentcheck_add_checkAInspect
Add a check to one of your targets. A check runs an input (http path/prompt, mcp tool call, a2a message) on a schedule and judges the answer with a golden: exact, contains, regex and json_schema cost nothing; rubric and baseline use the LLM judge (baseline = same outcome as the last known-good answer). Returns the check id. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | what to run: http (GET path or POST prompt), mcp_tools_list, mcp_tool_call (tool + args), a2a_task (message) | |
| name | Yes | short name shown on the status page | |
| input | No | http: { path?, method?, body?, prompt? } · mcp_tool_call: { tool, args } · a2a_task: { message } | |
| golden | Yes | how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer. | |
| targetId | Yes | id from agentcheck_create_target or GET /api/v1/targets | |
| intervalSec | No | seconds between runs; the tier's interval is the floor (Free hourly, Starter+ 5 min) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare the write/idempotency profile; the description adds genuinely useful context beyond them: it returns the check id, requires an API key, and discloses the cost model (exact/contains/regex/json_schema are free, rubric/baseline invoke the LLM judge). It does not discuss scheduling floors or error behavior, which the schema partly covers.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences, front-loaded with purpose and outcome before the cost/auth notes. No wasted phrasing, though the cost clause is somewhat redundant with the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a nested-object mutation tool with no output schema, the description covers the essential missing pieces: the return value (check id) and the auth prerequisite. Scheduling cadence and tier floors are left to the schema, but nothing critical for correct invocation is absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all six parameters, including the golden kinds and cost. The description restates the golden cost distinction and the meaning of baseline, adding marginal value beyond the schema's own nested descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ('Add a check to one of your targets') plus a compact definition of what a check is ('runs an input... on a schedule and judges the answer with a golden'). This clearly distinguishes it from siblings like create_target, run_now, and record.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the workflow (you need a target first, per the targetId reference) and notes the API key requirement, but it never states when to prefer this over run_now or record, nor any exclusions or prerequisites beyond the auth note.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_create_targetAInspect
Enroll something to monitor: an http endpoint (JSON or OpenAI-style chat), a remote MCP server (Streamable HTTP url) or an A2A agent (origin with /.well-known/agent-card.json). Pass checks to create checks in the same call (POST /api/v1/probe proposes three). Returns the target id, the public status page, the badge SVG URL and a README snippet. The first check runs on the next minute tick; call agentcheck_run_now to run immediately. Free tier: 1 target, hourly; Starter+: 5-minute checks. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | endpoint URL (http/mcp) or the agent's origin (a2a) | |
| auth | No | credential the prober sends to your endpoint: {type:'bearer',token} or {type:'header',name,value}. Stored encrypted and never returned or logged. | |
| kind | Yes | http (JSON or chat endpoint), mcp (Streamable HTTP url, or npx/uvx package spec), a2a (agent card) | |
| name | No | display name; defaults to the host | |
| slug | No | URL slug for /<owner>/<slug>; defaults to a slug of the name | |
| alerts | No | where to send an incident when a check starts failing. Any combination; omit it and incidents are visible only on the status page. | |
| checks | No | checks to create right away (the probe proposes three) | |
| format | No | http: openai_chat (POST /chat/completions body), json (raw POST), get (plain fetch) | |
| headers | No | extra request headers | |
| isPublic | No | public status page + badge (default true) | |
| modelVar | No | the model your agent runs on (e.g. claude-sonnet-5); enables model-drift re-runs | |
| corpusUrl | No | RAG targets: public corpus URL for the nightly groundedness scorer (Pro) | |
| modelHeader | No | request header your endpoint accepts to override the model (drift re-runs try the new model) | |
| packageSpec | No | mcp only: `npx @org/server` / `uvx server` — runs on the mcpcheck runner, not the minute checks | |
| agentCardUrl | No | a2a: explicit agent card URL when it is not at /.well-known/agent-card.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare this is a non-idempotent write in an open world, but the description adds substantial behavior beyond that: the exact response contents (target id, public status page, badge SVG, README snippet), the next-minute-tick scheduling semantics, the free vs Starter+ interval floors, and "Requires an API key". Credential handling ("Stored encrypted and never returned or logged") is also surfaced in the schema, reinforcing the safety profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action, then orders sentences by task flow: what to enroll, the checks shortcut, what comes back, when it runs, tier limits, auth requirement. Each sentence carries distinct information with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by enumerating the return values (target id, status page, badge URL, README snippet). Combined with the auth requirement, scheduling tick, tier limits, and the explicit alternative for immediate runs, an agent has everything needed to invoke and interpret this 15-parameter, nested-object tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description adds real semantic value beyond the fields: it clarifies that `checks` creates checks in the same call and that the probe endpoint proposes three, and it explains the interval-floor rule ("Free hourly, Starter+ 5 min") that governs `intervalSec`. It does not, however, touch the many other parameters (auth, alerts, modelVar, corpusUrl), which is acceptable given full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ("Enroll something to monitor") and immediately enumerates the three monitorable kinds (http endpoint, remote MCP server, A2A agent), which matches the `kind` enum in the schema. An agent can distinguish this from sibling tools like agentcheck_add_check or agentcheck_run_now without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States when to use the same-call path ("Pass `checks` to create checks in the same call") and names the alternative for immediate execution ("call agentcheck_run_now to run immediately"). It also notes tier-based scheduling limits, but does not explicitly route to agentcheck_add_check for adding checks to an existing target, leaving one gap in the when/when-not picture.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_get_pricingARead-onlyIdempotentInspect
Machine-readable pricing for agentcheck: tiers with monthly USD price, target limits, check interval and features, plus per-run add-ons. Same data as /pricing.json. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description adds useful behavioral context by stating no API key is required and that the data is machine-readable, which helps an agent decide whether it can call the tool without auth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, each earning its place: core purpose, data equivalence to /pricing.json, and auth requirement. It is front-loaded and contains no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only pricing endpoint with no output schema, the description provides everything needed: what data is included, where it matches from, and that no key is needed. Nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema coverage is 100%, so there is nothing for the description to explain about inputs. The baseline for a no-parameter tool is 4, and the description adds no unnecessary parameter-related content.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource (agentcheck pricing) and its contents (tiers, monthly USD price, limits, interval, features, add-ons), and it is distinct from all sibling tools. It uses a noun phrase rather than an explicit verb like 'retrieves', but the meaning is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context: this is the machine-readable pricing data, equivalent to /pricing.json, and no API key is needed. It does not explicitly discuss when not to use it or compare it to sibling tools, but for a zero-parameter read-only pricing lookup this is sufficient guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_get_statusARead-onlyIdempotentInspect
Current status of a public monitored target: overall state, uptime over 24h/7d/30d, last check time, last nightly scores, open incidents and the badge/status URLs. Use the owner (GitHub login) and target slug from the status page URL https://agentwares-agentcheck.vercel.app//. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
| slug | Yes | target slug, e.g. demo | |
| owner | Yes | GitHub login of the target's owner, e.g. demo |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description aligns with the readOnlyHint and idempotentHint annotations by presenting a read-only status lookup. It adds useful behavioral context such as 'public monitored target' and 'No API key needed', though it does not explicitly state side-effect-free behavior; annotations already cover this.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant wording. The description front-loads the tool's purpose, then lists the included data points, and finishes with the URL-derived parameter guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple two-parameter input and no output schema, the description sufficiently explains both how to call the tool and what will be returned. It covers the key output categories and necessary context for selecting this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and both owner and slug parameters are explained. The description adds the practical hint to take these values from the status page URL, but does not materially expand beyond the schema's existing parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('current status') and resource ('public monitored target'), and enumerates the exact data returned (overall state, uptime, last check, nightly scores, open incidents, badge/status URLs). It is clearly distinguishable from sibling tools like list_incidents or run_now.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context on when to use the tool: to retrieve status for a public monitored target using owner and slug from a status page URL, and notes no API key is needed. It does not explicitly contrast with sibling tools, but the scope is clear enough for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_list_incidentsARead-onlyIdempotentInspect
Open and recent incidents across your targets (or one target): when they opened/closed, the failing check and the cause. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| targetId | No | narrow to one target; omit for incidents across every target on the account |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and non-open-world, so the safety profile is covered. The description adds genuinely new behavioral context by stating 'Requires an API key', an auth prerequisite not present in the structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence, front-loaded with the resource and scope, followed by the payload description and the auth requirement. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read tool with no output schema, the description covers scope, returned fields, and the auth requirement, which is close to sufficient. It stops short of routing guidance among the sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With a single parameter at 100% schema description coverage, the baseline is 3. The phrase 'or one target' loosely echoes the targetId semantics ('narrow to one target; omit for incidents across every target') but adds no meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource ('list incidents') with useful elaboration on what an incident record contains (open/close times, failing check, cause). It is distinguishable from siblings like agentcheck_get_status or agentcheck_record by resource, though it never explicitly contrasts itself with them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical '(or one target)' implies the two usage modes (account-wide vs. narrowed), but there is no explicit when-to-use guidance versus siblings such as get_status, and no stated exclusions. The API-key prerequisite is the only operational cue.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_promote_traceAIdempotentInspect
Turn an imported or recorded trace into a replayable check whose golden is the recorded outcome (tool sequence + final-answer rubric). The check runs daily and in the nightly replay (Pro). Returns the check. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | trace id from agentcheck_record or POST /api/v1/targets/{id}/traces |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses important behaviors: creates a replayable check, runs daily and in nightly replay (Pro), returns the check, and requires an API key. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant wording, front-loading the core purpose and including essential constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, no-output-schema tool, the description provides enough context about behavior, scheduling, return value, and API key requirement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema fully covers the traceId parameter, and the description adds context by explaining the trace must be imported or recorded.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb 'Turn ... into a replayable check' identifies the action and resource, and it is distinct from sibling tools like agentcheck_record and agentcheck_run_now.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States the input is an imported or recorded trace, implying when to use it, but does not explicitly contrast it with sibling tools like agentcheck_add_check.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_recordAInspect
Record one production interaction with your agent (the prompt or messages, the final answer, the tools it called, optionally the model) as a trace on a target. Call it from your agent or from a proxy in front of it after each task; promote a good trace with agentcheck_promote_trace to replay it nightly and catch regressions. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | short label, e.g. 'refund for order A-1029' | |
| input | Yes | the prompt, the messages array, or an object with prompt/messages | |
| model | No | the model that produced this answer, so drift re-runs can compare across models | |
| output | Yes | the agent's final answer | |
| targetId | Yes | id from agentcheck_create_target or GET /api/v1/targets | |
| toolCalls | No | tool calls in order |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false and idempotentHint=false, so the write/non-idempotent profile is known. The description adds context annotations don't carry: it requires an API key and is meant to be invoked per-task from an agent or proxy. It stops short of describing duplicate-handling or limits, so not a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core action, then integration point, then the promote-to-replay payoff. No redundant restatement of the tool name; the API-key prerequisite is a compact trailing note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers the action, invocation context, auth prerequisite, and the follow-up workflow. Safety is largely carried by annotations; only idempotency/duplicate behavior is left implicit, keeping it just short of 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so every parameter including targetId, input, model, and toolCalls is already documented in the schema. The description paraphrases the field set but adds no syntax or format detail beyond it, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Record) and resource (one production interaction / trace) and enumerates the payload (prompt/messages, final answer, tool calls, model). It also names the downstream sibling agentcheck_promote_trace, letting an agent differentiate it from siblings without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to call it ('from your agent or from a proxy in front of it after each task') and routes to the follow-on action (promote a good trace with agentcheck_promote_trace to replay nightly). The usage condition and next step are fully spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_run_nowAInspect
Run every check of one of your targets immediately (outside the schedule) and return pass/fail per check with latency, judge cost and any incident opened or closed. Use it right after enrolling, or to confirm a fix. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| targetId | Yes | target id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly=false. Description adds that incidents may be opened or closed, making the side effects concrete. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences. The main action and result are front-loaded, followed by use-case guidance. No unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Describes both the action and the expected return values (pass/fail, latency, judge cost, incidents). Enough for an agent to understand what the tool does and what it produces.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter targetId is described only as 'target id' in the schema, and the tool description adds little beyond that. It is self-explanatory, but no extra meaning is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool runs every check for a target immediately and lists the returned data. Distinct from siblings like add_check or get_status by focusing on executing all checks now.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance: right after enrolling or to confirm a fix. Does not explicitly contrast with siblings, but the intended scenarios are clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
- Changed
agentcheck_add_check1 field changed- added
Input schema / properties / golden / descriptionAdded value: +"how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer."
- Changed
agentcheck_create_target3 fields changed- added
Input schema / properties / alerts / descriptionAdded value: +"where to send an incident when a check starts failing. Any combination; omit it and incidents are visible only on the status page." - added
Input schema / properties / auth / descriptionAdded value: +"credential the prober sends to your endpoint: {type:'bearer',token} or {type:'header',name,value}. Stored encrypted and never returned or logged." - added
Input schema / properties / checks / items / properties / golden / descriptionAdded value: +"how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer."
- Changed
agentcheck_list_incidents1 field changed- added
Input schema / properties / targetId / descriptionAdded value: +"narrow to one target; omit for incidents across every target on the account"
- Changed
agentcheck_record2 fields changed- added
Input schema / properties / model / descriptionAdded value: +"the model that produced this answer, so drift re-runs can compare across models" - added
Input schema / properties / targetId / descriptionAdded value: +"id from agentcheck_create_target or GET /api/v1/targets"
8 tool updates
- First observed
agentcheck_add_check - First observed
agentcheck_create_target - First observed
agentcheck_get_pricing - First observed
agentcheck_get_status - First observed
agentcheck_list_incidents - First observed
agentcheck_promote_trace - First observed
agentcheck_record - First observed
agentcheck_run_now
Related MCP Connectors
AI agent testing: replay real sessions against a rebuilt staging environment to catch regressions.
Browser-based QA for AI-built software. Test pages with real browsers via agents.
AI agent run monitoring with incident replay and SLA receipts.
Monitoring that agents set up for themselves — cron jobs, CI/CD pipelines and AI agent runs.
Related MCP Servers
- AlicenseAqualityCmaintenanceAI-native browser testing, directly from your coding agent.3MIT
- AlicenseAqualityAmaintenanceMonitoring that agents set up for themselves — cron jobs, CI/CD pipelines and AI agent runs.502MIT
- AlicenseNot gradedqualityBmaintenanceEnables evidence-first regression testing for AI agents by turning production traces into reviewed, replayable cases that gate releases. It supports reproducible, auditable agent evaluation with controlled tool execution, evidence-based judging, and versioned quality gates.MIT
- AlicenseAqualityAmaintenanceProvides coding agents with visibility into test health through tools for flaky test detection, test quality linting, and LLM evaluation harness, enabling them to triage failures, review test quality, and check prompt changes for regressions.9Apache 2.0
Glama MCP Gateway
Add one secure layer between your agents and this server.