agentcheck
Server Details
Synthetic checks, nightly regression replay and model-drift alerts for AI agents
- Status
- Healthy
- Uptime
- 100.0% over 39 days
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- agentwares/servers
- GitHub Stars
- 0
TDQS
Scored across 11 tools
Most tools target clearly distinct operations (create_target, add_check, record, list_incidents, run_now). Two pairs risk mild confusion: api_sunsets vs model_retirements (both deprecation lookups) and get_status vs statuspage_preview (both read status, though one is agentcheck's own targets and the other external Atlassian pages), but descriptions distinguish them well.
All tools share the agentcheck_ prefix and are predominantly verb_noun or noun_phrase (add_check, create_target, get_status, list_incidents, promote_trace, run_now). Minor deviations exist: bare verbs/nouns like record, api_sunsets and model_retirements break the pattern slightly but remain readable and predictable.
11 tools is well within a healthy range and each earns its place: core monitoring (create_target, add_check, run_now, record, promote_trace), observation (get_status, list_incidents), and supplementary lookups (pricing, api_sunsets, model_retirements, statuspage_preview). No obvious redundancy or filler.
Core lifecycle of enrolling targets, adding checks, running and recording traces is covered, but there is no way to list targets or checks, nor to update or delete them, which are notable management gaps. The read/observe side is solid (status, incidents, run_now), but the manage side is incomplete.
Available Tools
11 toolsagentcheck_add_checkAInspect
Add a check to one of your targets. A check runs an input (http path/prompt, mcp tool call, a2a message) on a schedule and judges the answer with a golden: exact, contains, regex and json_schema cost nothing; rubric and baseline use the LLM judge (baseline = same outcome as the last known-good answer). Returns the check id. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | what to run: http (GET path or POST prompt), mcp_tools_list, mcp_tool_call (tool + args), a2a_task (message) | |
| name | Yes | short name shown on the status page | |
| input | No | http: { path?, method?, body?, prompt? } · mcp_tool_call: { tool, args } · a2a_task: { message } | |
| golden | Yes | how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer. | |
| targetId | Yes | id from agentcheck_create_target or GET /api/v1/targets | |
| intervalSec | No | seconds between runs; the tier's interval is the floor (Free hourly, Starter+ 5 min) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this is a non-read-only, non-destructive, non-idempotent, closed-world mutation. The description adds real context beyond them: the API-key auth requirement, the returned check id, and the cost split between free deterministic judges and LLM-judge goldens. It stops short of stating duplicate-creation behavior or rate limits, which matter for a non-idempotent create.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the action and the concept definition, then the return value and auth note. The middle clause listing judge kinds and costs is dense and slightly run-on, but every sentence carries information and nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter mutation with a rich nested schema, full annotation coverage and no output schema, the description supplies the missing pieces an agent needs: what a check is, that a target must pre-exist, the auth requirement, the return value, and the cost implications of golden choice. Nothing essential is left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents kind, input shapes, golden fields, targetId provenance and intervalSec tier floors. The description largely restates the golden cost/judge distinction that the schema itself carries, adding no syntax or format detail beyond it. Baseline 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Add a check to one of your targets') and then defines the domain concept a check embodies: an input run on a schedule judged by a golden. It also names the sibling that produces the required targetId (agentcheck_create_target), so an agent can distinguish this from create/run/promote siblings without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the prerequisite 'one of your targets' and the explicit API-key requirement, and the golden taxonomy gives selection guidance among judge kinds. However, there is no explicit when-to-use/when-not, and no routing against the closest sibling agentcheck_run_now (immediate execution) or agentcheck_promote_trace.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_api_sunsetsARead-onlyIdempotentInspect
When does the API version this code pins stop working, and what happens then? Pass provider and version pairs as your code pins them (shopify 2025-10, meta v21.0, google-ads v22, hubspot v1, twilio-flex 2.16.2, twilio-notify v1, keap xml-rpc, exchange-online ews, stripe 2024-06-20). For each, returns every dated end the provider's own versioning, changelog or retirement page lists: status, end date (and whether the provider gives only a month), what happens after it (error, silent fall-forward to another version, app delisting, or unsupported), whether the failure is silent, the replacement version, the source URL with the date it was read, and a page with the details. Where the provider's own pages give different dates for the same end, other_dates lists each one with its page and date_note says which the counts run to. A pair with no entry comes back with the reason: older than every listed version, no end date published yet, or a provider that publishes none (Stripe). Covers Shopify, HubSpot, Meta Graph and Marketing APIs, Google Ads and Content API for Shopping, Twilio Flex UI and Notify, Thryv's Keap XML-RPC, and Exchange Web Services (EWS) in Exchange Online. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
| pins | Yes | provider and version pairs, e.g. [{"provider": "shopify", "version": "2025-10"}] |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations cover only the safety profile (readOnly, idempotent, non-destructive, closed-world). The description adds substantial behavior beyond that: the per-entry return fields, the failure taxonomy (error, silent fall-forward, delisting, unsupported), silent-failure flagging, replacement version, source URL with read date, other_dates/date_note conflict handling, and the explicit reasons a pair may return no entry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The question-style lead is well front-loaded and the return semantics are worth stating, but the passage is a dense single block and the provider coverage list ('Covers Shopify, HubSpot, Meta Graph...') restates providers already enumerated in the opening example list. That redundancy costs it against a tight definition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must carry the return contract, and it does: field-by-field contents, conflict handling via other_dates/date_note, and the three reasons an unmatched pair returns empty. For a read-only lookup tool this leaves nothing an agent needs missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the provider/version/api fields and their valid values are already documented structurally. The description's version examples (2025-10, v21.0, xml-rpc, ews, 2024-06-20) largely duplicate the schema's, and it never mentions the optional 'api' narrowing field, so it adds only marginal value. Baseline 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line states a precise verb-plus-resource job: given provider/version pairs your code pins, return when each version stops working and what happens then. It lists concrete provider/version examples (shopify 2025-10, meta v21.0, stripe 2024-06-20) and names the scope, so an agent can instantly separate it from siblings like agentcheck_model_retirements.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The when-to-use case ('when does the API version this code pins stop working') and the exact input shape are explicit, and it clarifies that no API key is needed. It never names a sibling to prefer or an exclusion condition, so it stops short of full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_create_targetADestructiveInspect
Enroll something to monitor: an http endpoint (JSON or OpenAI-style chat), a remote MCP server (Streamable HTTP url) or an A2A agent (origin with /.well-known/agent-card.json). Pass checks to create checks in the same call (POST /api/v1/probe proposes three). Returns the target id, the public status page, the badge SVG URL and a README snippet. The first check runs on the next scheduled tick, within five minutes; call agentcheck_run_now to run immediately. Free tier: 1 target, hourly; Starter+: 5-minute checks. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | endpoint URL (http/mcp) or the agent's origin (a2a) | |
| auth | No | credential the prober sends to your endpoint: {type:'bearer',token} or {type:'header',name,value}. Stored encrypted and never returned or logged. | |
| kind | Yes | http (JSON or chat endpoint), mcp (Streamable HTTP url, or npx/uvx package spec), a2a (agent card) | |
| name | No | display name; defaults to the host | |
| slug | No | URL slug for /<owner>/<slug>; defaults to a slug of the name | |
| alerts | No | where to send an incident when a check starts failing. Any combination; omit it and incidents are visible only on the status page. | |
| checks | No | checks to create right away (the probe proposes three) | |
| format | No | http: openai_chat (POST /chat/completions body), json (raw POST), get (plain fetch) | |
| headers | No | extra request headers | |
| isPublic | No | public status page + badge (default true) | |
| modelVar | No | the model your agent runs on (e.g. claude-sonnet-5); enables model-drift re-runs | |
| corpusUrl | No | RAG targets: public corpus URL for the nightly groundedness scorer (Pro) | |
| modelHeader | No | request header your endpoint accepts to override the model (drift re-runs try the new model) | |
| packageSpec | No | mcp only: `npx @org/server` / `uvx server` — runs on the mcpcheck runner, not the scheduled checks | |
| agentCardUrl | No | a2a: explicit agent card URL when it is not at /.well-known/agent-card.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructive/non-idempotent/open-world, and the description goes beyond them with scheduling behavior (first check on the next tick, within five minutes), rate/tier limits, the API-key requirement, and the shape of the return payload. It still doesn't say what happens on a slug/name collision or whether repeated calls create duplicate targets, which matters given idempotentHint=false.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded: enrollment scope first, then same-call checks, then return payload, then timing, then tier limits, then the auth prerequisite. Every sentence carries information, though the long parenthetical enumerations make it heavier than it needs to be.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 15-parameter, nested-object creation tool with no output schema and no output description, the description does the important work: it explains what is returned, the async first-run timing, the auth requirement, and the tier floors that bound intervalSec. Remaining gaps (collision behavior, whether duplicate enrollments are rejected) are minor but real.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3 and the schema already carries kind, url, auth, alerts, checks, modelVar, etc. The description adds only one behavioral note about parameters — that passing `checks` creates them in the same call and that POST /api/v1/probe proposes three — so it does not meaningfully exceed the schema for the other 14 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence states a concrete verb (enroll) and enumerates the exact resource types it accepts — http endpoint (JSON or OpenAI-style chat), remote MCP server (Streamable HTTP url), or A2A agent. It also states what comes back (target id, status page, badge SVG, README snippet), so an agent can distinguish it from siblings like agentcheck_add_check without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives real usage context: call agentcheck_run_now to run immediately instead of waiting up to five minutes for the next tick, and it surfaces tier constraints (Free: 1 target hourly; Starter+: 5-minute checks). It does not, however, say when NOT to use it — e.g. that adding checks to an existing target is agentcheck_add_check's job.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_get_pricingARead-onlyIdempotentInspect
Machine-readable pricing for agentcheck: tiers with monthly USD price, target limits, check interval and features, plus per-run add-ons. Same data as /pricing.json. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent and non-destructive, so the safety profile is covered. The description adds real context beyond them: it discloses that no authentication is required and that the payload is identical to /pricing.json, which helps an agent reason about caching and auth expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the purpose and followed by the payload contents and the no-API-key note. Every clause carries information; nothing is redundant with the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless, read-only lookup with no output schema, the description does the remaining work by describing exactly what the response includes and that no key is needed. An agent has everything required to call it correctly and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to clarify and the baseline of 4 applies. The description's enumeration of payload fields is return-value content, not parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('machine-readable pricing for agentcheck') and enumerates exactly what the response contains (tiers, monthly USD price, target limits, check interval, features, per-run add-ons). This is clearly distinguishable from every sibling (checks, targets, incidents, status), none of which deal with pricing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied rather than stated: an agent can infer this is the tool to call when it needs plan/tier or cost information. The 'No API key needed' note is a useful invocation precondition, but there is no explicit when-to-use vs when-not guidance or naming of alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_get_statusARead-onlyIdempotentInspect
Current status of a public monitored target: overall state, uptime over 24h/7d/30d, last check time, last nightly scores, open incidents and the badge/status URLs. Use the owner (GitHub login) and target slug from the status page URL https://agentwares-agentcheck.vercel.app//. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
| slug | Yes | target slug, e.g. demo | |
| owner | Yes | GitHub login of the target's owner, e.g. demo |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and closed-world behavior, so the safety profile is covered. The description adds value beyond that by disclosing the auth requirement ('No API key needed') and scoping the tool to public targets only, which the annotations do not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both front-loaded: the payload of returned fields comes first, the parameter-sourcing note second. The field enumeration is dense but each item is distinct and informative, with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by enumerating the returned fields, and it covers auth, target scope, and parameter derivation. Nothing an agent needs to call this correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description goes further by clarifying that owner is a GitHub login and that both owner and slug are read directly off the status page URL. That sourcing guidance is genuinely useful beyond the schema's bare examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific retrieval verb and resource ('Current status of a public monitored target') and enumerates exactly what is returned: overall state, uptime windows, last check time, nightly scores, open incidents, and badge/status URLs. An agent can distinguish it from siblings like list_incidents or statuspage_preview without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete context for when the tool applies ('public monitored target') and explains how to obtain the two required arguments from the status page URL. It stops short of naming an alternative tool or an explicit when-not-to-use condition, so it is clear context rather than full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_list_incidentsARead-onlyIdempotentInspect
Open and recent incidents across your targets (or one target): when they opened/closed, the failing check and the cause. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| targetId | No | narrow to one target; omit for incidents across every target on the account |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds two things beyond that: the API-key authentication requirement and the shape of what comes back (open/close times, failing check, cause).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence with the resource and scope front-loaded, followed by return-field hints and the auth prerequisite. Every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description compensates by naming the returned fields, and the auth requirement is stated. It stops short of describing ordering, limits, or what 'recent' means in practice.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single targetId parameter is fully documented in the schema, so the baseline is 3. The description only restates the targeting concept ('or one target') without adding format or constraint detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (list incidents), the scope (open and recent), and the axis of filtering (all targets or one). No sibling in the list covers incident retrieval, so the agent can route unambiguously.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the parenthetical '(or one target)' and the schema's 'omit for incidents across every target', but the description never states when to prefer this tool over another view or when it's not appropriate. Adequate, not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_model_retirementsARead-onlyIdempotentInspect
Is this model retiring? Pass model IDs as your code or config names them (gpt-5-mini, claude-sonnet-4-5-20250929, gemini-2.5-flash, openai/o3-mini). For each, returns every announced shutdown on OpenAI's, Anthropic's and Google's own deprecation pages (OpenAI API, Claude API, Gemini API, Vertex AI): status, shutdown date, recommended replacement, the replacement's list-price ratio per token, the API changes the provider documents, the source URL with the date it was read, and a page with the details. retiring: false means no shutdown for that ID is on those pages. Language models only. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
| models | Yes | model IDs exactly as your code or config names them, e.g. ["gpt-5-mini", "claude-sonnet-4-5"]; router prefixes such as openai/ or models/ are ignored, and fine-tuned IDs (ft:…) match their base model |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, non-destructive), so the description goes further and adds real operational context: no API key required, the exact upstream sources consulted (OpenAI/Anthropic/Google deprecation pages), data provenance with read dates, and the crucial negative-result semantics that 'retiring: false' means absence from those pages rather than a guarantee.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Core purpose is front-loaded in the first clause, and each subsequent sentence carries information. The long comma-list of returned fields is dense but justified since there is no output schema; still, it is close to exhaustive rather than optimally trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must describe return values, and it does so fully: status, shutdown date, replacement, price ratio, documented API changes, and source URL with read date. Combined with the negative-result explanation and source scope, an agent has everything needed to call and interpret it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the schema already documents prefix handling and fine-tuned ID matching, so the baseline is 3. The description still adds value by emphasizing 'as your code or config names them' and enumerating concrete ID styles across three providers, which reinforces the accepted format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a direct question that names both the verb and resource ('Is this model retiring?' / 'returns every announced shutdown'), and scopes it to language models only. It is clearly separable from siblings like agentcheck_api_sunsets and agentcheck_get_pricing without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Tells the agent the trigger condition (checking whether a model ID is retiring) and constraints ('Language models only', 'No API key needed'), plus how to supply IDs from code/config. It does not explicitly name which sibling to use for non-model deprecations, so no exclusions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_promote_traceAIdempotentInspect
Turn an imported or recorded trace into a replayable check whose golden is the recorded outcome (tool sequence + final-answer rubric). The check runs daily and in the nightly replay (Pro). Returns the check. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | trace id from agentcheck_record or POST /api/v1/targets/{id}/traces |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (not read-only, idempotent, non-destructive), so the description is free to add the operational context it does: the check is scheduled daily and in the nightly replay (Pro), it returns the created check, and it requires an API key. The auth requirement and scheduling behavior are genuinely useful details not present in the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with the transformation, then runtime behavior, then return value and auth requirement. No filler and nothing repeated from structured fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers what the tool creates, where the result runs, and the auth prerequisite, which is close to sufficient. It does not state whether promoting the same trace twice creates duplicates (relevant given idempotentHint=true), leaving one small gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single parameter (traceId) is documented in the schema, including its source (agentcheck_record or the traces endpoint). The description adds no further parameter syntax or constraints, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource: it turns an imported/recorded trace into a 'replayable check' with a defined golden (tool sequence + final-answer rubric). This is far more concrete than the sibling agentcheck_add_check, but the description never names or contrasts that sibling, so the differentiation is implicit rather than explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied: the tool operates on an 'imported or recorded trace', which suggests it follows agentcheck_record, and it notes the check runs daily and in the nightly replay (Pro). However, there is no explicit guidance on when to choose this over agentcheck_add_check or agentcheck_run_now.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_recordAInspect
Record one production interaction with your agent (the prompt or messages, the final answer, the tools it called, optionally the model) as a trace on a target. Call it from your agent or from a proxy in front of it after each task; promote a good trace with agentcheck_promote_trace to replay it nightly and catch regressions. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | short label, e.g. 'refund for order A-1029' | |
| input | Yes | the prompt, the messages array, or an object with prompt/messages | |
| model | No | the model that produced this answer, so drift re-runs can compare across models | |
| output | Yes | the agent's final answer | |
| targetId | Yes | id from agentcheck_create_target or GET /api/v1/targets | |
| toolCalls | No | tool calls in order |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false, idempotentHint=false, destructiveHint=false, so the write/non-idempotent nature is already conveyed structurally. The description adds genuinely useful context beyond that: it is a write operation that creates a trace on a target, and it 'Requires an API key' — an auth prerequisite the annotations do not express. It stops short of describing return payload or failure behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and its payload, then usage, then the sibling hand-off. Two sentences carry a lot without padding, though the parenthetical field list is slightly dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a six-parameter mutation tool with no output schema and full schema description coverage, the description covers purpose, invocation timing, sibling routing, and the API-key prerequisite. The only soft gap is what the call returns (a trace id), which no output schema promises, so this is close to complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all six parameters are documented in the schema itself (e.g. targetId points at agentcheck_create_target, model exists for drift comparison across models). The description mirrors those fields but adds no syntax, format, or ordering detail beyond the schema's baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Record') and resource ('one production interaction ... as a trace on a target'), and enumerates the captured payload (prompt/messages, final answer, tool calls, model). It is clearly distinguishable from siblings, and even names agentcheck_promote_trace as the follow-on step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says where and when to call it ('from your agent or from a proxy in front of it after each task') and routes the agent onward with 'promote a good trace with agentcheck_promote_trace to replay it nightly and catch regressions.' The use condition and the alternative are both spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_run_nowAInspect
Run every check of one of your targets immediately (outside the schedule) and return pass/fail per check with latency, judge cost and any incident opened or closed. Use it right after enrolling, or to confirm a fix. Requires an API key.
| Name | Required | Description | Default |
|---|---|---|---|
| targetId | Yes | target id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond the annotations by disclosing the auth requirement ('Requires an API key') and the side effects implied by 'any incident opened or closed', which is consistent with readOnlyHint=false and openWorldHint=true. It also explains that execution happens outside the schedule, which the annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action and the effect, followed by the usage cue and the auth note. Dense and mostly waste-free, though the parenthetical '(outside the schedule)' and the auth sentence could be tightened slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by enumerating the return payload (pass/fail per check, latency, judge cost, incidents opened/closed) and stating the auth prerequisite. Minor gaps remain, such as behavior when the target has no checks or whether incident creation is a side effect worth warning about.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents targetId. The phrase 'one of your targets' adds only a marginal hint about what the id refers to; there is no id format or lookup guidance beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb (Run) plus resource (every check of one of your targets) with a clear scope qualifier (immediately, outside the schedule). It is easily distinguished from siblings like agentcheck_add_check, agentcheck_get_status and agentcheck_create_target, which manage checks/targets rather than executing them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit trigger moments: 'right after enrolling, or to confirm a fix.' That is clear situational guidance, but it does not name an alternative tool (e.g. when to prefer the scheduled run or get_status instead) or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agentcheck_statuspage_previewARead-onlyIdempotentInspect
Read a public Atlassian Statuspage page once: its name, overall status, components (id, name, status, group) and unresolved incidents, from the page's own /api/v2/summary.json. Pass any address on the page: status.example.com, https://status.example.com/history or the JSON URL. Use it before POST /api/v1/statuspage, which turns the components you give a URL into a live agentcheck status page checked by agentcheck itself, with no signup; the answer carries that request's body. Same read as /statuspage/preview: only public addresses, 5-second timeout, 512 KB cap. Private pages cannot be read. No API key needed.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | the status page's address, e.g. https://status.example.com or its summary.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable operational details not in annotations: 5-second timeout, 512 KB cap, only public addresses, no API key needed, and that the answer carries the related POST request's body. It doesn't discuss retry behavior or rate limits, but this is solid added context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core read operation and its return fields, then layers on usage guidance and operational constraints. It is denser than necessary in places and could benefit from splitting the POST-related context, but every sentence adds information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single-parameter, read-only tool with full annotation coverage and no output schema, the description is broadly complete: it covers what is read, constraints (public only, timeout, size cap), auth (no key), and the related write flow. Minor gaps remain around return format specifics and error behavior, but nothing critical is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single url parameter is documented in the schema. The description adds useful flexibility hints (pass any address on the page, not just the JSON URL) which is beyond the schema, but does not detail format constraints or edge cases beyond that. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a precise verb and resource: reads a public Atlassian Statuspage page, enumerating exactly what is returned (name, overall status, components, unresolved incidents) and the data source (/api/v2/summary.json). This is distinguishable from every sibling tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit usage context (use it before POST /api/v1/statuspage) and describes the related write flow. It also states the exclusion (private pages cannot be read). However, it does not direct to a specific sibling alternative for cases where this tool is unsuitable, only noting private pages are out of scope.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
1 tool update
- Changed
agentcheck_api_sunsets3 fields changed- changed
Input schema / properties / pins / items / properties / api / descriptionPrevious value: -"optional, to narrow one provider to one API: admin or storefront (Shopify), graph-api or marketing-api (Meta), ads-api or content-api (Google), pipelines, rest or developer-platform (HubSpot); omit it to get every API of that provider with this version"New value: +"optional, to narrow one provider to one API: admin or storefront (Shopify), graph-api or marketing-api (Meta), ads-api or content-api (Google), pipelines, rest or developer-platform (HubSpot), flex-ui or notify (Twilio), exchange-online (Microsoft); omit it to get every API of that provider with this version" - changed
Input schema / properties / pins / items / properties / provider / descriptionPrevious value: -"the provider as you would name it: shopify, hubspot, meta (or facebook), google-ads, content-api, twilio-flex, keap (or thryv, infusionsoft), stripe"New value: +"the provider as you would name it: shopify, hubspot, meta (or facebook), google-ads, content-api, twilio-flex, twilio-notify, keap (or thryv, infusionsoft), ews (or exchange-online, microsoft), stripe" - changed
Input schema / properties / pins / items / properties / version / descriptionPrevious value: -"the version as the code pins it: 2025-10, v21.0, v22, v1, 2025.2, 2.16.2, xml-rpc, 2024-06-20"New value: +"the version as the code pins it: 2025-10, v21.0, v22, v1, 2025.2, 2.16.2, xml-rpc, ews, 2024-06-20"
1 tool update
- Added
agentcheck_statuspage_preview
1 tool update
- Added
agentcheck_api_sunsets
1 tool update
- Added
agentcheck_model_retirements
1 tool update
- Changed
agentcheck_create_target1 field changed- changed
Input schema / properties / packageSpec / descriptionPrevious value: -"mcp only: `npx @org/server` / `uvx server` — runs on the mcpcheck runner, not the minute checks"New value: +"mcp only: `npx @org/server` / `uvx server` — runs on the mcpcheck runner, not the scheduled checks"
4 tool updates
- Changed
agentcheck_add_check1 field changed- added
Input schema / properties / golden / descriptionAdded value: +"how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer."
- Changed
agentcheck_create_target3 fields changed- added
Input schema / properties / alerts / descriptionAdded value: +"where to send an incident when a check starts failing. Any combination; omit it and incidents are visible only on the status page." - added
Input schema / properties / auth / descriptionAdded value: +"credential the prober sends to your endpoint: {type:'bearer',token} or {type:'header',name,value}. Stored encrypted and never returned or logged." - added
Input schema / properties / checks / items / properties / golden / descriptionAdded value: +"how the answer is judged. exact / contains / regex / json_schema are free and deterministic; rubric and baseline call the LLM judge, and baseline compares against the last known-good answer."
- Changed
agentcheck_list_incidents1 field changed- added
Input schema / properties / targetId / descriptionAdded value: +"narrow to one target; omit for incidents across every target on the account"
- Changed
agentcheck_record2 fields changed- added
Input schema / properties / model / descriptionAdded value: +"the model that produced this answer, so drift re-runs can compare across models" - added
Input schema / properties / targetId / descriptionAdded value: +"id from agentcheck_create_target or GET /api/v1/targets"
8 tool updates
- First observed
agentcheck_add_check - First observed
agentcheck_create_target - First observed
agentcheck_get_pricing - First observed
agentcheck_get_status - First observed
agentcheck_list_incidents - First observed
agentcheck_promote_trace - First observed
agentcheck_record - First observed
agentcheck_run_now
Related MCP Connectors
AI agent testing: replay real sessions against a rebuilt staging environment to catch regressions.
Browser-based QA for AI-built software. Test pages with real browsers via agents.
AI agent run monitoring with incident replay and SLA receipts.
AI agent monitoring: alerts when your agents fail, stall or need you
Related MCP Servers
- AlicenseAqualityCmaintenanceAI-native browser testing, directly from your coding agent.3MIT
- AlicenseAqualityAmaintenanceMonitoring that agents set up for themselves — cron jobs, CI/CD pipelines and AI agent runs.503MIT
- AlicenseNot gradedqualityBmaintenanceEnables evidence-first regression testing for AI agents by turning production traces into reviewed, replayable cases that gate releases. It supports reproducible, auditable agent evaluation with controlled tool execution, evidence-based judging, and versioned quality gates.MIT
- AlicenseAqualityAmaintenanceProvides coding agents with visibility into test health through tools for flaky test detection, test quality linting, and LLM evaluation harness, enabling them to triage failures, review test quality, and check prompt changes for regressions.9Apache 2.0
Glama MCP Gateway
Add one secure layer between your agents and this server.