Skip to main content
Glama

Create API Test

create_api_test

Define an API test — a scheduled test of a production HTTP endpoint or MCP server: request + assertions (status/latency/headers/body or MCP tier) executed on an interval, tracking uptime and correctness. Use this to start watching an HTTP endpoint or an MCP server. Set type="HTTP" and the httpTarget/statusPattern params for a web endpoint, or type="MCP" and the mcpTarget params for an MCP server. The URL, headers, body and MCP tool arguments accept Postman-style variables that are resolved once per run: {{uuid}}, {{now}} (ISO 8601 UTC), {{date}}, {{timestamp}} (Unix seconds), {{timestampMs}}, {{timestampNs}}, {{randomInt}} (0-1000) and {{traceId}} (the run's W3C trace id, also sent as the traceparent header). Unknown names are left as written; write {{ for a literal {{. Returns the created API test, including its generated id and current state. Destinations (channelIds) are optional and ADMIN-only; see that parameter. Before it is saved, the test is run once exactly as the schedule will run it (one attempt, no retries). If that trial run passes, the test is created and the response includes the run as trialRun. If it fails, nothing is saved and this call returns an error that says what failed; fix the configuration and call again rather than retrying unchanged. The target URL must be reachable from the public internet; private, loopback and cloud-internal addresses are refused. Run detect_mcp_protocol on the server URL first to see which protocol revisions it speaks.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>",
      "type": "<type>",
      "intervalSeconds": "<intervalSeconds>",
      "timeoutSeconds": "<timeoutSeconds>",
      "retries": "<retries>",
      "failureThreshold": "<failureThreshold>",
      "enabled": "<enabled>"
    },
    "name": "create_api_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
nameYesHuman-readable name for the API test, e.g. 'checkout API health'
typeYesAPI test type: 'HTTP' for a web endpoint, 'MCP' for an MCP server
mcpUrlNoMCP only: the MCP server URL to probe
enabledYesWhether the API test starts enabled (scheduled) or paused
httpUrlNoHTTP only: the http(s) URL to probe. Supports {{uuid}}, {{now}}, {{date}}, {{timestamp}}, {{timestampMs}}, {{timestampNs}}, {{randomInt}} and {{traceId}} variables, resolved once per run; write \{{ for a literal {{.
mcpTierNoMCP only: assertion depth — UPTIME (the endpoint answers as an MCP server, or gates access with 401, 402 or 403; takes no credential and no protocol revision), HANDSHAKE (initialize completes), TOOLS_LIST (check expectedTools are advertised) or TOOL_CALL (invoke a tool)
retriesYesRetries per run before recording a failure (0-5)
httpBodyNoHTTP only: request body to send; takes the same variables as the URL
toolNameNoMCP TOOL_CALL tier: name of the tool to invoke
channelIdsNoNotification destinations for this test: registry channel UUIDs (the id field from the investigation service's list_alert_channels, not Slack channel ids). Omit to inherit the tenant's API test default (Settings > Notifications); pass an empty list to switch this test's alerts off. Requires the ADMIN role. Not stored here: notification routing is resolved per test, then per source, then the tenant default. Read current destinations with get_api_test_destinations. Change later with set_api_test_destinations.
httpMethodNoHTTP only: request method — GET, POST, PUT, PATCH, DELETE or HEAD
httpHeadersNoHTTP only: request headers to send, as a name->value map; values take the same variables as the URL
expectedToolsNoMCP TOOLS_LIST tier: tool names the server must advertise
statusPatternNoHTTP only: expected status matcher — 3 chars, digits or 'x' wildcards. '200' matches exactly 200; '2xx' matches any 2xx; '20x' matches 200-209
headerMatchersNoHTTP only: response headers that must match, as a name->value map
timeoutSecondsYesPer-run timeout in seconds, 1-30
intervalSecondsYesHow often to run the API test, in seconds (minimum 30)
mcpCredentialIdNoMCP only: id of a stored credential to authenticate the MCP session
failureThresholdYesConsecutive failing runs before the API test flips to DOWN (1-10)
httpCredentialIdNoHTTP only: id of a stored credential to authenticate the request (from list_api_test_credentials); omit for an unauthenticated call
maxLatencyMillisNoHTTP only: fail if response takes longer than this many milliseconds
toolArgumentsJsonNoMCP TOOL_CALL tier: JSON object of arguments to pass to the tool; omit it for a tool that takes none. Supports {{uuid}}, {{now}}, {{date}}, {{timestamp}}, {{timestampMs}}, {{timestampNs}}, {{randomInt}} and {{traceId}} variables, resolved once per run; write \{{ for a literal {{.
bodyValidationTierNoHTTP only: body validation tier — NONE, VALID_JSON, JSON_CONTAINS (sample fields must be present with the same types, extra fields allowed), JSON_SHAPE (exact field set) or EXACT_MATCH
mcpMaxLatencyMillisNoMCP only: fail if the API test takes longer than this many milliseconds
mcpProtocolRevisionNoMCP only: protocol revision policy — ANY (the default when omitted), NEWEST_SUPPORTED, or a dated revision: 2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25, 2026-07-28. ANY checks that the endpoint answers as an MCP server and records whatever revision it speaks, without asserting it. NEWEST_SUPPORTED runs against the newest protocol revision the server speaks and moves with it when the server changes. A dated revision pins the test to it and the test fails if the server stops accepting it. Get the server's revisions from detect_mcp_protocol.
bodyValidationSampleNoHTTP only: sample body for JSON_CONTAINS/JSON_SHAPE (a shape template) or EXACT_MATCH tiers
mcpResultValidationTierNoMCP TOOL_CALL tier: result body validation tier — NONE, VALID_JSON, JSON_CONTAINS (sample fields must be present with the same types, extra fields allowed), JSON_SHAPE (exact field set) or EXACT_MATCH
mcpResultValidationSampleNoMCP TOOL_CALL tier: sample result for JSON_CONTAINS/JSON_SHAPE/EXACT_MATCH validation

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / toolArgumentsJson / description
      Previous value: -"MCP TOOL_CALL tier: JSON object of arguments to pass to the tool. Supports {{uuid}}, {{now}}, {{date}}, {{timestamp}}, {{timestampMs}}, {{timestampNs}}, {{randomInt}} and {{traceId}} variables, resolved once per run; write \\{{ for a literal {{."New value: +"MCP TOOL_CALL tier: JSON object of arguments to pass to the tool; omit it for a tool that takes none. Supports {{uuid}}, {{now}}, {{date}}, {{timestamp}}, {{timestampMs}}, {{timestampNs}}, {{randomInt}} and {{traceId}} variables, resolved once per run; write \\{{ for a literal {{."
  2. Changed2 schema fields changed
    • changedInput schema / properties / mcpProtocolRevision / description
      Previous value: -"MCP only: protocol revision policy — ANY (the default when omitted), NEWEST_SUPPORTED, or a dated revision: 2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25, 2026-07-28. ANY checks that the endpoint answers as an MCP server and records whatever revision it speaks, without asserting it; at the HANDSHAKE tier, a server that answers 401 or 403 when the test has no credential counts as up. NEWEST_SUPPORTED runs against the newest protocol revision the server speaks and moves with it when the server changes. A dated revision pins the test to it and the test fails if the server stops accepting it. Get the server's revisions from detect_mcp_protocol."New value: +"MCP only: protocol revision policy — ANY (the default when omitted), NEWEST_SUPPORTED, or a dated revision: 2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25, 2026-07-28. ANY checks that the endpoint answers as an MCP server and records whatever revision it speaks, without asserting it. NEWEST_SUPPORTED runs against the newest protocol revision the server speaks and moves with it when the server changes. A dated revision pins the test to it and the test fails if the server stops accepting it. Get the server's revisions from detect_mcp_protocol."
    • changedInput schema / properties / mcpTier / description
      Previous value: -"MCP only: assertion depth — HANDSHAKE (just connect), TOOLS_LIST (check expectedTools are advertised) or TOOL_CALL (invoke a tool)"New value: +"MCP only: assertion depth — UPTIME (the endpoint answers as an MCP server, or gates access with 401, 402 or 403; takes no credential and no protocol revision), HANDSHAKE (initialize completes), TOOLS_LIST (check expectedTools are advertised) or TOOL_CALL (invoke a tool)"
  3. Changed1 schema field changed
    • changedInput schema / properties / mcpProtocolRevision / description
      Previous value: -"MCP only: protocol revision the test speaks — 2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25, 2026-07-28 or NEWEST_SUPPORTED (the default when omitted). NEWEST_SUPPORTED runs against the newest protocol revision the server speaks and moves with it when the server changes; a dated revision pins the test to it and the test fails if the server stops accepting it. Get the server's revisions from detect_mcp_protocol."New value: +"MCP only: protocol revision policy — ANY (the default when omitted), NEWEST_SUPPORTED, or a dated revision: 2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25, 2026-07-28. ANY checks that the endpoint answers as an MCP server and records whatever revision it speaks, without asserting it; at the HANDSHAKE tier, a server that answers 401 or 403 when the test has no credential counts as up. NEWEST_SUPPORTED runs against the newest protocol revision the server speaks and moves with it when the server changes. A dated revision pins the test to it and the test fails if the server stops accepting it. Get the server's revisions from detect_mcp_protocol."
  4. Changed2 schema fields changed
    • addedInput schema / properties / mcpProtocolRevision
      Added value: +{
      +  "description": "MCP only: protocol revision the test speaks — 2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25, 2026-07-28 or NEWEST_SUPPORTED (the default when omitted). NEWEST_SUPPORTED runs against the newest protocol revision the server speaks and moves with it when the server changes; a dated revision pins the test to it and the test fails if the server stops accepting it. Get the server's revisions from detect_mcp_protocol.",
      +  "type": "string"
      +}
    • changedInput schema / properties / timeoutSeconds / description
      Previous value: -"Per-run timeout in seconds; must be positive and not exceed intervalSeconds"New value: +"Per-run timeout in seconds, 1-30"
  5. Changed5 schema fields changed
    • changedInput schema / properties / bodyValidationSample / description
      Previous value: -"HTTP only: sample body for JSON_SHAPE (a shape template) or EXACT_MATCH tiers"New value: +"HTTP only: sample body for JSON_CONTAINS/JSON_SHAPE (a shape template) or EXACT_MATCH tiers"
    • changedInput schema / properties / bodyValidationTier / description
      Previous value: -"HTTP only: body validation tier — NONE, VALID_JSON, JSON_SHAPE or EXACT_MATCH"New value: +"HTTP only: body validation tier — NONE, VALID_JSON, JSON_CONTAINS (sample fields must be present with the same types, extra fields allowed), JSON_SHAPE (exact field set) or EXACT_MATCH"
    • changedInput schema / properties / channelIds / description
      Previous value: -"Notification destinations for this test: registry channel UUIDs (the id field from the investigation service's list_alert_channels, not Slack channel ids). Omit to inherit the tenant's API test default (Settings > Notifications); pass an empty list to switch this test's alerts off. Requires the ADMIN role. Not stored here: forwarded to the notify gateway, which routes per test, then per source, then the tenant default. Read current destinations with get_api_test_destinations. Change later with set_api_test_destinations."New value: +"Notification destinations for this test: registry channel UUIDs (the id field from the investigation service's list_alert_channels, not Slack channel ids). Omit to inherit the tenant's API test default (Settings > Notifications); pass an empty list to switch this test's alerts off. Requires the ADMIN role. Not stored here: notification routing is resolved per test, then per source, then the tenant default. Read current destinations with get_api_test_destinations. Change later with set_api_test_destinations."
    • changedInput schema / properties / mcpResultValidationSample / description
      Previous value: -"MCP TOOL_CALL tier: sample result for JSON_SHAPE/EXACT_MATCH validation"New value: +"MCP TOOL_CALL tier: sample result for JSON_CONTAINS/JSON_SHAPE/EXACT_MATCH validation"
    • changedInput schema / properties / mcpResultValidationTier / description
      Previous value: -"MCP TOOL_CALL tier: result body validation tier — NONE, VALID_JSON, JSON_SHAPE or EXACT_MATCH"New value: +"MCP TOOL_CALL tier: result body validation tier — NONE, VALID_JSON, JSON_CONTAINS (sample fields must be present with the same types, extra fields allowed), JSON_SHAPE (exact field set) or EXACT_MATCH"
  6. Changed8 schema fields changed
    • addedInput schema / $schema
      Added value: +"https://json-schema.org/draft/2020-12/schema"
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / failureThreshold / format
      Added value: +"int32"
    • addedInput schema / properties / intervalSeconds / format
      Added value: +"int32"
    • addedInput schema / properties / maxLatencyMillis / format
      Added value: +"int64"
    • addedInput schema / properties / mcpMaxLatencyMillis / format
      Added value: +"int64"
    • addedInput schema / properties / retries / format
      Added value: +"int32"
    • addedInput schema / properties / timeoutSeconds / format
      Added value: +"int32"
  7. Changed1 schema field changed
    • changedInput schema / properties / channelIds / description
      Previous value: -"Notification destinations for this test: registry channel UUIDs (the id field from the investigation service's list_alert_channels, not Slack channel ids). Omit to inherit the tenant's API test default (Settings > Notifications); pass an empty list to switch this test's alerts off. Requires the ADMIN role. Not stored here: forwarded to the notify gateway, which routes per test, then per source, then the tenant default. Read current destinations with the investigation service's get_notification_destinations (source e2e-monitoring, subject = this test's id). Change later with set_api_test_destinations."New value: +"Notification destinations for this test: registry channel UUIDs (the id field from the investigation service's list_alert_channels, not Slack channel ids). Omit to inherit the tenant's API test default (Settings > Notifications); pass an empty list to switch this test's alerts off. Requires the ADMIN role. Not stored here: forwarded to the notify gateway, which routes per test, then per source, then the tenant default. Read current destinations with get_api_test_destinations. Change later with set_api_test_destinations."
  8. Changed1 schema field changed
    • addedInput schema / properties / channelIds
      Added value: +{
      +  "description": "Notification destinations for this test: registry channel UUIDs (the id field from the investigation service's list_alert_channels, not Slack channel ids). Omit to inherit the tenant's API test default (Settings > Notifications); pass an empty list to switch this test's alerts off. Requires the ADMIN role. Not stored here: forwarded to the notify gateway, which routes per test, then per source, then the tenant default. Read current destinations with the investigation service's get_notification_destinations (source e2e-monitoring, subject = this test's id). Change later with set_api_test_destinations.",
      +  "items": {
      +    "type": "string"
      +  },
      +  "type": "array"
      +}
  9. Added
  10. Removed
  11. Changed4 schema fields changed
    • changedInput schema / properties / httpBody / description
      Previous value: -"HTTP only: request body to send"New value: +"HTTP only: request body to send; takes the same variables as the URL"
    • changedInput schema / properties / httpHeaders / description
      Previous value: -"HTTP only: request headers to send, as a name->value map"New value: +"HTTP only: request headers to send, as a name->value map; values take the same variables as the URL"
    • changedInput schema / properties / httpUrl / description
      Previous value: -"HTTP only: the http(s) URL to probe"New value: +"HTTP only: the http(s) URL to probe. Supports {{uuid}}, {{now}}, {{date}}, {{timestamp}}, {{timestampMs}}, {{timestampNs}}, {{randomInt}} and {{traceId}} variables, resolved once per run; write \\{{ for a literal {{."
    • changedInput schema / properties / toolArgumentsJson / description
      Previous value: -"MCP TOOL_CALL tier: JSON object of arguments to pass to the tool"New value: +"MCP TOOL_CALL tier: JSON object of arguments to pass to the tool. Supports {{uuid}}, {{now}}, {{date}}, {{timestamp}}, {{timestampMs}}, {{timestampNs}}, {{randomInt}} and {{traceId}} variables, resolved once per run; write \\{{ for a literal {{."
  12. Added

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes far beyond the annotations (readOnly=false, openWorld=true, destructive=false): it discloses the pre-save trial run (one attempt, no retries), that nothing is saved on failure, that the response includes trialRun, that the target must be publicly reachable (private/loopback/cloud-internal refused), that channelIds are ADMIN-only, and how Postman-style variables are resolved per run. This is exactly the mutation-permission and side-effect context the annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and mode selection, then the variable semantics, then the trial-run contract — a sensible order. It is dense rather than padded, but the variable list is spelled out in full and then repeated nearly verbatim in three schema parameter descriptions, so a little effort is spent restating structured content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 28-parameter mutation tool with no output schema, it covers the essentials an agent needs before calling: what gets created, that a trial run gates creation, what comes back (created test with id and state plus trialRun), and what error behavior to expect. The remaining gaps are per-parameter details already carried by 100%-covered schema descriptions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage the schema already documents each parameter, so the description's value is in cross-parameter guidance: selecting type decides which parameter family applies, and it notes credential sourcing (list_api_test_credentials) and destination resolution ordering. Minor inaccuracy hurts slightly: it refers to 'httpTarget/mcpTarget' parameters that do not exist in the schema (the actual names are httpUrl/mcpUrl).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Define an API test') and precisely frames scope: scheduled test of a production HTTP endpoint or MCP server, with request + assertions on an interval. It distinguishes the two modes explicitly (type="HTTP" with httpUrl/statusPattern vs type="MCP" with mcpUrl) and names related siblings (detect_mcp_protocol, get/set_api_test_destinations), so an agent can separate it from update_api_test or list_api_tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear when-to-use context ('start watching an HTTP endpoint or an MCP server'), a prerequisite ordering ('Run detect_mcp_protocol on the server URL first'), and explicit failure handling ('fix the configuration and call again rather than retrying unchanged'). It stops short of naming when NOT to use it — e.g. that update_api_test is the tool for an existing test — so it is not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources