Skip to main content
Glama
verifyax

VerifyAX MCP

Official

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
VERIFYAX_API_KEYYesYour VerifyAX API key

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

This server publishes no instructions, or was last inspected before Glama recorded them.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tasks
{
  "list": {},
  "cancel": {},
  "requests": {
    "tools": {
      "call": {}
    }
  }
}
tools
{
  "listChanged": true
}

Tools

Functions exposed to the LLM to take actions

NameDescription
list_compatible_tagsA

Lists the full skill-tag catalogue filtered to tags compatible with a given scenario type (info_exchange or interview). Use recommend_scenario_tags or search_scenario_tags for a ranked shortlist when the user gave natural language or the list is too large. Returns each tag’s name, category, and description, and flags QnA tags that must be the only tag.

recommend_scenario_tagsA

Suggests skill tags for a scenario type using the same Workbench pipeline as the Scenario Generator (embeddings plus LLM, up to about 20 tags). Use when the user describes a scenario or agent and wants a ranked shortlist before generate_scenario. Prefer this over list_compatible_tags when the catalogue is large or the user gave natural-language context. Read-only (no generation credits). Provide context_prompt and/or agent_uuid (at least one). On recommender timeout or unavailability, try search_scenario_tags or list_compatible_tags.

search_scenario_tagsA

Finds skill tags for a scenario type by embedding similarity to a natural-language query (no LLM). Cheaper than recommend_scenario_tags — good for “find tags about X”. Read-only. A per-user tag-search rate limit applies in addition to the workspace public API limit; on rate limit, honor Retry-After, wait and retry, or use list_compatible_tags for the full filtered catalogue.

register_agentA

Registers an AI agent in VerifyAX given its name, connector type (A2A, API, DIRECTLINE for Copilot Studio, or MCP), URL, and optional auth or connector settings. For A2A agents it first verifies the agent card is reachable; DIRECTLINE and MCP agents are probed before creation. Returns the new agent’s uuid and whether connectivity was checked. Note: any token, Direct Line secret, or password passed here transits the conversation — prefer supplying credentials out of band where possible.

list_agentsB

Lists the AI agents registered in your VerifyAX workspace, optionally filtered by connector type (A2A, API, DIRECTLINE, EXTENSION, or MCP). Returns each agent’s uuid, name, type, and URL.

delete_agentA

Permanently deletes an agent from your VerifyAX workspace by its uuid. This cannot be undone. Returns confirmation of the deletion.

generate_scenarioA

Generates a new test scenario of a given type (info_exchange or interview) with optional skill tags and context. Invalid tag names or incompatible tags fail immediately with 400/422 before a job is queued; use recommend_scenario_tags, search_scenario_tags, or list_compatible_tags to pick valid tag names. Typically takes 30s–2min; task-capable MCP clients receive a pollable task handle immediately, while others block until generation finishes. Set num_scenarios greater than 1 for batch mode (requires tag_pool). Returns the new scenario’s uuid, or batch uuids when batching, or a structured error if generation fails for other reasons after the job starts.

list_scenariosA

Lists the test scenarios in your VerifyAX workspace, optionally filtered by type (info_exchange or interview) and status. Returns each scenario’s uuid, name, type, and status.

delete_scenarioA

Permanently deletes a scenario from your VerifyAX workspace by its uuid. This cannot be undone and fails if simulation runs still reference the scenario. Returns confirmation of the deletion.

evaluate_agentA

Runs an agent against a scenario and evaluates the result end to end. Typically takes 30s–30min; task-capable MCP clients receive a pollable task handle immediately, while others block until the evaluation completes. Give it an agent uuid and a scenario uuid; it previews cost, runs the simulation, waits for it, and returns the evaluation scores. Optional timeout_minutes (1–240) overrides the scenario default for this run.

list_recent_runsA

Lists recent simulation runs in your VerifyAX workspace, optionally filtered by status, agent, scenario, date range, search text, or run group. Returns each run’s uuid, status, agent, scenario, and evaluation handle.

get_run_detailsA

Fetches the full details of a single simulation run by its uuid, including its status and the evaluation results when they are available. Use after a run to inspect scores and outcome.

get_usage_summaryB

Summarizes VerifyAX usage events over an optional time range or for a specific simulation, scenario, or job. Paginates across all matching events (up to a cap) and returns the total event count, a breakdown by product area, and total platform spend in USD when the API reports it.

preview_run_costA

Estimates the credit cost of running an agent against a scenario before triggering it. Returns the estimated credits, your current balance, and any pending committed spend. Optional timeout_minutes (1–240) affects the run-cost estimate.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

TDQS

A3.9/5.0

Scored across 14 tools

Disambiguation4/5

The three tag-discovery tools (list_compatible_tags, recommend_scenario_tags, search_scenario_tags) share an overlapping goal but their descriptions clearly differentiate full-catalogue filtering, LLM-ranked recommendation, and embedding-based search. Most other tools target distinct resource/action pairs (scenario, agent, run, cost/usage).

Naming Consistency5/5

All tool names use a consistent snake_case verb_noun pattern: delete_scenario, list_agents, register_agent, generate_scenario, evaluate_agent, get_run_details, preview_run_cost, etc. Deviations are minor and still follow the same convention.

Tool Count5/5

14 tools is well within the ideal 3–15 range for a platform covering agents, scenarios, tags, runs, evaluation, and usage. Each tool appears to earn its place by covering a distinct lifecycle operation or query type.

Completeness3/5

Core create/list/delete operations exist for agents and scenarios, plus run evaluation, cost preview, and usage summary. However, update operations for agents and scenarios are missing, and there is no get_scenario or get_agent detail tool—only list endpoints—creating notable gaps for managing existing resources.

Maintenance

ActivityMaintained
ResponsivenessResponsive