Skip to main content
Glama
Sign In

AnalyticsProof

by com.analyticsproof
OverviewDiscussionsAdmin

Get a CI recipe for a saved test

get_ci_recipe
Read-onlyIdempotent

Get a CI recipe for a saved test (permission: read)

The CI job that gates each pull or merge request on this saved test: it waits for the preview deploy, replays the journeys against it as a dry run (POST /targets/{id}/scans with url) and fails the job on a critical diff or a broken step. The YAML embeds only the API base, the target id and the pinned journey id, and reads the key from the ANALYTICSPROOF_API_KEY secret (an edit key; secret.where says where the platform keeps it). placement: on GitHub content is the whole workflow file at path; on GitLab (merge_job) and Bitbucket (merge_step) it is a fragment to merge into the existing file at path, never written over it. readiness.blockers are fixed in AnalyticsProof, not in the repository; readiness.warnings do not stop the job. journeys defaults to all when more than one journey of the test can run, else one; journeyId pins one journey (not with all). preview defaults to github_deployments on GitHub (the newest successful non-production deployment of the head commit) and placeholder elsewhere (fill in PREVIEW_URL); deploy_job (GitLab) takes previewJob, the job whose dotenv report sets PREVIEW_URL; pattern takes previewPattern, an http(s) URL that may use the CI variables of the platform. A refused value is 400 invalid_request on its parameter; a non-website target is 400 scan_type_mismatch; a journey of another target is 400 journey_mismatch on journeyId.

The job reads the API key from the CI secret the answer names; section 10 of https://analyticsproof.com/agents.md says how to provide it.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "id": "<id>",
      "platform": "<platform>"
    },
    "name": "get_ci_recipe"
  }
}

Input Schema

NameRequiredDescriptionDefault
idYesSaved test id: from GET /targets or the POST /targets answer.
previewNoWhere the job takes the preview URL from; default github_deployments on GitHub, placeholder elsewhere.
journeysNoReplay one journey or all runnable journeys of the test; default all when more than one can run and no journeyId is pinned, else one.
platformYesThe CI platform the job is written for.
journeyIdNoPin one journey of this test (journeys=one).
previewJobNoWith preview=deploy_job (GitLab): the deploy job named in needs:.
previewPatternNoWith preview=pattern: the preview URL, e.g. https://deploy-preview-$BITBUCKET_PR_ID--site.netlify.app (GitHub: $GITHUB_HEAD_REF, $PR_NUMBER; GitLab: $CI_MERGE_REQUEST_IID, $CI_COMMIT_REF_SLUG; Bitbucket: $BITBUCKET_PR_ID).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changed
    • addedInput schema / properties / id / description
      Added value: +"Saved test id: from GET /targets or the POST /targets answer."
    • addedInput schema / properties / journeys / description
      Added value: +"Replay one journey or all runnable journeys of the test; default all when more than one can run and no journeyId is pinned, else one."
    • addedInput schema / properties / platform / description
      Added value: +"The CI platform the job is written for."
    • addedInput schema / properties / preview / description
      Added value: +"Where the job takes the preview URL from; default github_deployments on GitHub, placeholder elsewhere."
  2. First observed

TDQS

A4.2/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent and non-destructive, yet the description still contributes substantive behavior: the returned YAML embeds only API base, target id and pinned journey id, the API key is read from the ANALYTICSPROOF_API_KEY CI secret, and the answer names where the platform keeps it. It further discloses the platform-specific write semantics (GitHub gets the whole workflow file at path; GitLab/Bitbucket get a fragment to merge, never written over) and the exact failure codes, none of which appear in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Nearly all sentences carry real information, but they are delivered as one dense run-on paragraph mixing return shape, secret handling, per-platform placement, defaults, constraints and error codes. There is no front-loading beyond the first clause and no bullet or grouping structure, which makes it slow for an agent to scan for the one fact it needs. It also delegates key provisioning to an external URL (section 10 of agents.md), adding an out-of-band hop.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 7 parameters, 3 enums and no output schema, the description carries the full burden and does so: it describes the shape and variable sources of the returned YAML, the per-platform placement rules, the default resolution for journeys and preview, and the error surface. An agent can call this tool and interpret the result correctly without opening any other document, apart from the deliberately externalized secret provisioning step.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, and the description earns one step above it by adding cross-parameter constraints the schema only partially conveys: journeyId pins one journey and is incompatible with journeys=all, preview defaults differ per platform, and deploy_job/pattern consume previewJob/previewPattern respectively. It also maps each rejected value to its specific 400 error code, which the schema does not do. Some of this overlaps heavily with the already-thorough enum descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Get a CI recipe for a saved test') and immediately unpacks what 'recipe' means: a CI job that waits for a preview deploy, replays journeys as a dry run via POST /targets/{id}/scans, and fails on a critical diff. No sibling tool in the list produces CI configuration, so there is no realistic confusion; the only minor cost is that 'recipe' is jargon that requires the second sentence to decode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is concrete: this is for gating each pull/merge request on a saved test, with per-platform guidance on where the generated content goes. It also pre-empts misuse by stating that readiness.blockers are fixed in AnalyticsProof rather than in the repository and that warnings do not stop the job. It does not phrase an explicit 'do not use when' clause or name an alternative, but no plausible alternative exists among the siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All AnalyticsProof toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources