Skip to main content
Glama

EvalGate

Server Details

Run AI evaluations, record agent traces and gate releases on evidence, with scoped org access.

If you are the author of this connector, you can claim ownership by verifying the domain or GitHub account it belongs to. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

B3.1/5.0

Scored across 12 tools

Disambiguation3/5

The five connection-related tools (connections_catalog, connections_graph, connections_list, connections_roots, connector_health) all perform read-only inspection of connector state, and an agent could easily misselect among catalog/graph/list since their boundaries are subtle. Distractors like evaluations_list vs runs_list and baseline_lineage vs run_explain also overlap in the evidence-lineage theme. Descriptions do attempt to differentiate, but the connection cluster remains confusing.

Naming Consistency4/5

All names use snake_case with a resource_action or resource_attribute pattern (connections_list, docs_search, runs_list, project_plan), which is readable and mostly consistent. Minor deviations include noun_noun forms like connector_health and baseline_lineage, plus plural/singular mismatches between connections_* and connector_health, and run_explain vs runs_list.

Tool Count4/5

At 12 tools the count is well within a reasonable range and each tool nominally earns its place. However, five closely related connection tools make the set feel slightly heavy on one resource family compared to the rest.

Completeness3/5

The surface is almost entirely read-only inspection, listing, and explanation — there is no create/update/delete for evaluations, runs, baselines, or connections, and no way to trigger a run or promote a baseline. For an evaluation-gating server this leaves notable lifecycle gaps that force agents to work around missing operations.

Available Tools

12 tools
baseline_lineageC
Read-onlyIdempotent
Inspect

Show the published baseline anchor and its evidence lineage.

NameRequiredDescriptionDefault
evaluationIdYesEvaluation ID returned by evaluations_list in the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 8 or "8").

Output Schema

NameRequiredDescription

No output parameters

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this read-only, idempotent, non-destructive and closed-world, so safety is covered. Beyond that, the description reveals nothing: it does not say what happens when no baseline has been published, whether the lineage is a flat list or a chain, or how deep ancestry is traversed. With annotations carrying the safety profile, this is a missed opportunity rather than a contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence with zero padding, and the core resource is front-loaded. It is efficient but borders on under-specification given the jargon it relies on.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return shape needn't be explained, but the description still leaves the central concept ('evidence lineage') opaque and says nothing about the empty/no-baseline case or the relationship to evaluations_list. For a provenance tool the description should orient the agent on what lineage means here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single evaluationId parameter is fully documented in the schema, including its integer-or-string form. The description adds no additional meaning such as scope or interpretation of that ID. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The verb 'Show' and resource 'baseline anchor and its evidence lineage' are specific enough to suggest a read of baseline provenance data. However, 'baseline anchor' and 'evidence lineage' are unexplained domain jargon, so an agent cannot form a confident picture of what is returned. No sibling overlaps directly, but the description does nothing to distinguish it from evaluations_list beyond the shared evaluationId.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to call this versus evaluations_list or runs_list, nor of prerequisites such as having a published baseline. The only implicit signal is the required evaluationId, which the schema already supplies.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections_catalogB
Read-onlyIdempotent
Inspect

Inspect all connector definitions and exact installed setup paths. Unavailable blueprints and live verification are separate facts.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
connectorsYes
liveVerifiedYes
registryVersionYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds a genuinely useful boundary — that this is a static definition catalog and that blueprint unavailability and live verification are separate facts not to be inferred from it — but stops short of explaining completeness, staleness, or ordering.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler, and the scope caveat is placed immediately after the purpose. It is slightly too terse — the second sentence is cryptic on first read — but nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

A zero-parameter, read-only tool with an output schema and full annotation coverage has modest description needs, so the bar is low and this mostly clears it. However, the description never clarifies what a 'connector definition' record contains relative to connections_list/connections_graph, which is the one thing an agent still has to guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and schema description coverage is 100%, so there is nothing for the description to disambiguate; the baseline applies. The description correctly does not waste space restating an empty input schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The verb 'Inspect' plus 'all connector definitions and exact installed setup paths' gives a rough sense of the resource, but 'connector definitions' and 'installed setup paths' are not concretely defined, and the tool is not clearly distinguished from siblings like connections_list or connections_graph. The second sentence hints at scope boundaries rather than stating the purpose more precisely.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: 'Unavailable blueprints and live verification are separate facts' suggests that health/availability questions belong to another tool (likely connector_health), but no alternative is named and no explicit 'use this when...' condition is given. An agent must infer the routing decision rather than read it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections_graphC
Read-onlyIdempotent
Inspect

Read canonical connection references and currently visible exact resource revisions. No source access grant or execution.

NameRequiredDescriptionDefault
idYesStable graph UUID returned by connections.backfill, not a credential ID.

Output Schema

NameRequiredDescription
idYes
nameYes
sourceYes
driverIdYes
revisionYes
endpointsYes
revokedAtYes
configurationYes
authorityEpochYes
lifecycleStateYes
selectionEpochYes
dispatchEnabledYes
selectionVisibilityYes
selectedResourceRevisionIdsYes

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description's genuinely additive contribution is "No source access grant or execution" – clarifying that reading the graph does not require or trigger access to the underlying source – plus the "currently visible" qualifier implying permission-filtered results. That is real but thin context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with the core action front-loaded and no filler. Brevity here slightly hurts clarity, but nothing is redundant or padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists and annotations are rich, so return values and safety need not be described. However, for a tool sitting among five similarly named connection siblings, the description omits the routing information an agent needs most to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the schema already documents that id is a graph UUID returned by connections.backfill and not a credential ID. The description adds nothing further about the parameter, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Read canonical connection references and currently visible exact resource revisions" names a verb (read) and a resource (connection references/revisions), but the phrasing is jargon-heavy and abstract. It does not differentiate from connections_list, connections_roots, or connections_catalog, so an agent can't tell from the description alone which of the connection tools to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this tool versus the many sibling connection tools, nor any prerequisite or exclusion beyond the trailing caveat. The reader is left to infer usage entirely from the schema's hint that the id comes from connections.backfill.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections_listA
Read-onlyIdempotent
Inspect

List current workspace connection metadata, with source restrictions, freshness and bounded continuation. Does not reveal secrets or grant source access.

NameRequiredDescriptionDefault
qNoCase-insensitive connection name or service search; up to 200 characters.
limitNoMaximum connections in one page; defaults to 25, capped at 100.
cursorNoOpaque nextCursor from this caller and inventory revision; refresh if the inventory changes.
sourceNoFilter by canonical store; direct metadata still requires admin role and scope.

Output Schema

NameRequiredDescription
itemsYes
hasMoreYes
sourcesYes
completeYes
nextCursorYes
observedAtYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive, and closed-world behavior. The description adds useful security and scope context: it discloses that secrets are not revealed and that source restrictions and freshness metadata are surfaced, along with bounded continuation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loaded with the core action and followed by the key safety boundary. Every clause contributes to understanding the tool's scope or limitations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema, full parameter descriptions, and strong annotations, so the description need not explain return shapes or safety semantics. It covers the listing scope, security boundary, and pagination behavior, though sibling differentiation is the main remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already well documented. The description loosely gestures at source filtering and continuation, but it does not add syntax or format details beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a clear verb and resource: 'List current workspace connection metadata.' It is more specific than the tool name alone, but it does not explicitly distinguish itself from siblings such as connections_catalog, connections_graph, or connections_roots.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the listing scope and by the boundary statement 'Does not reveal secrets or grant source access,' which tells the agent what this tool will not do. However, it names no alternatives and gives no explicit when-to-use or when-not-to-use routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections_rootsA
Read-onlyIdempotent
Inspect

Page independent configured account roots using current administrator authority. Does not list secrets or imply verified connector availability.

NameRequiredDescriptionDefault
qNoLiteral case-insensitive root name/driver search; wildcard characters have no special meaning.
limitNoBounded page size; defaults to 25 and is capped at 100.
cursorNoOpaque nextCursor for the same caller, workspace, search, limit and observed metadata revision; refresh after changes.

Output Schema

NameRequiredDescription
itemsYes
hasMoreYes
nextCursorYes
dispatchEnabledYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive behavior, and closed-world scope. The description adds useful context beyond that by noting the administrator authority requirement and clarifying that results do not expose secrets or imply connector verification.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tightly written sentences. It front-loads the core action and follows with a useful scope caveat, with no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema and fully described input schema, so the description does not need to explain return values. It adds relevant auth and scope caveats, though it could say slightly more about pagination expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully documents all three parameters, including q, limit, and cursor semantics. The description only implies pagination through the verb 'Page' and adds no parameter details beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it pages independent configured account roots. It is clear what the tool does, but it does not explicitly differentiate itself from siblings such as connections_list or connections_catalog.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'using current administrator authority' implies an admin-only context, and the exclusion of secrets and verified connector availability narrows expectations. However, it gives no explicit guidance on when to use this tool rather than connections_list, connections_catalog, or connections_graph.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connector_healthB
Read-onlyIdempotent
Inspect

Show the latest connector-readiness reports for the organization.

NameRequiredDescriptionDefault
connectorKeyNoExact report key or metadata connector key to filter the latest 50 persisted reports; omit for all reports in that snapshot. Requires admin:org.

Output Schema

NameRequiredDescription
reportsYes
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
configuredYesWhether this query found persisted readiness reports; not a live credential or provider-health check.
explanationYesHuman-readable interpretation and evidence limitations.
connectorKeyYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so the safety profile is fully covered. The description adds only the notion of 'latest' and organizational scope; it says nothing about snapshot cadence, staleness, or what 'readiness' means, which would be genuinely additive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence with no filler. It is efficient, though so terse that it borders on under-specification rather than wasteful verbosity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema and rich annotations carry most of the burden, and the lone optional parameter is fully documented in the schema. Still, the term 'connector-readiness reports' is never explained, so an agent cannot anticipate what the snapshot contains or how fresh it is.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter's description is detailed (exact report key vs. metadata connector key, latest 50, requires admin:org). The description adds nothing beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('show') and resource ('connector-readiness reports') with an organizational scope, so the agent knows exactly what it returns. It does not explicitly differentiate itself from siblings such as connections_catalog or connections_list, which keeps it out of 5 territory.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to call this versus the sibling connection/catalog tools, and no mention of prerequisites beyond the permission note that lives in the schema. The agent must infer that this is a read-only status snapshot.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evaluations_listA
Read-onlyIdempotent
Inspect

List evaluations with a stable cursor for the organization.

NameRequiredDescriptionDefault
limitNoMaximum results in this page, from 1 to 50; defaults to 20.
cursorNoOpaque nextCursor from the preceding page of this tool. Omit for the first page.
statusNoRestrict evaluations to this lifecycle status; omit to include all statuses.

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is fully covered. The description adds only the org-scoping and 'stable cursor' pagination property; it says nothing about page exhaustion, ordering, or rate limits beyond that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the scope constraint follows the verb immediately. Nothing is wasted or buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With rich annotations, a complete output schema, and 100% parameter coverage, the description needs little more than it provides. The only shortfall is that it doesn't characterize what an 'evaluation' record contains or clarify relation to runs_list.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so limit, cursor, and status are already fully documented with ranges and enum semantics. The description adds no parameter-level meaning beyond the schema, making the baseline 3 correct.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List evaluations') and adds scope ('for the organization'), so an agent knows exactly what it retrieves. However, it never distinguishes itself from the closely related sibling runs_list, which an agent may confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the verb 'List' – the description never says when to call this versus runs_list, nor any preconditions or exclusions. It is adequate but leaves routing entirely to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

project_inspectC
Read-onlyIdempotent
Inspect

Inspect the evaluations and project keys available in the authenticated organization.

NameRequiredDescriptionDefault
projectKeyNoExact project key to filter the first 50 organization evaluations; omit to inspect that entire snapshot.

Output Schema

NameRequiredDescription
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
projectKeyYes
evaluationsYes
explanationYesHuman-readable interpretation and evidence limitations.
organizationIdYes
evaluationCountYesCount in this bounded snapshot, not a total across the organization.

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world behavior, so the safety profile is fully covered. The description adds a modest amount — that scope is limited to the authenticated organization — but stays silent on the notable behavior (the 50-evaluation cap, which only appears in the schema) and on what an "inspect" response contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no wasted words. It is efficient, though its brevity borders on under-specification rather than deliberate compactness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a rich output schema, full annotation coverage, and 100%-described parameters, much of the informational burden is already carried elsewhere. Still, a tool whose verb is "inspect" should explain what it returns or how it differs from `evaluations_list`; the definition leaves that hole for the agent to fill.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the sole `projectKey` parameter is thoroughly documented in the schema, including the 50-evaluation cap and the omit-to-inspect-everything default. The description contributes nothing about the parameter, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a verb ("Inspect") and two resources (evaluations and project keys) scoped to the authenticated organization, but "inspect" is a weak, ambiguous verb — it could mean list, snapshot, summarize, or validate. It also does not distinguish itself from the sibling `evaluations_list`, leaving the agent to guess how the two overlap.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this tool versus `evaluations_list`, `project_plan`, or the `connections_*` family, and no exclusions or prerequisites are given. The only contextual cue is the word "authenticated," which hints at an auth requirement but does not guide tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

project_planA
Read-onlyIdempotent
Inspect

Return a non-mutating next-step plan for connecting a repository, evaluation, or trace source.

NameRequiredDescriptionDefault
targetNoWorkflow to plan: local repository setup, hosted evaluation review, or trace ingestion. Defaults to repository; never executes the plan.repository

Output Schema

NameRequiredDescription
stepsYes
targetYes
mutatesYes
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
explanationYesHuman-readable interpretation and evidence limitations.

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description repeats that the plan is non-mutating but adds little behavioral context beyond what the annotations and schema description already provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no wasted words. It communicates the core behavior immediately and avoids unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with rich annotations and an output schema, the description is nearly complete. It explains the tool's planning role and non-mutating behavior, though sibling routing and explicit usage guidance are left implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description mentions the three workflow categories, which correspond to the target enum, but the schema already documents the target parameter at 100% coverage with richer semantics for each enum value. With high schema coverage, the description meets the baseline by not adding much beyond the structured schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: return a next-step plan for connecting a repository, evaluation, or trace source. It clearly identifies the tool as planning-oriented rather than execution-oriented, though it does not explicitly contrast itself with siblings such as project_inspect or connections_list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The purpose implies when the tool is useful: before connecting a repository, evaluation, or trace source. However, it gives no explicit when-to-use versus alternatives guidance and does not mention sibling tools that might handle inspection or connection listing instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_explainB
Read-onlyIdempotent
Inspect

Explain a run using its status, failed-case metadata, and evidence-lineage summary.

NameRequiredDescriptionDefault
runIdYesRun ID returned by runs_list; must belong to the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 3 or "3").

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint false, and destructiveHint false, so the safety profile is covered. The description adds some substance by naming the three components the explanation draws on, but it does not disclose constraints, limitations, or other behavioral details beyond that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It is appropriately sized for a one-parameter read-only tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich annotations, complete input schema, and the presence of an output schema, the description provides enough context to understand what the tool does. It is slightly minimal in not explicitly tying the runId source to runs_list, but that detail is covered by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single runId parameter is thoroughly documented in the schema, including format and ownership constraints. The description adds no parameter-level meaning, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (Explain) and resource (a run), and it names the ingredients of the explanation: status, failed-case metadata, and evidence-lineage summary. It does not explicitly differentiate itself from runs_list, so it falls short of the 5 level.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no when-to-use guidance, no exclusions, and does not name runs_list as the source for runId or as an alternative. The implied usage is essentially just the purpose itself, so there is no real usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runs_listB
Read-onlyIdempotent
Inspect

List evaluation runs with stable cursors and regression signals.

NameRequiredDescriptionDefault
limitNoMaximum results in this page, from 1 to 50; defaults to 20.
cursorNoOpaque nextCursor from the preceding page of this tool. Omit for the first page.
statusNoExact persisted run status to filter by; omit to include all statuses, including incomplete runs.
evaluationIdYesEvaluation ID returned by evaluations_list in the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 8 or "8").

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so the safety profile is fully covered. The description adds value by noting cursors are stable across pages and that results carry regression signals, but leaves what a 'regression signal' is, and how incomplete runs are handled, unexplained.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler; the resource comes first and the distinguishing traits follow. It is efficient, though the trailing phrase 'regression signals' is vague enough to feel like a compressed hint rather than a precise claim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema and rich annotations, the description needn't explain return values, so brevity is defensible. However, it introduces unexplained concepts ('regression signals') and leaves the relationship to run_explain and evaluations_list unstated, which an agent would need in order to route correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so limit, cursor, status and evaluationId are all documented in the schema itself. The description only alludes loosely to cursors and does not extend semantics for status filtering or the ID format — baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource with scope ('List evaluation runs') and adds two distinguishing traits: stable cursors and regression signals. It never names a sibling tool or explicitly contrasts with run_explain or evaluations_list, so it stops short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives no indication of when to reach for this tool versus run_explain or evaluations_list, and no preconditions are stated. The only contextual hint is 'with stable cursors,' which implies pagination but doesn't guide tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 12 tool updates
    • First observedbaseline_lineage
    • First observedconnections_catalog
    • First observedconnections_graph
    • First observedconnections_list
    • First observedconnections_roots
    • First observedconnector_health
    • First observeddocs_search
    • First observedevaluations_list
    • First observedproject_inspect
    • First observedproject_plan
    • First observedrun_explain
    • First observedruns_list

Related MCP Connectors

Related MCP Servers

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources