Skip to main content
Glama

EvalGate

Server Details

Create and run AI evaluations, record traces, and inspect results with scoped organization access.

If you are the author of this connector, you can claim ownership by verifying the domain or GitHub account it belongs to. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL

TDQS

B3.3/5.0

Scored across 12 tools

Disambiguation3/5

The four connections.* tools (catalog, graph, list, roots) all concern connector/connection information and their boundaries—definitions vs. references vs. metadata vs. account roots—require careful reading to distinguish. connector.health also overlaps with connections.catalog, since both describe connector readiness/availability. Other tools (evaluations.list, runs.list, run.explain, docs.search) are reasonably distinct.

Naming Consistency4/5

The namespace.action dotted pattern (connections.list, project.inspect, docs.search) is applied consistently and reads predictably. The main deviation is inconsistent grammatical number across namespaces: 'connector' vs 'connections', 'run' vs 'runs', and 'evaluations' vs 'runs', which slightly muddies the scheme.

Tool Count4/5

12 tools is a well-scoped size for a domain spanning connections, evaluations, runs, projects, and docs. Each tool maps to a plausible inspection need, though the connection-related cluster could arguably be consolidated.

Completeness4/5

The surface covers listing, inspection, explanation, planning, lineage, and health across the evaluation/connection domain, with stable cursors and bounded pagination. It is intentionally read-only (no create/update/delete or trigger operations), which fits the stated non-mutating purpose but leaves no path for managing or executing evaluations.

Available Tools

12 tools
baseline.lineageC
Read-onlyIdempotent
Inspect

Show the published baseline anchor and its evidence lineage.

ParametersJSON Schema
NameRequiredDescriptionDefault
evaluationIdYesEvaluation ID returned by evaluations.list in the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 8 or "8").

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the safety and idempotency profile is fully covered without the description. The description adds one useful qualifier — that it shows the *published* baseline, implying unpublished/draft baselines are not returned — but says nothing about visibility permissions or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler. It is efficiently structured, though it is terse enough that it borders on under-specified rather than merely concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and the lone parameter is fully documented in the schema. The remaining gap is routing: an agent cannot tell from this description why it would call baseline.lineage instead of the sibling evaluation or run tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter is documented at 100% schema coverage, including the accepted integer-or-canonical-decimal-string forms and its source (evaluations.list), so the schema does the heavy lifting. The description adds no parameter meaning beyond that, which is the expected baseline for a fully covered schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a clear verb ("Show") and a resource ("published baseline anchor and its evidence lineage"), so the agent knows it is a read of lineage data. However, "baseline anchor" is domain jargon and the description does nothing to distinguish this from siblings like evaluations.list, run.explain, or project.inspect, which could plausibly surface related data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus the many sibling read tools (evaluations.list, run.explain, project.inspect). No prerequisites, no scope conditions, no exclusions. The only implied usage is that you need an evaluation ID.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections.catalogA
Read-onlyIdempotent
Inspect

Inspect all connector definitions and exact installed setup paths. Unavailable blueprints and live verification are separate facts.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
connectorsYes
liveVerifiedYes
registryVersionYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so safety is covered. The description adds useful scope context that the catalog treats unavailable blueprints and live verification as distinct facts, but 'separate facts' is ambiguous about what the output actually contains, and no auth or freshness caveats are given.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with the core action front-loaded and no filler. The second sentence is dense but somewhat cryptic, which slightly weakens its value relative to its terseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complexity is low (no parameters, output schema present, safety covered by annotations), so the description need not explain return values. It covers purpose and scope limits adequately, though it stops short of routing the agent among the numerous sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline of 4 applies. The description appropriately adds no parameter detail because there is nothing to parameterize.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Inspect all connector definitions and exact installed setup paths.' It scopes the tool against related concepts ('live verification are separate facts'), signaling it is a catalog view rather than a health/liveness check. It does not, however, name the sibling (e.g. connector.health, connections.list) that handles those separate concerns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The clause 'live verification [is a] separate fact' implies this tool is not for live checks, and 'unavailable blueprints' implies a boundary, but no explicit when-to-use / when-not-to-use or named alternative is given. Usage is only inferable from the many siblings (connections.list, connections.graph, connector.health).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections.graphA
Read-onlyIdempotent
Inspect

Read canonical connection references and currently visible exact resource revisions. No source access grant or execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesStable graph UUID returned by connections.backfill, not a credential ID.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYes
nameYes
sourceYes
driverIdYes
revisionYes
endpointsYes
revokedAtYes
configurationYes
authorityEpochYes
lifecycleStateYes
selectionEpochYes
dispatchEnabledYes
selectionVisibilityYes
selectedResourceRevisionIdsYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered structurally. The description adds useful side-effect context — that no source access grant is issued and no execution is triggered — and 'currently visible' implies permission-filtered results, but it says nothing about pagination, limits, or result shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with zero filler, and the read scope is front-loaded before the boundary clause. Every clause carries meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and annotations plus a fully documented parameter cover the mechanics. The remaining gap is routing: nothing tells an agent why to pick this over connections.list or baseline.lineage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: the single 'id' parameter is already documented as a stable graph UUID returned by connections.backfill rather than a credential ID. The description adds no syntax, format, or sourcing detail beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Read') and concrete resources ('canonical connection references', 'currently visible exact resource revisions'), so an agent can tell this is a graph-read operation rather than a listing or lineage call. It does not, however, distinguish itself from nearby siblings such as connections.list, connections.catalog, or baseline.lineage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the verb 'Read' plus the closing boundary 'No source access grant or execution', which signals a non-mutating inspection context. There is no explicit when-to-use statement, no prerequisites, and no named alternative for cases where the agent needs listing, catalog, or lineage instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections.listB
Read-onlyIdempotent
Inspect

List current workspace connection metadata, with source restrictions, freshness and bounded continuation. Does not reveal secrets or grant source access.

ParametersJSON Schema
NameRequiredDescriptionDefault
qNoCase-insensitive connection name or service search; up to 200 characters.
limitNoMaximum connections in one page; defaults to 25, capped at 100.
cursorNoOpaque nextCursor from this caller and inventory revision; refresh if the inventory changes.
sourceNoFilter by canonical store; direct metadata still requires admin role and scope.

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemsYes
hasMoreYes
sourcesYes
completeYes
nextCursorYes
observedAtYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds useful context: bounded continuation, freshness, source restrictions, and explicitly that secrets are not exposed and source access is not granted. It still omits pagination expiry and admin-role requirements implied by the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with the purpose front-loaded and the security boundary last. The middle phrase "source restrictions, freshness and bounded continuation" is dense and abstract but not padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return values need not be explained, and annotations cover the safety profile. The description covers listing scope, continuation bounds and the security boundary, leaving only role/scope prerequisites implied rather than stated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents q, limit, cursor and source, including enum values and defaults. The description alludes to source filtering and bounded continuation but adds no syntax or format detail beyond the schema; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource: "List current workspace connection metadata". This clearly separates it from siblings like connections.graph, connections.roots and connector.health, though it does not explicitly distinguish itself from the close sibling connections.catalog.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or when-not-to-use guidance, and no alternatives named despite several connection-related siblings (connections.catalog, connections.roots). The clause "Does not reveal secrets or grant source access" hints at scope but is not a usage condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connections.rootsA
Read-onlyIdempotent
Inspect

Page independent configured account roots using current administrator authority. Does not list secrets or imply verified connector availability.

ParametersJSON Schema
NameRequiredDescriptionDefault
qNoLiteral case-insensitive root name/driver search; wildcard characters have no special meaning.
limitNoBounded page size; defaults to 25 and is capped at 100.
cursorNoOpaque nextCursor for the same caller, workspace, search, limit and observed metadata revision; refresh after changes.

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemsYes
hasMoreYes
nextCursorYes
dispatchEnabledYes

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly, idempotent, non-destructive and closed-world, so the bar is lower. The description still adds real context: an administrator-authority requirement, a data-scope caveat ('does not list secrets'), and a result-interpretation caveat ('does not imply verified connector availability') — useful behavioral framing beyond the structured hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the operation and scoped by the two caveats. No filler and nothing restated from the name or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and the description covers auth requirements and result caveats. It leaves pagination/ordering semantics to the schema and offers no sibling routing, so it is strong but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with q, limit and cursor each fully documented in the schema, so the baseline is 3. The description adds nothing about parameter syntax, filtering behavior, or cursor semantics beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Page') and resource ('independent configured account roots') plus the authority context ('current administrator authority'). It does not explicitly name a sibling such as connections.list or connections.catalog to disambiguate, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description never says when to choose this over siblings like connections.list or connections.catalog, nor does it state any exclusion or precondition beyond 'administrator authority'. Usage is only faintly implied by the resource being paged.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

connector.healthB
Read-onlyIdempotent
Inspect

Show the latest connector-readiness reports for the organization.

ParametersJSON Schema
NameRequiredDescriptionDefault
connectorKeyNoExact report key or metadata connector key to filter the latest 50 persisted reports; omit for all reports in that snapshot. Requires admin:org.

Output Schema

ParametersJSON Schema
NameRequiredDescription
reportsYes
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
configuredYesWhether this query found persisted readiness reports; not a live credential or provider-health check.
explanationYesHuman-readable interpretation and evidence limitations.
connectorKeyYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is fully covered elsewhere. The description adds only the framing that results are 'latest' and org-scoped, without explaining snapshot semantics, the 50-report cap, or the admin:org requirement (all of which appear in the schema instead).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the core resource and recency scope are the first things the agent reads.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema, rich annotations, and a single well-documented optional parameter, the description is nearly sufficient. What is missing is positioning relative to the sibling connection tools and a hint at what 'readiness' means in the report payload.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so connectorKey, its filtering behavior, the 50-report limit, and the admin:org requirement are already documented in the schema. The description adds no parameter meaning beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (show), a specific resource (connector-readiness reports), and scope (latest, organization-scoped). It does not, however, distinguish itself from siblings like connections.list or connections.catalog, which an agent could plausibly consider for connector-related lookups.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description offers no when-to-use or when-not-to-use guidance and names no alternatives among the many connections.* and project.* siblings. The only conditional language ('omit for all reports') lives in the schema, not the description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

docs.searchB
Read-onlyIdempotent
Inspect

Search the published OpenAPI operation catalog for relevant help.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum matching operations, from 1 to 20; defaults to 10.
queryYesSearch terms for the public OpenAPI operation catalog (1–200 characters); all whitespace-separated terms must match.

Output Schema

ParametersJSON Schema
NameRequiredDescription
queryYes
resultsYes
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
explanationYesHuman-readable interpretation and evidence limitations.

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and a closed-world scope, so the safety profile is fully covered. The description adds the useful scope detail that only the *published* catalog is searched, but says nothing about result freshness, ranking behavior, or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence with no filler, and the search target is front-loaded. It is efficient, though it is arguably terse enough that a little more substance would have helped without harming readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is not required, and both parameters are well documented. For a simple closed-world read tool this is close to sufficient, but with zero usage guidance the definition is only minimally viable rather than complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both 'query' (term matching, all whitespace-separated terms must match) and 'limit' (1–20, default 10) are already fully documented. The description adds no additional parameter meaning, which is the expected baseline when the schema carries the burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource: 'Search the published OpenAPI operation catalog.' An agent knows exactly what corpus is queried and that the goal is finding relevant operations/help. It falls short of a 5 only because it does no sibling differentiation, though the listed siblings (connections.*, runs.*, project.*) are unrelated namespaces so the risk of confusion is low.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance and no named alternative. The agent must infer from the resource name alone that this tool is for documentation lookup rather than, say, connector health or run inspection. No prerequisites or context for invocation are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evaluations.listB
Read-onlyIdempotent
Inspect

List evaluations with a stable cursor for the organization.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum results in this page, from 1 to 50; defaults to 20.
cursorNoOpaque nextCursor from the preceding page of this tool. Omit for the first page.
statusNoRestrict evaluations to this lifecycle status; omit to include all statuses.

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds one genuinely useful trait beyond them: the cursor is described as "stable," telling the agent that pagination ordering will not shift between pages. It gives no information on ordering key, page size behavior, or what an empty result implies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the resource and the pagination trait come first. It is arguably under-specified rather than over-long, but nothing in it is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema exists and the annotations plus full schema coverage carry the parameter and safety burden, so the description need not explain return shape. It is still thin for a tool with three sibling list operations, since it never says what an evaluation is or how this list differs from runs.list.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with limit bounds, enum values for status, and cursor provenance all documented in the schema itself. The description only echoes cursor and adds no format, ordering, or interaction detail beyond it, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ("List") and resource ("evaluations") and scopes it to the organization, so an agent knows exactly what it returns. It does not differentiate from the sibling runs.list, which is also a paginated list operation, leaving the reader to infer that the resources differ only by name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this rather than runs.list or docs.search, no prerequisite/auth conditions, and no mention of the status filter as a selection mechanism. The only usable guidance is the implicit "listing" semantics of the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

project.inspectB
Read-onlyIdempotent
Inspect

Inspect the evaluations and project keys available in the authenticated organization.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyNoExact project key to filter the first 50 organization evaluations; omit to inspect that entire snapshot.

Output Schema

ParametersJSON Schema
NameRequiredDescription
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
projectKeyYes
evaluationsYes
explanationYesHuman-readable interpretation and evidence limitations.
organizationIdYes
evaluationCountYesCount in this bounded snapshot, not a total across the organization.

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds the useful constraint that the tool operates on the authenticated organization, which is not in the annotations. It says nothing about result shape or limits beyond what the schema notes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence with no filler. Brief, but the brevity reflects thin content rather than wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need no explanation, and the single optional parameter is fully described in the schema. The description supplies the scope and auth context, which is about the right amount for a low-parameter, well-annotated read tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the single projectKey parameter is fully documented in the schema, including the "first 50 evaluations" filter behavior. The description adds no parameter detail beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a verb ("inspect") and the resources (evaluations and project keys) within an organizational scope. However, "inspect" is vague about what actually happens, and the description does not differentiate this tool from the sibling evaluations.list, which likely overlaps heavily. Adequate but with clear ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance and no mention of alternatives such as evaluations.list or project.plan, despite those siblings existing. The agent must infer which tool to pick from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

project.planA
Read-onlyIdempotent
Inspect

Return a non-mutating next-step plan for connecting a repository, evaluation, or trace source.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetNoWorkflow to plan: local repository setup, hosted evaluation review, or trace ingestion. Defaults to repository; never executes the plan.repository

Output Schema

ParametersJSON Schema
NameRequiredDescription
stepsYes
targetYes
mutatesYes
deepLinkYesEvalGate web path or absolute URL for inspecting these results.
explanationYesHuman-readable interpretation and evidence limitations.

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the 'non-mutating' wording largely restates structured data. The description does confirm the tool produces a plan rather than performing the connection, which adds a little context, but nothing about plan content, freshness, or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler. It is efficient, though arguably so terse that it leaves behavioral questions unanswered for a planning tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and annotations cover the safety profile. For a one-optional-parameter tool with 100% schema coverage, the description is adequate, though it omits any indication of sibling selection or plan scope.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single enum parameter is fully documented in the schema, including the note that the plan is never executed. The description merely echoes the three target workflows, adding no syntax or behavioral detail beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (return) and resource (next-step plan) and scopes it to three workflows: repository, evaluation, trace. However, it does not distinguish itself from the sibling project.inspect, which an agent might reasonably confuse with planning versus inspecting.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'next-step plan' implies the usage situation (you want guidance rather than execution), but there is no explicit when-to-use or when-not-to-use guidance and no mention of project.inspect or other siblings. Usage must be inferred from the purpose statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run.explainB
Read-onlyIdempotent
Inspect

Explain a run using its status, failed-case metadata, and evidence-lineage summary.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesRun ID returned by runs.list; must belong to the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 3 or "3").

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world behavior. The description adds no behavioral context beyond annotations—no auth requirements, rate limits, or note on whether the run must be complete. It only describes the informational components of the explanation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single front-loaded sentence with no filler; every word contributes to the purpose. Appropriately sized for a simple read-only tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only tool with a full output schema and comprehensive annotations, the description gives enough to call it correctly. It does not distinguish from sibling baseline.lineage or mention runs.list as the runId source, but those gaps are minor and partly covered by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%; runId is fully documented in the schema, including format and source (runs.list). The description adds no parameter-level detail, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the specific verb 'Explain' and resource 'run', and identifies the informational components used: status, failed-case metadata, and evidence-lineage summary. It does not explicitly name sibling alternatives like runs.list or baseline.lineage, so it is clear but not differentiated from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no explicit when-to-use, when-not-to-use, or alternative guidance. The only implied context is that it is for understanding a run, which essentially restates the purpose. Alternatives such as runs.list are not mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runs.listB
Read-onlyIdempotent
Inspect

List evaluation runs with stable cursors and regression signals.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum results in this page, from 1 to 50; defaults to 20.
cursorNoOpaque nextCursor from the preceding page of this tool. Omit for the first page.
statusNoExact persisted run status to filter by; omit to include all statuses, including incomplete runs.
evaluationIdYesEvaluation ID returned by evaluations.list in the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 8 or "8").

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is fully covered. The description adds genuine context beyond them — cursor stability across pages and the presence of regression signals in results — but says nothing about rate limits, auth scope, or ordering.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no wasted words and the resource named immediately. It is efficient, though arguably too terse to carry usage context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return-value detail is unnecessary, and all four parameters plus the safety profile are covered by structured fields. What is missing is routing guidance relative to evaluations.list and run.explain, which a one-line description does not supply.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with limit, cursor, status and evaluationId each fully documented in the schema, so the baseline is 3. 'Stable cursors' hints at cursor semantics but adds no format or usage detail beyond the schema's 'Opaque nextCursor' text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a clear verb+resource ('List evaluation runs') and adds two distinguishing traits (stable cursors, regression signals) that separate it from evaluations.list. It stops short of explicitly naming the sibling it differs from, so an agent still has to infer that evaluations.list returns evaluations rather than runs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use guidance and no alternative is named. The mention of cursors only implies pagination mechanics, not the conditions under which this tool should be chosen over evaluations.list or run.explain.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 12 tool updates
    • First observedbaseline.lineage
    • First observedconnections.catalog
    • First observedconnections.graph
    • First observedconnections.list
    • First observedconnections.roots
    • First observedconnector.health
    • First observeddocs.search
    • First observedevaluations.list
    • First observedproject.inspect
    • First observedproject.plan
    • First observedrun.explain
    • First observedruns.list

Related MCP Connectors

Related MCP Servers

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources