Skip to main content
Glama

Web Search API Eval

Compare providers

compare_providers
Read-onlyIdempotent

Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').

Input Schema

NameRequiredDescriptionDefault
use_caseNooverall, gtm, insurance or financeoverall
providersYesTwo or more of: You.com, Tavily, Exa, Parallel
metric_filterNoOptional keyword to keep only matching metrics

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world, and non-destructive, so the safety profile is fully covered. The description adds some value by clarifying that metric_filter is a case-style keyword substring match and by naming typical keywords ('latency', 'accuracy', 'fresh', 'cost', 'empty'), which hints at the metric vocabulary, but it says nothing about what the comparison returns or how a metric missing for one provider is handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that progressively adds scope (providers), axis (overall/use case), and optional refinement (metric filter) with no filler. Every clause carries new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is small (three params, one enum) and there is no output schema, so the description should compensate by indicating what the comparison yields — a metrics table, per-provider rows, scores — but it never does. For a comparison tool, the shape of the output is exactly what an agent needs to plan the call correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description earns an extra point by giving concrete keyword examples for metric_filter, which the schema only calls 'Optional keyword to keep only matching metrics'. It also reinforces the minimum of two providers and the four valid provider names from the enums.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (compare) and resource (the four named search providers) and states the two axes of comparison (overall or per use case) plus the optional filter. An agent can distinguish it from get_overall_results or get_scorecard, which return data for a fixed provider rather than a side-by-side comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by describing the comparison modes, so an agent can infer it should call this when it wants a cross-provider comparison rather than a single scorecard. However, it never explicitly says when to prefer it over siblings like recommend_provider or get_scorecard, nor does it state prerequisites such as needing at least two providers.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.