Skip to main content
Glama

Web Search API Eval

Server Details

Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.

If you are the author of this connector, you can claim ownership by verifying the domain or GitHub account it belongs to. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
Repository
psinglap/websearch-api-eval
GitHub Stars
0

TDQS

A3.8/5.0

Scored across 11 tools

Disambiguation3/5

Several tools overlap in purpose: get_overall_results and get_scorecard('overall') cover much of the same ground, while compare_providers, recommend_provider, and get_overall_results all surface 'which API fits which agent' guidance. The descriptions add nuance, but an agent could reasonably misselect among these comparison/summary tools.

Naming Consistency4/5

Most tools follow a clear verb_noun pattern (get_charts, get_methodology, get_question, list_questions, compare_providers, recommend_provider). A few use bare verbs or phrases (search, fetch, how_to_run_locally), a minor deviation but overall readable and predictable.

Tool Count5/5

11 tools is well within the ideal range and each appears to earn its place covering discovery, detailed data, comparison, and methodology. Nothing feels redundant enough to warrant removal at the count level.

Completeness4/5

The read-only benchmark domain is well covered: overview, methodology, question listing/retrieval, per-use-case and overall scorecards, charts, provider comparison/recommendation, and local rerun guidance. Minor gaps exist (e.g. no direct metric-listing or cross-use-case aggregation tool beyond overall), but agents can work around these via search/fetch.

Available Tools

11 tools
compare_providersCompare providersA
Read-onlyIdempotent
Inspect

Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').

NameRequiredDescriptionDefault
use_caseNooverall, gtm, insurance or financeoverall
providersYesTwo or more of: You.com, Tavily, Exa, Parallel
metric_filterNoOptional keyword to keep only matching metrics

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world, and non-destructive, so the safety profile is fully covered. The description adds some value by clarifying that metric_filter is a case-style keyword substring match and by naming typical keywords ('latency', 'accuracy', 'fresh', 'cost', 'empty'), which hints at the metric vocabulary, but it says nothing about what the comparison returns or how a metric missing for one provider is handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that progressively adds scope (providers), axis (overall/use case), and optional refinement (metric filter) with no filler. Every clause carries new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is small (three params, one enum) and there is no output schema, so the description should compensate by indicating what the comparison yields — a metrics table, per-provider rows, scores — but it never does. For a comparison tool, the shape of the output is exactly what an agent needs to plan the call correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description earns an extra point by giving concrete keyword examples for metric_filter, which the schema only calls 'Optional keyword to keep only matching metrics'. It also reinforces the minimum of two providers and the four valid provider names from the enums.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (compare) and resource (the four named search providers) and states the two axes of comparison (overall or per use case) plus the optional filter. An agent can distinguish it from get_overall_results or get_scorecard, which return data for a fixed provider rather than a side-by-side comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by describing the comparison modes, so an agent can infer it should call this when it wants a cross-provider comparison rather than a single scorecard. However, it never explicitly says when to prefer it over siblings like recommend_provider or get_scorecard, nor does it state prerequisites such as needing at least two providers.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetchFetch a benchmark documentA
Read-onlyIdempotent
Inspect

Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).

NameRequiredDescriptionDefault
idYesDocument ID from search

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is fully covered and the description does not need to repeat it. The description's one genuine behavioral contribution is 'full text,' implying this returns complete untruncated content rather than a snippet, but it says nothing about what an invalid or unknown ID yields or how large documents are handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, zero filler, with the resource scope front-loaded and the illustrative IDs parenthesized rather than padding out a separate sentence. Nothing could be removed without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only getter with no output schema and full annotation coverage, the description supplies what an agent needs to call it: what it returns ('full text'), by what key, and valid key shapes. The remaining gap is the lack of any statement about error behavior for bad IDs or the relationship to sibling getters, which matters given four overlapping siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter's own description ('Document ID from search') already explains provenance, so the baseline would be 3. The description earns an extra point by enumerating concrete ID shapes (question-F5, scorecard-finance, overall, methodology), which teaches the agent the ID grammar and origin that the schema never spells out.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource — returns the full text of exactly one benchmark document, identified by ID — and the parenthetical examples (question-F5, scorecard-finance, overall, methodology) make the resource concrete. It does not, however, distinguish this generic fetch from the specialized siblings get_question, get_scorecard, get_overall_results, and get_methodology, whose names suggest the same content, so an agent cannot tell from the description alone which to prefer.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: 'Document ID from search' hints at the search-then-fetch workflow, and the single-document scope implies 'use once you know the ID.' There is no explicit statement of when to use this versus the per-document-type siblings, nor any exclusion or condition that would make the routing decision deterministic.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_chartsChartsA
Read-onlyIdempotent
Inspect

The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.

NameRequiredDescriptionDefault
chart_idNoe.g. overall_scorecard, finance_key_metrics, gtm_query_grid, time_steps_activation
use_caseNoFilter the listall

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent and non-destructive, so the safety profile is covered. The description adds value beyond that by disclosing what is returned in each mode (titles and image links vs. the image itself), which an agent needs to plan the call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loaded with the resource and then the mode switch. Dense but every clause carries information; no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description has to convey return shape, and it does: list mode yields titles and image links, single mode yields the image. For a two-parameter read-only tool this is largely sufficient, with only the sibling-relationship guidance missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the chart_id examples are already in the schema, so the baseline is 3. The description goes further by explaining the behavioral consequence of supplying versus omitting chart_id, which the schema does not state, though it says nothing about how use_case interacts with chart_id selection.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource ('the published graphs') and enumerates the content families (overall scorecard, key metrics, per-query grids for GTM, insurance, finance, developer activation). This is clear, but the 'overall scorecard' / 'key metrics' phrasing overlaps conceptually with siblings like get_scorecard and get_overall_results without saying how they differ.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It does explain the two operating modes implicitly: no chart_id lists charts, with chart_id returns the image. That is usage-relevant, but there is no explicit guidance on when to pick this over get_scorecard or get_overall_results, which are the obvious alternatives for overlapping content.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_methodologyMethodologyA
Read-onlyIdempotent
Inspect

How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the safety profile is covered. The description adds real behavioral value by disclosing the document's scope, including that it covers grading standards and stated limits, which tells the agent what answers it can obtain here.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence fragment with a dense, scannable content list and no filler. Every clause names a distinct topic the document covers.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-argument static reference tool with full annotations and no output schema, the description is essentially complete. The only minor omission is noting the return form (e.g., document text), which is easily inferred.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has zero properties, so there is nothing to document and the baseline of 4 applies. The description correctly implies a parameterless, fetch-the-whole-document call.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource (benchmark methodology) and enumerates exactly what it covers: query design, run count, latency timing, SimpleQA/FreshQA grading, metrics, and limits. That distinguishes it from data-returning siblings like get_scorecard or get_overall_results, though it never names those siblings explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the content inventory — an agent can infer it should call this to understand how scores were produced or what a metric means. It never says when to prefer this over how_to_run_locally or the scorecard tools, nor any exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_overall_resultsOverall resultsA
Read-onlyIdempotent
Inspect

Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world, non-destructive behavior, so the bar is low. The description still adds useful context: this is a no-input aggregate over a fixed benchmark scope of 30 queries and 90 calls per API, and it covers a per-use-case summary and developer activation time, which is more than the annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with 'Headline findings and the overall scorecard' before the metric list, and every clause contributes concrete returned content. The metric enumeration is long but earns its place by telling the agent what is available without an output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema available, the description carries the burden of describing return content and does so thoroughly, covering metrics, per-use-case summary, provider fit, and activation time. The one gap is that it never clarifies how this aggregate differs from the sibling get_scorecard.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, which is the baseline-4 case per the rubric. The description correctly implies no filtering or scoping inputs exist, consistent with the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific deliverable (headline findings plus an overall scorecard) and enumerates the exact metrics returned, so an agent knows precisely what data comes back. It does not distinguish itself from the overlapping sibling get_scorecard or from get_charts, leaving some ambiguity about which aggregate view to call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: words like 'overall', 'across all 30 queries', and 'headline findings' suggest this is the top-level summary entry point. There is no explicit when-to-use, no when-not, and no reference to the near-synonym sibling get_scorecard.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_questionGet a questionA
Read-onlyIdempotent
Inspect

One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).

NameRequiredDescriptionDefault
idYesQuestion ID, e.g. G3, I4 or F5

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false, and openWorldHint=false, so safety behavior is covered. The description adds valuable return-content context beyond annotations: it lists the fields returned and explains the accuracy score scale (0-2, mean of 3 runs).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is compact and front-loads the key fact: one benchmark question by ID. The remainder is a dense but useful enumeration of return content. It is slightly fragmentary and list-heavy, but no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description appropriately explains the returned content in detail. For a one-parameter retrieval tool with safety annotations, it is nearly complete; only minor behavior such as invalid-ID handling is unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the single id parameter is documented with the same examples used in the description. The description repeats the ID examples but adds no new constraint or format detail beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies a retrieval of one benchmark question by ID and enumerates what the response includes (task, search call, parameters, answer key, API returns, accuracy score). It distinguishes itself implicitly from list_questions by emphasizing a single question by ID, though it does not name the sibling or explicitly contrast them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: an agent should call this when it needs a single benchmark question identified by ID. However, the description gives no explicit when-to-use guidance, exclusions, or comparison to alternatives such as list_questions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_scorecardScorecardA
Read-onlyIdempotent
Inspect

Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.

NameRequiredDescriptionDefault
use_caseYesoverall, gtm, insurance or finance

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and non-destructive, so safety is covered. The description adds real value beyond that by enumerating what the scorecard contains (latency percentiles, accuracy, freshness, tokens, cost), which compensates for the absent output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence, front-loaded with the resource and scope, followed by a compact metric list. The metric enumeration is dense but earns its place given there is no output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-param, non-destructive read tool with full schema coverage and annotations, the description supplies the one thing structured data lacks: what metrics come back. Nothing critical to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with an enum that already documents the four valid values, so the description's repetition of 'gtm, insurance, finance or overall' adds no new syntax or format detail. Baseline 3 is appropriate when the schema does the heavy lifting on a single enum param.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource (full metric scorecard) and scopes it to a single use case with the enumerated values. It distinguishes itself implicitly from siblings like get_overall_results and compare_providers by being the per-use-case metric dump, though it never names those alternatives directly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the mention of 'one use case (gtm, insurance, finance) or overall' signals the selection axis, but there is no explicit when-to-use guidance or pointer to compare_providers/get_overall_results when a different view is wanted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

how_to_run_locallyRun it yourselfA
Read-onlyIdempotent
Inspect

How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds genuinely useful context beyond that: calling this tool executes nothing on the server — it only surfaces instructions. That prevents an agent from expecting an actual benchmark run.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core purpose and finished with the key caveat. No filler, nothing repeated from the annotations or title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a trivial zero-parameter informational tool with no output schema, the description covers what it does and confirms nothing executes server-side. It could say slightly more about the form of the response (e.g., setup steps vs. a link), but nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so per the baseline this scores 4. The description correctly implies no arguments are needed — you get generic instructions applicable to whatever queries the user supplies locally.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action — returning instructions for rerunning the benchmark locally with your own API keys and queries. That is clearly distinct from siblings like get_methodology or get_scorecard, though it never names an alternative directly. An agent can tell this is a documentation/instruction tool rather than a data tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Nothing runs on this server' implies this is the tool to call when the user wants to reproduce the benchmark themselves, and 'including on your own queries' widens the use case. However, there is no explicit guidance on when to prefer this over get_methodology, nor any stated prerequisites (e.g., API keys required for the local run).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_questionsList questionsA
Read-onlyIdempotent
Inspect

The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.

NameRequiredDescriptionDefault
use_caseNoall, gtm, insurance or financeall

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent and closed-world, so the safety profile is covered. The description usefully discloses the fields each entry returns, but says nothing about ordering, pagination or whether the list is static.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence that front-loads the count and group breakdown. Efficient, though the phrasing 'ID, what the agent is asked, the search call and freshness' is slightly telegraphic.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden and does so by naming the per-question fields. For a one-parameter read-only listing tool this is largely complete, with only ordering/filtering behavior left unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single use_case enum is fully documented in the schema. The description's mention of GTM/insurance/finance groups loosely mirrors the enum values but adds no filtering semantics beyond it, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names the specific resource (30 benchmark questions) and enumerates the ID prefixes per use case plus what each entry carries (ID, prompt, search call, freshness). It is clearly the bulk-list counterpart to get_question, though it never names that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The contents imply it's the way to browse the full question set while get_question presumably fetches one, but the description never states when to use this vs get_question or that use_case can narrow the list. Usage is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

recommend_providerRecommend a providerA
Read-onlyIdempotent
Inspect

Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.

NameRequiredDescriptionDefault
agent_needNoWhat the agent does, e.g. 'voice agent answering in under a second' or 'finance research'

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is covered. The description adds genuine value beyond them by stating the response contains 'the guidance with the supporting numbers' — the only signal about return shape, since no output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause and the second sentence covers the return. The category enumeration is dense but functional; nothing is padding, though the list could be trimmed without loss.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-optional-parameter read-only lookup with full annotation coverage, the description supplies purpose, triggering intents, and a hint at the response payload. Only the default behavior when no agent_need is provided is left unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is optional with its own examples, so the schema carries the semantics. The description's category list roughly mirrors the parameter's example values but adds no format, granularity, or default-behavior detail (e.g., what happens when agent_need is omitted).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific question the tool answers ('Which web search API fits an agent') grounded in a named benchmark, and enumerates the query intents it covers. It implicitly distinguishes itself from compare_providers by framing the output as a recommendation rather than a comparison, though it never names that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The enumerated intent list (voice/chat turns, deep research, semantic/long-tail queries, docs lookups, domain use cases) gives concrete conditions for when this tool is the right pick. It stops short of saying when to use compare_providers instead, so the boundary between the two is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 11 tool updates
    • First observedcompare_providers
    • First observedfetch
    • First observedget_charts
    • First observedget_methodology
    • First observedget_overall_results
    • First observedget_question
    • First observedget_scorecard
    • First observedhow_to_run_locally
    • First observedlist_questions
    • First observedrecommend_provider
    • First observedsearch

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    A
    maintenance
    Evidence-backed web research for AI agents. Real-time search with cited claims, confidence scores, and compare mode showing raw LLM hallucination vs evidence-backed answers.
    5
    20
    Apache 2.0
  • A
    license
    A
    quality
    F
    maintenance
    Web search for AI agents across 6 engines (Serper, Brave, Exa, Tavily, Firecrawl, Perplexity) through one search tool. Routes each query to the cheapest engine that clears a quality bar and caches repeats. Hosted, streamable-HTTP, BYOK supported.
    1
    1
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Universal Search-First Knowledge Acquisition Plugin for LLMs. Enables real-time web search and deep page browsing via MCP or CLI. Zero-cost, privacy-first, supports DuckDuckGo, Bing, Google, Brave, Wikipedia, Arxiv, YouTube, Reddit and more.
    2
    8 npm
    27 PyPI
    19
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.