Web Search API Eval
Server Details
Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- psinglap/websearch-api-eval
- GitHub Stars
- 0
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 11 tools
Several tools overlap in purpose: get_overall_results and get_scorecard('overall') cover much of the same ground, while compare_providers, recommend_provider, and get_overall_results all surface 'which API fits which agent' guidance. The descriptions add nuance, but an agent could reasonably misselect among these comparison/summary tools.
Most tools follow a clear verb_noun pattern (get_charts, get_methodology, get_question, list_questions, compare_providers, recommend_provider). A few use bare verbs or phrases (search, fetch, how_to_run_locally), a minor deviation but overall readable and predictable.
11 tools is well within the ideal range and each appears to earn its place covering discovery, detailed data, comparison, and methodology. Nothing feels redundant enough to warrant removal at the count level.
The read-only benchmark domain is well covered: overview, methodology, question listing/retrieval, per-use-case and overall scorecards, charts, provider comparison/recommendation, and local rerun guidance. Minor gaps exist (e.g. no direct metric-listing or cross-use-case aggregation tool beyond overall), but agents can work around these via search/fetch.
Available Tools
11 toolscompare_providersCompare providersARead-onlyIdempotentInspect
Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').
Inputs: providers (required), use_case, metric_filter.
fetchFetch a benchmark documentARead-onlyIdempotentInspect
Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).
Inputs: id (required).
get_chartsChartsARead-onlyIdempotentInspect
The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.
Inputs: chart_id, use_case.
get_methodologyMethodologyARead-onlyIdempotentInspect
How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.
Inputs: no named parameters.
get_overall_resultsOverall resultsARead-onlyIdempotentInspect
Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.
Inputs: no named parameters.
get_questionGet a questionARead-onlyIdempotentInspect
One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).
Inputs: id (required).
get_scorecardScorecardARead-onlyIdempotentInspect
Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.
Inputs: use_case (required).
how_to_run_locallyRun it yourselfARead-onlyIdempotentInspect
How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.
Inputs: no named parameters.
list_questionsList questionsARead-onlyIdempotentInspect
The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.
Inputs: use_case.
recommend_providerRecommend a providerARead-onlyIdempotentInspect
Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.
Inputs: agent_need.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Web search for AI agents: one query across 10 providers, deduplicated, licensed results flagged.
Web search for AI agents — one tool across 6 engines, routed to the cheapest + cached.
The best web search for your AI Agent
AI-agent web search, answer-ready content. Beats Tavily: 60.7% of quality duels, 20.2% fewer tokens.
Related MCP Servers
- AlicenseAqualityBmaintenanceThe First Universal Search Gateway for AI Agents — Google (Serper), Tavily, Brave, Exa, and Firecrawl with auto-failover, 14 tools, and token-smart verbosity.15667 npm1Apache 2.0
- AlicenseAqualityBmaintenanceEvidence-backed web research for AI agents. Real-time search with cited claims, confidence scores, and compare mode showing raw LLM hallucination vs evidence-backed answers.520Apache 2.0
- AlicenseNot gradedqualityDmaintenanceWeb search, clean page reading & one-call research dossiers for AI agents. No API key — your agent does the synthesis.81 npmMIT
- AlicenseAqualityDmaintenanceWeb search for AI agents across 6 engines (Serper, Brave, Exa, Tavily, Firecrawl, Perplexity) through one search tool. Routes each query to the cheapest engine that clears a quality bar and caches repeats. Hosted, streamable-HTTP, BYOK supported.11MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.