agentery

create_custom_benchmark

Create a PRIVATE custom benchmark (a saved, calculated peer cohort) over Agentery's data — no account needed. Two modes: (A) explicit members: pass members (a list of exact handles; product names/domains resolve where unambiguous). (B) fork a market: pass base_niche (its slug) plus optional remove/add. Returns a one-time secret benchmark_id (cb_… token) — store it; it's your only key. Use it later in get/update/delete and in market_report/get_price_index/get_price_index_history. Ambiguous names are returned as candidates, never silently resolved; unresolved inputs block creation unless allow_partial:true. All prices/history are computed from Agentery's immutable observations; canonical market data is never changed.

Input Schema

TableJSON Schema

Name	Required	Description
`add`	No	members to add
`name`	No	Optional private label
`remove`	No	Mode B: members to drop from the forked market
`members`	No	Mode A: exact handles (preferred), product names or domains
`base_niche`	No	Mode B: slug of the canonical market to fork
`allow_partial`	No	Create with only the resolved members when some inputs don't resolve (default false)

TDQS

A5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses critical behaviors: returns a one-time secret ID, ambiguous names are not silently resolved but returned as candidates, unresolved inputs block unless allow_partial is true, and that all prices/history are computed from immutable observations (no mutation of canonical data). This is comprehensive given no annotations were provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise yet information-dense, with clear structure using (A), (B), and note sections. It avoids redundancy and every sentence adds value, making it easy to parse despite its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema, the description adequately explains the return value (one-time secret benchmark_id) and how to use it later with get/update/delete and other tools. It also covers edge cases (ambiguity, partial allowance) and ensures the user understands the tool's behavior completely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Each parameter is explained in context: members for mode A, base_niche for mode B, add/remove as optional for forking, allow_partial for partial resolution. The description adds meaning beyond the schema by clarifying the mode-specific usage and the semantics of each parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: creating a private custom benchmark (peer cohort) over Agentery's data, with explicit mention of two modes (A and B). It distinguishes from sibling tools by focusing on creation and using terms like 'PRIVATE' and 'custom', which are unique.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains when to use the tool (to create a custom benchmark) and how to use it, including both modes with specific parameters. It also clarifies the return value and mentions that it can be used later with other tools, providing clear usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All agentery tools Inspect

Try in Browser

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.2/5.0

Disambiguation4/5

Most tools have clearly distinct purposes, but a few pairs could cause confusion: get_provider vs get_provider_profile overlap in scope (one is a card, the other a full profile), and research_capability explicitly bundles find_niche, niche_report, price_benchmark, and search_providers, creating layered redundancy. However, the detailed descriptions clearly differentiate the entry-point tools from the lower-level ones.

Naming Consistency4/5

The majority follow a verb_noun pattern (create_, delete_, get_, search_, suggest_, etc.), but a few tools use bare noun phrases (demand_signals, market_gaps, niche_report, price_benchmark) rather than imperative or get_ prefixes. This is a minor inconsistency that doesn't harm readability.

Tool Count4/5

At 19 tools, the set is slightly above the ideal 3-15 range but each tool serves a distinct, justified function within the market-intelligence and provider-directory domain. The count feels appropriate for the breadth of features (search, research, comparison, pricing, indexes, benchmarks, and feedback).

Completeness5/5

The tool surface is very comprehensive: full CRUD for custom benchmarks, multiple discovery paths (search, research, gaps, demand signals), pricing and index tools, profile/detail tools, comparison and alternatives, plus outcome reporting. There are no obvious dead ends or missing critical operations for the stated purpose.

Resources

Need Help?