Skip to main content
Glama

List and filter crawlers

list_crawlers
Read-onlyIdempotent

The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every valid filter value. Example: category='ai-training', limit=20.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
qNoFree text over slug, name and operator.
limitNoMax rows, 1-200 (default 100).
categoryNoai-training, ai-search, user-fetch, dataset, search, seo, archive, preview or tool.
operatorNoOperator slug or name.
verificationNopublished-ranges, reverse-dns or none.
respects_robots_txtNodocumented, disputed, by-design-no or n-a.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

While annotations already declare read-only, idempotent, and non-destructive behavior, the description adds that each row includes what blocking the crawler costs, giving insight into the output's content and potential risk context. This goes beyond the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief but uses a poetic and slightly opaque opening ('The index, filtered or whole') that could be clearer. It conveys the essentials in two sentences, but the wording is not maximally direct.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list operation, the description covers the core aspects: filtering, output content (crawler identity and cost), and available filter values. It does not mention pagination or sorting, but these are not critical for a simple enumeration tool, so completeness is strong.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully describes all parameters, and the description reinforces that they act as filters (e.g., 'category' example) and that the reply lists valid filter values, aiding understanding of wildcard queries. This adds a small semantic layer beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description indicates the tool lists crawler entries with filtering and includes blocking cost information, which is a clear purpose. It is distinct from sibling tools by focusing on enumeration rather than classification or generation, though the phrasing is somewhat indirect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description offers an example filter and explains that the response lists valid filter values, providing some usage context. However, it does not explicitly contrast with alternatives like classify_user_agent or lookup_crawler, so guidance on when to prefer this tool is implicit rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.3/5.0
Disambiguation5/5

Each tool targets a distinct action: classifying user-agent strings, verifying IP addresses, listing, looking up, generating robots.txt, and checking data freshness. The explicit cross-reference between classify_user_agent and is_verified_crawler_ip clarifies rather than blurs their boundaries.

Naming Consistency4/5

Almost all tool names follow a predictable snake_case verb_noun pattern such as classify_user_agent, list_crawlers, and lookup_crawler. whats_changed deviates slightly from the verb_noun style but remains consistent in lowercase snake_case, so the naming is mostly coherent.

Tool Count5/5

Six tools is a well-scoped set for a crawler index service. Each tool covers a distinct need without overlapping or unnecessary additions.

Completeness5/5

The tool surface covers the full workflow for understanding and managing crawler data: discovery, identification via UA or IP, detail lookup, robots.txt generation, and freshness monitoring. No obvious dead ends or missing operations for the stated purpose.

Resources