Skip to main content
Glama
Sign In
Crawlora-org

Crawlora MCP

Official
by Crawlora-org
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

datasets_llms_item

Retrieve a site's AI-readiness record from the llms.txt dataset by domain, including stored /llms.txt and /llms-full.txt bodies, classification, and robots.txt stance toward tracked AI crawlers.

Instructions

Get a site from the AI-readiness (llms.txt) dataset. Returns one site by its domain from dataset id enum value llms, including the stored /llms.txt and /llms-full.txt bodies (llms.body, full.body; body_truncated marks a cut file, and a file flagged as an instruction override has its title, summary, and body withheld with body_withheld set to injection), their structure and classification, and the /robots.txt stance toward each tracked AI crawler. Returns 404 when the domain is not in the dataset or its owner asked for removal (privacy@crawlora.net).

Input Schema

NameRequiredDescriptionDefault
domainYesSite domain, e.g. shopify.com (a scheme and www. are stripped)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.17.12

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does unusually well: it documents field names, the body_truncated cut marker, the injection-withholding behavior, the robots.txt crawler stance, and the two 404 conditions. It omits auth requirements and any rate-limit or pagination behavior, so it is strong but not complete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded in the first sentence, and the dense parentheticals each carry distinct payload (truncation, injection withholding). It is lengthy but every clause informs behavior; the run-on nesting is the only structural weakness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must explain returns — and it does, enumerating llms/full bodies, truncation and withholding markers, structure/classification, and robots.txt stance. An agent has enough to call it and interpret results; only auth/limits are unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the domain parameter already documents the example format and the scheme/www stripping rule. The description only reiterates that lookup is by domain, adding little beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource — 'Get a site from the AI-readiness (llms.txt) dataset' — and clarifies it returns 'one site by its domain.' This distinguishes the item lookup from the datasets_llms_search sibling, though it never names that alternative explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the domain-keyed single-site retrieval suggests this is a detail fetch, and the 404 note tells the agent what happens on a miss. There is no explicit 'use datasets_llms_search to find domains first' routing or any when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools