Skip to main content
Glama
Sign In
genzagents

citable

by genzagents
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

citable

GitHub Marketplace PyPI

Can ChatGPT, Claude and Perplexity actually read and cite your site?

Most "AI visibility" checkers read robots.txt and stop. citable fetches your homepage as each AI bot and reports the blocks that robots.txt never mentions (Cloudflare "Block AI bots", WAF rules, rate limiters), then checks the rest of the list: llms.txt, JSON-LD, title and description, JS-only content, sitemap, canonical. Score out of 100, every failure with the fix. Also runs as an MCP server so your agent can check sites for you.

Zero dependencies. One file. Python 3.9+.

pipx install citable-cli     # from PyPI; on a Mac: brew install pipx first. Or: uvx --from citable-cli citable
citable example.com                # report
citable example.com --json         # for scripts and CI (exit 1 if below --min-score=N)
citable example.com --llms-txt     # a starter llms.txt built from your homepage
citable watch a.com b.com          # weekly watch: GitHub issue per site, comment (email) on regressions
citable track a.com "best X" "how Y"  # prompt tracking: which engines cite you, who gets cited instead

No install at all:

curl -sL https://raw.githubusercontent.com/genzagents/citable/main/citable/__init__.py | python3 - example.com

What a run looks like

citable 0.3.0: https://reddit.com/
  score 13/100  grade F   (final URL https://www.reddit.com/)

  [FAIL]  0/15 robots.txt allows answer engines: robots.txt blocks: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, ...
  [FAIL]  0/20 Answer engines not blocked at the edge: Fetching the homepage as a bot: ChatGPT-User got HTTP 403; a browser got 200.
         fix: Your CDN or WAF (Cloudflare 'Block AI bots', bot-fight mode, a rate limiter) is refusing them ...
  [FAIL]  0/15 /llms.txt: /llms.txt returns HTTP 404.
  ...
  Bot                 Vendor                 Role    robots.txt   live fetch
  OAI-SearchBot       OpenAI                 answer  block        HTTP 200
  ChatGPT-User        OpenAI                 answer  block        HTTP 403  <- BLOCKED
  Claude-SearchBot    Anthropic              answer  block        HTTP 200
  ...

Related MCP server: Crawl Readiness MCP Server

What it checks

Check

Points

Why it matters

robots.txt allows the answer bots (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, DuckAssistBot, Bingbot, Googlebot)

15

These fetch pages to answer live questions and cite the source. Blocked means never cited.

Not blocked at the edge: homepage fetched as each of those bots returns what a browser gets

20

CDN and WAF "block AI" toggles stop bots that robots.txt allows. Nothing else on this list matters until this passes.

Training crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, Applebot-Extended, meta-externalagent, Amazonbot, Bytespider)

info

Blocking them is a legitimate choice and does not affect citations. Reported, never penalised.

/llms.txt present, plain text, with an H1, a > summary and links

15

The convention answer engines and agents read first. A soft-404 HTML page counts as missing.

Content readable without JavaScript (words in the raw HTML)

15

Most AI crawlers do not run JS. A client-rendered shell is invisible to them.

JSON-LD structured data (Organization, WebSite, Article, FAQPage...)

10

Entities and FAQs get quoted directly.

Title and meta description, sensible lengths

8

Quoted verbatim in answers.

HTTPS, response time, canonical, Open Graph, sitemap, <html lang>, no noindex

22

Hygiene that decides which URL gets the citation.

As an MCP server (Claude Code, Claude Desktop, Cursor, any MCP client)

Tools: check_site(url, live=true) and draft_llms_txt(url).

Claude Code plugin (includes the MCP server and a skill that knows how to act on the report):

claude plugin marketplace add genzagents/marketplace
claude plugin install citable@genzagents

Any MCP client, by hand:

{ "mcpServers": { "citable": { "command": "python3", "args": ["/path/to/citable/citable/__init__.py", "mcp"] } } }

The server speaks MCP over stdio with no dependencies, so it runs anywhere Python does.

In CI

Fail the build when a deploy breaks AI access (a new WAF rule, a lost llms.txt, a Disallow: / that slipped in):

citable yoursite.com --min-score=70 --json > citable.json

Watch

A deploy that flips a WAF toggle silently removes you from every AI answer. The watch is the alarm. It runs as a GitHub Action in any repo you own (it need not be the site's repo), keeps one issue per site up to date, and comments on that issue the moment an answer bot is blocked, llms.txt disappears, the site goes down or the score drops, which GitHub emails to you. No server, no account, no dependencies. Example: issue #1 watches this project's own page.

Copy docs/watch.yml to .github/workflows/citable-watch.yml:

on:
  schedule: [{ cron: "17 6 * * 1" }]   # Mondays
  workflow_dispatch:
permissions: { issues: write, contents: read }
jobs:
  watch:
    runs-on: ubuntu-latest
    steps:
      - uses: genzagents/citable@v0.4.2
        with:
          sites: |
            example.com
            docs.example.com
          # min-score: 75

Inputs: sites (required), min-score (fail below it), live (fetch as each bot, default true), issue (track in an issue, default true), notify (login to @mention on a regression, default repo owner), fail-on-regression (default true). Outputs: score, regressed, changed. Same thing from a shell where gh is logged in:

citable watch example.com docs.example.com --repo=you/your-repo --notify=you

Hosted, $12/month (about £9): the same watch run by us, no repo needed, up to 5 domains, with a weekly digest. Live now through GitHub Sponsors:

  1. Sponsor genzagents at $12 a month →

  2. Open your Hosted watch issue → from the same account. Reports are posted on it (the issue is public).

Track

The watch tells you whether engines can read you. Track tells you whether they do: for the questions you want to be the answer to, does ChatGPT, Claude or Perplexity link to your site, and who gets linked instead?

citable track example.com "best tools for X" "how do I Y" --brand=Example
## citable track: example.com

**Visibility 50%**: share of prompt-and-engine pairs where an answer linked to your site.

| Prompt | anthropic | openai | perplexity |
|---|---|---|---|
| best tools for X | **cited** | mentioned | **cited** |
| how do I Y | no | no | no |

**Who gets cited instead** (answers citing them): rival.io (4), other.org (2)

Engines: anthropic (ANTHROPIC_API_KEY, the web_search tool), openai (OPENAI_API_KEY, Responses API web search), perplexity (PERPLEXITY_API_KEY, sonar) and claude-code (the claude CLI on your machine, no key). Every engine whose key is present runs; --engines= picks a subset. Add --issue --repo=you/repo (or run it from the Action with a prompts: input) and it keeps a "citable track: example.com" issue up to date and comments when a citation is lost, which is the weekly email the paid tools charge $29 to $39 a month for. Bring your own keys; a 10-prompt run on three engines costs a few cents.

Hosted ($12/month, about £9)

No repo, no keys, no cron: sponsor genzagents at $12 a month, open a Hosted watch issue with your sites (up to 5), the prompts that matter (up to 10) and your brand, and the first report is posted on the issue within the hour. After that you get a comment the moment something regresses and a digest once a week with the prompt tracking (GitHub emails you each one). Cancel from your GitHub sponsorships page.

citable hosted run --source=github matches active sponsors to the open hosted issues they authored and comments the reports there, so it needs only a GitHub token. The Polar + Resend mode below is for private email delivery.

The runner is in this repo (citable hosted), so you can also run the same thing yourself for your own customers: it polls Polar for active subscriptions, keeps one small state file per subscriber, and sends through Resend. Zero dependencies, one hourly cron. See docs/HOSTED.md.

citable hosted run --dry-run                  # against Polar, prints the emails instead of sending
citable hosted run --subscribers=subs.json    # against a local list (no Polar), e.g. for your own clients
citable hosted launchd                        # a launchd plist that runs it hourly on a Mac

Notes

  • MCP registry name: mcp-name: io.github.genzagents/citable (PyPI package citable-cli; the import and the command stay citable).

  • Bot user agents and roles follow the vendors' published strings (OpenAI, Anthropic, Perplexity). Role "answer" = fetches to answer or cite in a live conversation; "train" = builds training corpora.

  • The edge check compares HTTP status only. A site that returns 200 with a challenge page to bots will pass; file an issue with the domain and it gets a heuristic.

  • --no-live skips the bot fetches (faster, robots.txt only).

Development

python3 -m unittest discover -s tests -v

MIT. Built by genzagents.

Available Tools

2 tools
check_siteA

Check whether AI answer engines (ChatGPT search, Claude, Perplexity) can read and cite a website. Fetches robots.txt, llms.txt, sitemap and the homepage as each AI bot, scores 0-100 and lists fixes.

NameRequiredDescriptionDefault
urlYesSite URL or domain, e.g. example.com
liveNoAlso fetch the homepage as each AI bot to detect WAF/CDN blocks (default true)

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full behavioral burden. It does disclose valuable behavior beyond the schema, namely that multiple resources are fetched per bot and that the result is a numeric score with remediation items, but it says nothing about latency, rate limits, or what happens when a site is unreachable or fully blocked.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, and the core purpose is front-loaded before the mechanism and return shape. Every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description correctly compensates by naming the return shape (score 0-100, list of fixes). For a two-parameter read-only diagnostic it is nearly sufficient; only failure-mode behavior is left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both url and live are already documented, and the description's 'as each AI bot' phrasing merely restates the live flag's meaning. Nothing in the description adds format, syntax, or validation detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Check') plus the exact resource and target platforms (ChatGPT search, Claude, Perplexity), then enumerates the artifacts inspected and the output form (0-100 score plus fixes). It is unmistakable against the sibling draft_llms_txt, which produces a file rather than diagnosing one.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The diagnostic framing implies when to call it, but there is no explicit statement of context, prerequisites, or when to prefer draft_llms_txt instead. An agent must infer the use case, though the inference is an easy one.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

draft_llms_txtB

Draft a starter /llms.txt for a website from its homepage: title, summary and links to key pages.

NameRequiredDescriptionDefault
urlYesSite URL or domain

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description carries the full behavioral burden and falls short. It does not say whether the tool fetches the live homepage over the network, whether it writes a file anywhere or only returns text, whether authentication or rate limits apply, or how the 'key pages' are selected. For a generative/scraping operation these are material unknowns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that names the artifact first and then its expected contents. Every clause carries information; nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description should clarify what comes back (raw markdown text? a file path? a preview?) and how the page list is derived. It is minimally adequate for a one-parameter tool but leaves the return contract unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter and schema coverage is 100%, so the schema already documents 'url' as 'Site URL or domain'. The description adds the useful clarification that the input is treated as a homepage (root) rather than an arbitrary page, but nothing on URL format, trailing paths, or redirect handling.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Draft') plus a specific artifact ('/llms.txt') and its source ('from its homepage'), with the produced contents enumerated (title, summary, links to key pages). It does not reference the sibling check_site, so an agent gets no explicit signal about how this differs from that tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The word 'starter' implies this produces a scaffold to be refined rather than a finished file, which is mild usage context. There is no when-to-use/when-not phrasing, no prerequisites, and no mention of the alternative check_site, leaving routing entirely to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv0.1.0
    • First observedcheck_site
    • First observeddraft_llms_txt
Deploy Server
Shareable, User-Owned Context BackendPeople, applications, and agents read, write, and coordinate together in one context lake. Reach it over MCP, a CLI, a Python SDK, or REST.fulcradynamics.comAd

TDQS

A3.6/5.0

Scored across 2 tools

Disambiguation5/5

The two tools have clearly distinct purposes: check_site audits AI crawler accessibility, while draft_llms_txt generates a starter file. Their names and descriptions make selection unambiguous.

Naming Consistency5/5

Both names use consistent snake_case verb_noun structure: check_site and draft_llms_txt. The acronym in llms_txt is readable and does not break the pattern.

Tool Count3/5

Two tools is thin for a site-auditing and remediation domain, even though each tool is focused. It fits a narrow MVP scope but leaves little room for related operations.

Completeness3/5

The surface covers checking AI readability and drafting /llms.txt, but lacks obvious companion operations like generating robots.txt fixes, producing sitemap guidance, or validating an existing llms.txt. Agents may hit dead ends when asked to remediate issues found by check_site.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive

Resources

Related MCP Connectors

Related MCP Servers

  • Crawl Readiness MCP Server

    Grubbomatic
    A
    license
    A
    quality
    A
    maintenance
    Checks what GPTBot, ClaudeBot, PerplexityBot and 50+ other AI crawlers can read on a site and what its access policy allows, checks whether a page permits search, answers, training or agent use (with a signed receipt), generates llms.txt, robots.txt and JSON-LD, and reads your Answer Analysis. Nine tools for Claude Desktop, Cursor and other MCP clients.
    9
    698 npm
    MIT

  • aivis-mcp

    krisdiallo
    A
    license
    A
    quality
    A
    maintenance
    Lets an MCP client check whether an ecommerce store can be read by AI assistants, auditing robots.txt to see which AI search versus training crawlers are blocked and sampling product pages to confirm structured data and facts survive in raw HTML without JavaScript. It can also generate a training-only opt-out robots.txt that leaves answer-engine crawlers allowed, all inside the conversation the user is already in.
    2
    MIT

  • OpenAEO

    charlacsina
    A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI coding agents to score any live domain for visibility in AI answer engines and then generate and apply paste-ready fixes — robots.txt, llms.txt, JSON-LD, titles and meta — either directly to project files or as click-by-click admin steps for hosted platforms like Squarespace, Wix and Shopify. It also re-checks edits, reads server access logs to confirm which crawlers actually arrived and whether they were genuine, and offers hosted tools for citation testing, score history, drift and competitor comparison.
    145 npm
    MIT