Skip to main content
Glama
README.md
# clawifi

<!-- mcp-name: io.github.atakanelik34/clawifi-mcp -->

[![PyPI](https://img.shields.io/pypi/v/clawifi.svg)](https://pypi.org/project/clawifi/)
[![Python](https://img.shields.io/pypi/pyversions/clawifi.svg)](https://pypi.org/project/clawifi/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![MCP](https://img.shields.io/badge/MCP-server-purple.svg)](https://clawifi.dev/docs)
[![GitHub stars](https://img.shields.io/github/stars/ReARCLabs/clawifi-mcp?style=flat)](https://github.com/ReARCLabs/clawifi-mcp/stargazers)

Official Python SDK, CLI, and MCP server for [Clawifi](https://clawifi.dev), web search and page reading for AI agents: search many engines as one, read pages and PDFs, map and crawl sites, and get back only the passages that answer the question, each with its source. One typed API and MCP surface for all of it.

This package is a thin client over the hosted Clawifi API (hosted API version 1.12.1 at time of release); the Clawifi server is not included. Docs, pricing and status: [clawifi.dev/docs](https://clawifi.dev/docs) · [clawifi.dev/pricing](https://clawifi.dev/pricing) · [clawifi.dev/status](https://clawifi.dev/status) · [clawifi.dev/security](https://clawifi.dev/security).

## One-click install (hosted server, no key needed to try)

[![Add to Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=clawifi&config=eyJ1cmwiOiJodHRwczovL2FwaS5jbGF3aWZpLmRldi9tY3AifQ%3D%3D)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Clawifi-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect?url=vscode%3Amcp%2Finstall%3F%257B%2522name%2522%253A%2522clawifi%2522%252C%2522type%2522%253A%2522http%2522%252C%2522url%2522%253A%2522https%253A%252F%252Fapi.clawifi.dev%252Fmcp%2522%257D)

Both add the hosted remote server `https://api.clawifi.dev/mcp`. No key to copy: the free tools work at once, and your client asks you to sign in (a free account, 1,000 credits a month) when a call needs one. One line for every other client (Claude Code, Codex, Claude, ChatGPT, Hermes): [clawifi.dev/install](https://clawifi.dev/install?utm_source=github&utm_medium=readme&utm_campaign=mcp_install).

## Agent skill

```bash
npx skills add ReARCLabs/clawifi-mcp
```

Installs [`skills/clawifi/SKILL.md`](skills/clawifi/SKILL.md) into your coding agent: when to use Clawifi, which tool fits, and what each call costs. The same file is at [clawifi.dev/agent-onboarding/skill.md](https://clawifi.dev/agent-onboarding/skill.md).

## Install

```bash
pip install clawifi
```

## Get an API key

```bash
clawifi signup --email you@example.com --password "a strong password"
```

This creates an account, grants 1,000 free credits, and prints an API key. Export it:

```bash
export CLAWIFI_API_KEY="clawifi_live_..."
```

## Quicktest

```bash
clawifi quicktest --url https://example.com
```

## Set up your MCP client

```bash
clawifi init
```

finds the MCP clients installed on this machine (Claude Code, Codex, Cursor, Claude Desktop, VS Code) and prints, for each one, the install line for the local server (`clawifi-mcp`, stdio) and for the remote server, followed by a first call to try. It only prints; it never shows your key, only `$CLAWIFI_API_KEY` or a masked `clawifi_live_****`. `clawifi init --all` shows every client, found or not.

To have it edit a config file for you:

```bash
clawifi init --write cursor            # ~/.cursor/mcp.json, stdio entry
clawifi init --write cursor --remote   # ~/.cursor/mcp.json, remote entry
clawifi init --write claude-desktop    # claude_desktop_config.json, stdio entry
```

`--write` asks before it changes anything and copies the existing file to `<file>.bak-<timestamp>` first. Claude Code, Codex and VS Code have their own install commands, which `init` prints.

For Claude Code, for example:

```bash
# local (stdio)
claude mcp add --env CLAWIFI_API_KEY=$CLAWIFI_API_KEY --transport stdio clawifi -- clawifi-mcp
# remote
claude mcp add --transport http clawifi https://api.clawifi.dev/mcp --header "Authorization: Bearer $CLAWIFI_API_KEY"
```

## Use as an MCP server

The local MCP server runs as:

```bash
clawifi-mcp
```

reading `CLAWIFI_API_KEY` from the environment. `clawifi mcp config` prints a minimal config for it. Every tool carries a title and the MCP hints `readOnlyHint`, `destructiveHint`, `idempotentHint` and `openWorldHint`; none is destructive.

The billable tools (`fetch`, `scrape`, `extract`, `map_site`, `crawl`, `search_free`) take their common options as parameters and any other API field in `options`, which is merged into the request body. They refuse `dry_run`: use the `estimate` tool to price a call. Arguments a tool does not declare are refused rather than ignored.

`fetch` takes `content_scope` (`"main"`, the default, or `"full"` for the whole page text with scripts and styles removed) and `render` (`"never"`, the default; `"auto"` opens the browser only for a JavaScript shell; `"always"` reads in the browser). A browser render costs 3 credits instead of 1, and `estimate("fetch", {"url": ..., "render": "auto"})` quotes the range first. Any other value is refused and nothing is sent; both are sent only when they differ from the default. Each tool's description names the fields it returns (see [Response fields](#response-fields)).

## Remote MCP

Clawifi also runs a hosted MCP server, so nothing needs installing:

- URL: `https://api.clawifi.dev/mcp` (streamable HTTP)
- Auth: `Authorization: Bearer <your key>`, or the `X-Clawifi-Api-Key: <your key>` header

The old host `api.clawifi.qzz.io` (API and `/mcp`) keeps working, so `CLAWIFI_API_BASE=https://api.clawifi.qzz.io` and existing remote configs need no change.

Without a key the remote server offers a keyless tier of four tools, `search_free`, `fetch_url`, `scrape_url` and `map_site`, at 12 calls a minute and 50 a day per address. With a key you get the full tool list.

## Credits

Calls are charged per operation (a plain fetch is 1 credit, a browser render 3, a crawl 1 per page, and so on). See [clawifi.dev/pricing](https://clawifi.dev/pricing), or print the current table, which needs no key:

```bash
clawifi credits
```

The `credits` MCP tool and `client.credits()` show your own usage and balance instead.

## Use as a Python SDK

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY from the environment
page = client.fetch("https://example.com")
print(page["success"])
```

An async client is also available as `clawifi.AsyncClawifi`.

`search_content` searches and reads the top results; it is compact by default and `full=True` returns the full output:

```python
results = client.search_content("agent memory benchmarks", full=True)
```

Responses are returned as the API sends them, so fields such as `published_at`, `modified_at` and `fetched_at` come through untouched. Request fields are passed through as keyword arguments too:

```python
page = client.fetch("https://example.com/pricing", content_scope="full")   # the whole page text
page = client.fetch("https://app.example.com", render="auto")             # browser only for a JavaScript shell
```

### Response fields

Every response field, and the fields a call does not return, is listed on the endpoint pages:
[search](https://clawifi.dev/docs/api/post-search-free#response),
[search with content](https://clawifi.dev/docs/api/post-search-content#response),
[fetch](https://clawifi.dev/docs/api/post-fetch#response),
[scrape](https://clawifi.dev/docs/api/post-scrape#response) and
[extract](https://clawifi.dev/docs/api/post-extract#response).

Fields worth knowing (the search cache since hosted API 1.11.0, the page fields since 1.12.0):

- `search_free`: `cache_hit` is `true` when an identical search within 60 seconds was answered from the cache; such an answer costs 0 credits. Each result carries `url`, `title`, `snippet`, `rank`, `provider` and, when the engine gives a date, `published_at` and `modified_at`.
- `fetch`: `data.page_text_chars` (how much readable text the page holds), `data.content_chars` (the length of `content_markdown` before any `query` selection), `data.scope_used` (`main`, `whole`, `selector` or `full`) and `data.language` (the page's `<html lang>`). `mode_used` is `browser` when `render: "auto"` rendered a JavaScript shell; with the default `render: "never"` such a page answers `content_status: "needs_javascript"` and costs 0.

### Estimates (hosted API 1.7.0 and later)

```python
quote = client.estimate("fetch", url="https://example.com")
# {"dry_run": true, "route": "POST /fetch", "estimate": {"minimum": ..., "maximum": ..., "raises": [...]},
#  "balance": ..., "would_402": ..., "would_429": ..., "binding_limit": ..., "charged": 0}
client.crawl("https://example.com", max_pages=50, dry_run=True)
```

`estimate()` and `dry_run=True` work for `fetch`, `scrape`, `extract`, `map`, `crawl`, `search_free` and `search_content`. Before the first dry run the client checks the API version at `/product/manifest`; an API older than 1.7.0 would ignore `dry_run` and run (and bill) the call, so the client raises `ClawifiDryRunUnsupportedError` and sends nothing. Pre-releases such as `1.7.0rc1` count as older than 1.7.0.

`dry_run` is sent only when it is `True`, and only as the JSON boolean `true`: a normal call, or `dry_run=False`, has no `dry_run` key in its body, and a value that is not a bool raises `TypeError` before anything is sent. The API answers any `dry_run` key (even `false`) on a route that cannot dry-run with a 422, so `schema_assist` drops `dry_run=False` and refuses `dry_run=True`.

The `estimate` MCP tool does the same from an agent: `estimate(operation="fetch", args={"url": "https://example.com"})`.

### Rate limits

A `429` raises `ClawifiAPIError` with `retry_after` (seconds, from the `Retry-After` header) and `limit_scope` (which limit was hit, from `details.limit_scope`). Retryable calls wait out a `Retry-After` of up to 10 seconds before retrying; a longer one is raised straight away.

## MCP tools

| Tool | What it does |
| --- | --- |
| `fetch` | Fetch one URL (page or PDF) as Markdown. With `query`, only the passages that answer it; `content_scope="full"` for the whole page text; `render="auto"` opens the browser only for a JavaScript shell. |
| `scrape` | Scrape a URL with source-aware routing and return structured content. With `query`, only the passages that answer it. |
| `search_free` | Search the web and return ranked results (`url`, `title`, `snippet`, `rank`, `provider`, dates when the engine gives them). An identical search within 60 s is a `cache_hit` for 0 credits. |
| `extract` | Extract structured data from a URL. |
| `map_site` | Map a site's discoverable URLs starting from a seed URL. |
| `crawl` | Start an asynchronous crawl from a seed URL; returns a `crawl_id` to poll. |
| `estimate` | Price a `fetch`, `scrape`, `extract`, `map`, `crawl`, `search_free` or `search_content` call without running it or charging for it (hosted API 1.7.0+). |
| `get_crawl` | Get the current status and results of a crawl. |
| `create_job` | Create an asynchronous job for a billable operation; returns a `job_id` to poll. |
| `get_job` | Get the current status and results of a job. |
| `account` | Get the authenticated account's profile and plan. |
| `credits` | Get the authenticated account's credit usage and remaining balance. |
| `list_history` | List past requests for the authenticated account. |
| `list_monitors` | List configured Evidence Monitor checks for the authenticated account. |

## Documentation

See [clawifi.dev/docs](https://clawifi.dev/docs) for the full API reference, MCP tool list, and CLI command reference.

## License

MIT

TDQS

B3.4/5.0

Scored across 14 tools

Disambiguation3/5

Three tools (fetch, scrape, extract) all retrieve content from a single URL, and only fine wording distinguishes 'Markdown of a page' from 'structured content' from 'schema-based extraction'. There is also dual async machinery (crawl/get_crawl vs create_job/get_job) plus an `estimate` tool that mirrors every other operation, so boundaries are described but easy to misselect.

Naming Consistency3/5

The set mixes consistent verb_noun names (get_crawl, create_job, get_job, map_site, list_history, list_monitors) with bare verbs (fetch, scrape, extract, crawl, estimate) and bare nouns (account, credits). It stays readable, but there is no single predictable pattern and polling is split between get_crawl and get_job.

Tool Count5/5

14 tools is comfortably within a healthy range and each maps to a plausible capability of a scraping/search API (search, fetch, scrape, extract, map, crawl, pricing, jobs, account, credits, history, monitors). Nothing is bloated enough to be unwieldy.

Completeness4/5

Core read/lifecycle workflows (search, fetch, scrape, extract, map, crawl, async jobs, cost estimation, billing, history) are well covered. Minor gaps: no cancel/delete for jobs or crawls, and list_monitors has no create/get counterpart, leaving monitor management a dead end; estimate also references a search_content operation not exposed as a tool.

Maintenance

ActivityActive
ResponsivenessNo recent community issues