nab

Fetch any URL as clean markdown — with your browser cookies, anti-bot evasion, and 25x fewer tokens than raw HTML.

nab is a local, token-optimized HTTP client built for LLM pipelines. It converts web pages to clean markdown, injects your real browser session cookies for authenticated content, and spoofs browser fingerprints to bypass bot detection. No API keys. No cloud. Just fast, authenticated, LLM-ready output.

Why nab?

Quick Start

# Install (pick one)
brew install MikkoParkkola/tap/nab    # Homebrew
cargo install nab                      # From crates.io
cargo binstall nab                     # Pre-built binary

Fetch a page as clean markdown

nab fetch https://example.com

Access authenticated content with your browser cookies

# Auto-detects your default browser and injects session cookies
nab fetch https://github.com/notifications --cookies brave

No login flows. No API keys. nab reads your existing browser cookies (Brave, Chrome, Firefox, Safari, Edge, Dia) and uses them for the request. You stay logged in — nab just borrows the session.

Bypass bot detection with fingerprint spoofing

# Realistic Chrome/Firefox/Safari profiles — not a headless browser signature
nab fetch https://protected-site.com

nab ships with anti-fingerprinting by default: realistic TLS fingerprints, browser-accurate headers, and randomized profiles. Sites see a normal browser, not a scraping tool.

Features

• 11 Site Providers — Specialized extractors for Twitter/X, Reddit, Hacker News, GitHub, Google Workspace, YouTube, Wikipedia, StackOverflow, Mastodon, LinkedIn, and Instagram. API-backed where possible for structured output.

• Google Workspace Extraction — Fetch Google Docs, Sheets, and Slides as clean markdown using browser cookies. Extracts comments and suggested edits from OOXML (docx/xlsx/pptx).

• HTML-to-Markdown — Automatic conversion with boilerplate removal. 25x token savings vs raw HTML.

• PDF Extraction — PDF-to-markdown with heading and table detection (requires pdfium).

• Browser Cookie Auth — Auto-detects your default browser (Brave, Chrome, Firefox, Safari, Edge, Dia) and injects session cookies. Zero config.

• 1Password Integration — Credential lookup, auto-login with CSRF handling, TOTP/MFA support.

• Passkey/WebAuthn — Native passkey authentication via 1Password's open-source library.

• HTTP/3 (QUIC) — 0-RTT connection resumption, HTTP/2 multiplexing, TLS 1.3.

• Anti-Fingerprinting — Realistic Chrome/Firefox/Safari browser profiles to avoid bot detection.

• JS Engine (QuickJS) — Lightweight embedded JavaScript for pages that need it, without a full browser.

• Compression — Brotli, Zstd, Gzip, Deflate decompression built in.

• Query-Focused Extraction — BM25-lite scoring extracts only the sections relevant to your query. Send focus="authentication" and get back just the auth docs, not the entire page.

• Token Budget — Structure-aware truncation respects headings, code blocks, and tables. Never splits mid-block. Set max_tokens=2000 to fit any context window.

• Prefetch Link Graph — Extract same-site links from fetched pages, scored by relevance to your focus query. eTLD+1 filtering via Mozilla's public suffix list.

• Persistent Sessions — Named sessions with automatic cookie persistence across requests. LRU eviction (32 slots), cookie seeding from browser jars.

• CSS Extractor Plugins — Define custom extractors in plugins.toml using CSS selectors — no Rust code required.

• MCP Server — nab-mcp binary for direct integration with Claude Code and other MCP clients.

• Batch Fetching — Parallel URL fetching with connection pooling.

Site Providers

nab detects URLs for these platforms and uses their APIs or structured data instead of scraping HTML:

If no provider matches, nab falls back to standard HTML fetch + markdown conversion.

Usage

# Basic fetch (auto-cookies, markdown output)
nab fetch https://example.com

# Force specific browser cookies
nab fetch https://github.com/notifications --cookies brave

# With 1Password credentials
nab fetch https://internal.company.com --1password

# Google Docs (markdown with comments and suggested edits)
nab fetch --cookies brave "https://docs.google.com/document/d/DOCID/edit"

# Google Sheets (CSV rendered as markdown table)
nab fetch --cookies brave "https://docs.google.com/spreadsheets/d/SHEETID/edit"

# Google Slides (plain text with comments)
nab fetch --cookies brave "https://docs.google.com/presentation/d/SLIDEID/edit"

# Raw HTML output (skip markdown conversion)
nab fetch https://example.com --raw-html

# JSON output format
nab fetch https://api.example.com --format json

# Batch benchmark
nab bench "https://example.com,https://httpbin.org/get" -i 10

# Get OTP code from 1Password
nab otp github.com

# Generate browser fingerprint profiles
nab fingerprint -c 5

CLI Reference

Common flags for fetch:

PDF Extraction

nab converts PDF files to markdown with heading detection and table reconstruction. Requires pdfium (ships with Chromium, or install via Homebrew).

# Fetch a PDF and convert to markdown
nab fetch https://example.com/report.pdf

# Save PDF conversion to file
nab fetch https://arxiv.org/pdf/2301.00001 -o paper.md

The PDF pipeline extracts character positions via pdfium, reconstructs text lines, detects tables through column alignment, and renders clean markdown. Target performance is ~10ms/page. Maximum input size is 50 MB.

Proxy Support

nab supports HTTP and SOCKS5 proxies via the --proxy flag or environment variables.

# Explicit proxy
nab fetch https://example.com --proxy socks5://127.0.0.1:1080
nab fetch https://example.com --proxy http://proxy.company.com:8080

# Environment variables (checked in this order)
export HTTPS_PROXY=http://proxy:8080
export HTTP_PROXY=http://proxy:8080
export ALL_PROXY=socks5://proxy:1080

The --proxy flag takes precedence over environment variables. Both uppercase and lowercase variants (HTTPS_PROXY / https_proxy) are recognized.

Environment Variables

Configuration

nab requires no configuration files. It uses smart defaults: auto-detected browser cookies, randomized fingerprints, and markdown output.

Optional plugin configuration at ~/.config/nab/plugins.toml:

# Binary plugin — external process (original format)
[[plugins]]
name = "my-provider"
binary = "/usr/local/bin/nab-plugin-example"
patterns = ["example\\.com/.*"]

# CSS extractor — no external binary needed (new in v0.5)
[[plugins]]
name     = "internal-wiki"
type     = "css"
patterns = ["wiki\\.corp\\.com/.*"]

[plugins.content]
selector = "div.wiki-content"
remove   = ["nav", ".sidebar"]

[plugins.metadata]
title     = "h1.page-title"
author    = ".author-name"
published = "time.published"

Binary plugins receive a URL as JSON on stdin and return markdown on stdout. CSS extractors run in-process using configurable selectors — no code required.

Persistent state stored in ~/.nab/:

MCP Server

nab ships a native Rust MCP server (nab-mcp) for integration with Claude Code and other MCP clients.

Setup -- add to your MCP client configuration:

{
  "mcpServers": {
    "nab": {
      "command": "nab-mcp"
    }
  }
}

Available tools:

The MCP server uses MCP protocol 2025-11-25 (latest) over stdio and shares a single AcceleratedClient across all tool calls for connection pooling.

Protocol features:

• Tool annotations — read-only, destructive, and open-world hints on all 8 tools

• Structured output — outputSchema + structured_content on all 8 tools (machine-parseable JSON alongside human-readable text)

• URL elicitation — OAuth/SSO login sends the user to the auth URL in-browser (Google, GitHub, Microsoft, Apple, and 9 more)

• Form elicitation — interactive credential input and multi-select cookie source picker

• Task-augmented execution — fetch_batch can run asynchronously with progress notifications

• Server icons — globe SVG in light/dark themes

Benchmarks

HTML-to-markdown conversion throughput (via cargo bench):

Arena allocator vs Vec<String> for response buffering:

Run benchmarks yourself: cargo bench

Install

Homebrew (macOS/Linux)

brew tap MikkoParkkola/tap
brew install nab

From crates.io (requires Rust 1.93+)

cargo install nab

Pre-built binary (cargo-binstall)

cargo binstall nab

Or download directly from GitHub Releases:

From source

git clone https://github.com/MikkoParkkola/nab.git
cd nab && cargo install --path .

Library Usage

use nab::AcceleratedClient;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let client = AcceleratedClient::new()?;
    let html = client.fetch_text("https://example.com").await?;
    println!("Fetched {} bytes", html.len());
    Ok(())
}

Requirements

• Rust 1.93+ (for building from source)

• ffmpeg (optional, for streaming/analyze commands): brew install ffmpeg

• 1Password CLI (optional): Install guide

Architecture

See docs/ARCHITECTURE.md for the full internal architecture, module organization, data flow diagrams, and extension points.

Contributing

See CONTRIBUTING.md for development setup, code style guidelines, testing instructions, and pull request process.

Responsible Use

This tool includes browser cookie extraction and fingerprint spoofing capabilities. These features are intended for legitimate use cases such as accessing your own authenticated content and automated testing. Use responsibly and only on sites where you have authorization.

License

MIT License - see LICENSE for details.