Web Forager

The thing about information on the web is that it doesn't want to be found. It wants to hide behind cookie banners, keep itself to itself, and generally behave like a cat that knows it's time for the vet. Web Forager is the sort of dogged, slightly grubby assistant who goes out there anyway — accompanied by a duck of questionable temperament — rummages through DuckDuckGo, grabs pages directly when it can, and calls in Jina Reader when things get complicated. The results come back neatly converted for LLM consumption, which is to say, in a format that would make a librarian weep with either joy or despair, depending on the librarian.

Web Forager gives AI agents practical web research workflows as Agent Skills. The skills search DuckDuckGo, monitor news, fetch pages, and synthesize cited answers.

Default usage is skill-first. You do not need to configure an MCP server to use the research workflows.

Install For Your Coding Tool

Install as skills/plugins when your agent supports them. Use MCP only when your tool does not support skills, or when you want raw search/fetch tools instead of guided research workflows.

Install all five skills from the plugin marketplace:

/plugin marketplace add CyranoB/web-forager
/plugin install forager-skills@web-forager

Restart Claude Code, then check /skills.

MCP-only fallback:

claude mcp add --transport stdio web-forager -- uvx --python ">=3.10,<3.14" web-forager serve

Install the skills manually:

git clone https://github.com/CyranoB/web-forager.git
mkdir -p ~/.codex/skills
cp -R web-forager/skills/* ~/.codex/skills/

MCP-only fallback:

codex mcp add web-forager -- uvx --python ">=3.10,<3.14" web-forager serve

If your VS Code build supports agent plugins, use the command palette:

1. Open Cmd+Shift+P on macOS or Ctrl+Shift+P on Windows/Linux.

2. Run Chat: Install Plugin From Source.

3. Paste https://github.com/CyranoB/web-forager.

MCP-only fallback: configure a local MCP server with the standard config below.

If you want MCP tools:

gemini mcp add web-forager uvx --python ">=3.10,<3.14" web-forager serve

For skills, copy the folders from skills/ into the skills location supported by your Gemini environment.

Install globally for Pi:

git clone https://github.com/CyranoB/web-forager.git
mkdir -p ~/.pi/agent/skills
cp -R web-forager/skills/* ~/.pi/agent/skills/

Or install only for the current project:

git clone https://github.com/CyranoB/web-forager.git
mkdir -p .pi/skills
cp -R web-forager/skills/* .pi/skills/

Pi also discovers skills from the generic Agent Skills directories:

mkdir -p ~/.agents/skills
cp -R web-forager/skills/* ~/.agents/skills/

For one-off sessions, pass a skill path explicitly:

pi --skill web-forager/skills/web-research

Install globally for all Kiro CLI projects:

git clone https://github.com/CyranoB/web-forager.git
mkdir -p ~/.kiro/skills
cp -R web-forager/skills/* ~/.kiro/skills/

Or install only for the current workspace:

git clone https://github.com/CyranoB/web-forager.git
mkdir -p .kiro/skills
cp -R web-forager/skills/* .kiro/skills/

Kiro's default agent loads skills from both locations automatically. For custom agents, add skill resources such as:

{
  "resources": [
    "skill://.kiro/skills/*/SKILL.md",
    "skill://~/.kiro/skills/*/SKILL.md"
  ]
}

Use the standard MCP config below unless your client supports Agent Skills or plugins from a GitHub repository. If it does, install from:

https://github.com/CyranoB/web-forager

Individual Skills

For any Agent Skills-compatible tool, install one skill folder directly. For example:

claude install-skill github:CyranoB/web-forager/skills/web-research

Or install from a local checkout:

git clone https://github.com/CyranoB/web-forager.git
cd web-forager
claude install-skill ./skills/web-research

For agents without claude install-skill, copy a folder from skills/ into your agent's skills directory.

Use The Skills

After installing, ask your agent naturally. The matching skill should be selected automatically by skill metadata.

Examples:

Research the current state of open-source browser agents.
Fact check: did Apple announce a foldable iPhone?
What's new with Anthropic this month?
Map the competitive landscape for AI meeting assistants.
Should we adopt Bun for a production Node service?

Available Skills

How Search Works

The skills prefer tools in this order:

1. Existing MCP search/fetch tools, if your agent already has them.

2. Built-in agent web search/fetch tools.

3. The packaged Web Forager CLI through uvx.

4. A direct ddgs fallback through uv run --no-project.

Python 3.10-3.13 is supported. Python 3.14 is not supported yet, so all documented uvx commands pin --python ">=3.10,<3.14".

Optional: MCP Server

Use the MCP server only if you want reusable search/fetch tools exposed directly to an MCP-compatible client. Skills work without this setup.

Add a local stdio MCP server with this standard config:

{
  "mcpServers": {
    "web-forager": {
      "command": "uvx",
      "args": ["--python", ">=3.10,<3.14", "web-forager", "serve"]
    }
  }
}

Some clients use a different top-level config shape, but the command and args are the same.

Optional: CLI

Run commands without installing the package:

uvx --python ">=3.10,<3.14" web-forager search "your search query" --max-results 5 --output-format text
uvx --python ">=3.10,<3.14" web-forager news "your topic" --max-results 10 --output-format text
uvx --python ">=3.10,<3.14" web-forager fetch "https://example.com" --format markdown

Or install locally:

uv pip install web-forager
web-forager search "your search query"
web-forager news "your topic"
web-forager fetch "https://example.com"

MCP Tools

The MCP server exposes:

Search and news tools return JSON by default and support output_format="text" for LLM-friendly formatted results.

Development

git clone https://github.com/CyranoB/web-forager.git
cd web-forager
uv pip install -e ".[dev]"
pytest

Useful local commands:

web-forager serve
web-forager version --debug

Notes

• Search and news search use the ddgs package.

• Fetch tries direct HTTP plus trafilatura first, then falls back to Jina Reader for JavaScript-heavy or bot-protected pages.

• The plugin marketplace manifest lives in .claude-plugin/marketplace.json.

License

MIT. See LICENSE.