Net-Syphon

A stdio-only MCP server for web search and anonymous public-page retrieval:

net_syphon_search_web(query, max_results=5, search_category="general", include_domains=[],
                      time_range=None, start_date=None, end_date=None)
net_syphon_get_pages(urls, max_characters=20000)

Search returns links in the provider's order, and optional previews, not verified evidence. Retrieval returns bounded plain text, title, requested and provider-reported final URL, timestamp, media type, truncation flag, and a hash of the returned text. Net-Syphon owns validation, request limits, errors, and auditing. Consumers own planning, cross-call budgets, safe rendering, and synthesis. Ordinary search uses SearXNG; news or filtered search and retrieval use Firecrawl's direct API. Consumers select intent, never providers. There is no automatic fallback or local browser.

Status and roadmap

Search, filtered/news search and retrieval are implemented with fixture-based tests. The deployed SearXNG live contract passed on 2026-09-06 after operator maintenance; see the diagnostic history. Firecrawl live verification requires an operator-configured key and remains outstanding. The ORIS checkout now uses Net-Syphon for search plus bounded page retrieval; deployment and live acceptance remain separate. See the current plan.

Tavily fallback and local browser support are not planned. Prefer simple, explicit behavior; add complexity only when actual use demonstrates a need.

Future considerations—not committed work

Net-Syphon's scope is broad web search and scraping. Net-Razor handles targeted information collection, including its existing RSS/Atom support. OpenAlex is a possible Net-Razor addition, not a Net-Syphon proposal. Page-change monitoring is not planned here.

Keep these ideas for later evaluation; none is approved for implementation:

• Operator diagnostics: a doctor command for configuration, permissions, audit availability, and explicit connectivity checks with actionable errors.

• Small-model-friendly retrieval: bounded text, useful headings, clear truncation, and possibly section selection. Summarization stays in ORIS.

• Additional search controls: language filtering if real usage needs it.

• Usage safeguards: optional operator-set request allowances and rate-limit cooldowns, without automatic retries or fallback. Daily limits would need to survive server restarts.

• Local source ingestion: accept PDFs, manuals, and saved offline content, store them as a source collection, and make that information retrievable by a local AI through MCP. This could be a Net-Syphon module or a separate tool; ownership, storage, and retrieval design remain undecided. It would require a separate review of document parsing, resource limits, privacy, retention, and untrusted content. Source storage would be separate from metadata-only audit logs.

• Prompt-injection screening in ORIS: evaluate a small dedicated classifier, such as Meta Prompt Guard 2, before external content reaches the main model. This is not implemented or a selected dependency; see the security considerations.

• Sentinel in ORIS: a roadmap idea for audit review, not implemented and with no current implementation commitment. If built, it would be a separate implementation inside ORIS, not a Net-Syphon component or current safeguard.

Related MCP server: web-research-mcp

Setup

Use a current Python 3.12 patch release, uv, and a SearXNG instance with JSON output enabled. The filesystem protections target macOS/POSIX.

uv sync --locked

Configure your MCP consumer with an absolute interpreter path and your instance's base URL, not its /search endpoint:

{
  "command": "/absolute/path/to/net-syphon/.venv/bin/python",
  "args": ["-m", "net_syphon"],
  "env": {"NET_SYPHON_SEARXNG_URL": "http://192.0.2.10:8080"}
}

Replace both example paths/addresses. Alternatively, store the setting in ~/.net-syphon/.env using .env.example. Environment values take precedence. The directory must be owned by the runtime user with mode 0700; the file must be owned, regular, single-link, and mode 0600. Symlinks and repository-local dotenv files are not supported.

Set NET_SYPHON_FIRECRAWL_API_KEY privately in the same protected dotenv file or launch environment for retrieval and filtered/news search. Missing configuration disables only the affected capability. No key is needed for ordinary SearXNG search. Use a client timeout of at least 210 seconds to accommodate a bounded batch. The net-syphon console command starts stdio transport. Its one subcommand, net-syphon doctor, checks configuration, permissions and audit health and exits non-zero when something needs fixing. It creates and repairs nothing. Add --connect to make one real request per configured capability, which spends one hosted retrieval request. Connectivity is skipped when the audit is unwritable, because egress is disabled in that state.

Operation

• Queries allow 1–500 characters; result limits are integers from 1–30. Provider-control modifiers are rejected. Discover the tool's schema for the complete contract.

• Search accepts up to ten domain restrictions, a relative day/week/month/year, or paired ISO dates (start inclusive, end exclusive). Relative and absolute periods cannot be combined. Dates constrain the hosted search; they are not verified publication dates. published_at uses the provider's own date where there is one. News recency arrives as a relative label and is resolved against the retrieval time, so it is accurate to the unit stated rather than exact. It is null when the provider supplies nothing, which is common: ordinary search carries a date on roughly a fifth of results, and filtered non-news search carries none at all.

• Retrieval supports public HTTP port 80 and HTTPS port 443, HTML and plain text only. It preflights DNS but does not connect to page servers locally. Hosted redirects remain the provider's responsibility. A final URL is provider-reported, or null.

• Every page in a batch gets the same allowance, whatever the batch size. It defaults to 20,000 characters and max_characters moves it between 1,000 and 50,000. Raise it to read one page deeply; lower it to survey several cheaply. A longer page comes back with truncated set. Batches accept 1–5 URLs sequentially and preserve per-URL success and errors. Deadlines: SearXNG 15 seconds; hosted request 25; individual retrieval 40; batch 180. Already completed pages survive a batch deadline. Response caps are 1 MiB for SearXNG and 2 MiB for hosted requests, including decompressed data.

• Empty results are valid. partial=true means the response is short of what you asked for: entries were rejected and could not be replaced, or the search service reported engine failures. Malformed entries beyond your requested limit cost you nothing and do not set it. On a retrieval batch it means at least one page failed. Errors contain call_id, code, message, and retryable; retries are never automatic.

• not_configured: check configuration and permissions. access_denied: check instance access and JSON support. busy: another tool call is active in this process. engines_unavailable: zero usable results with reported engine failures; audit counters distinguish CAPTCHA, rate-limit, denial and timeout categories.

• Private audit logs are at ~/.net-syphon/logs/. They record IDs, timings, status and counts—not queries, URLs, or content. Retention is 30 UTC dates, pruned during use, with a 10 MiB daily cap.

• Audit failure disables further requests in that process. Investigate the stderr diagnostic, preserve logs, fix the cause, then restart. A full daily file normally requires waiting for the next UTC day. Retention deletion is permanent without backups.

Read the security guidance before connecting a consumer. Contributors should start with AGENTS.md and ARCHITECTURE.md.

Development

uv run --locked pytest
uv run --locked ruff check .
uv run --locked ruff format --check .
uv run --locked pip-audit

Default tests use local fixtures, including temporary loopback HTTP/TLS servers. To intentionally query the configured service: uv run --locked pytest --live -m live.