agent-gateway

English | 简体中文

A self-hosted AI capability layer for desktop agents. It provides the parts an agent needs beyond the model itself — durable memory, code intelligence, web access and a model gateway — behind MCP and plain HTTP.

The package runs as a standalone Python service or as a subprocess of a larger host. It does not depend on a desktop framework, and hosts integrate through MCP tools, HTTP endpoints and JSON contracts instead of importing internals.

Status: 0.1.0. The public surface is usable, but interfaces may still change between minor releases.

Features

Durable memory

• Two-tier storage. The hot layer keeps raw facts in mem0 backed by an embedded Qdrant collection. The cold layer indexes consolidated knowledge in a separate collection. Both share one embedder and one local vector store.

• Write-time hygiene. Near-duplicate facts are deduplicated, contradictory memories are checked, and facts from the same episode are grouped so related context can be recalled together.

• Lifecycle control. Memories move through active, superseded, archived and deleted states. Important entries can be protected, and superseded versions stay traceable instead of being overwritten.

• Memory graph. A SQLite graph records version chains, episode membership, derivation, contradictions, related entries and mentions, and exposes them through memory_graph.

• Bounded growth. Scheduled jobs archive stale entries, consolidate summaries and fold summary budgets, so the hot layer does not grow without limit.

Retrieval

• Three retrieval arms — dense hot memory, dense cold memory and lexical project notes — are fused and gated by a single pipeline.

• Relevance floors are calibrated per embedding model and can be re-calibrated when the model changes.

• Responses carry a status (ok / weak_match / no_match) and trace metadata, so callers can tell an empty result from a weak one.

Project intelligence

• A resident tree-sitter indexer builds a per-project symbol and call graph for Python, Swift, JavaScript/TypeScript/TSX, Java, Kotlin, Objective-C and Vue.

• project_lookup returns definitions with one-hop caller context.

• project_impact walks callers or callees to show what a change touches.

• project_notes searches project Markdown by meaning, using hybrid semantic, keyword and path ranking with de-duplication.

Web access

• websearch_engine queries configurable search engines, including Bing, DuckDuckGo, Brave, Sogou, 360, Baidu and Marginalia. Google can be plugged in through an external command provider.

• read_url converts pages to Markdown with trafilatura, falls back to a lighter converter for short or empty pages, and discards boilerplate.

• Search caching, per-engine rate limits and optional top-page fetching are built in.

Model gateway

• The token proxy accepts Anthropic- and OpenAI-shaped traffic on /v1/*, with /open/* and /raw/* aliases for compatibility and transparent forwarding.

• Requests are routed by exact model name across multiple providers; the upstream model can be remapped.

• Streaming responses, gzip/deflate/br/zstd decoding, SSE usage extraction and per-request accounting are handled in one place.

• Optional request/response rewrites include tool-output compaction, repeated skill de-duplication and a per-request annotation of what was rewritten.

• Usage is stored in SQLite and exposed through /usage/summary, /usage/tools and /usage/layers.

Local-first operation

• config.yaml is the single configuration source; environment variables override individual values.

• Vector data, caches, logs and SQLite databases stay under GATEWAY_HOME and GATEWAY_CACHE.

• No hosted control plane and no telemetry. The embedding model is downloaded once and then served from the local cache.

Related MCP server: atlasbrain

Architecture

flowchart TD
    Client["Desktop agent / MCP client"] -->|MCP streamable HTTP| MCP["services.mcp.server"]
    Client -->|HTTP| REST["/api/*"]
    Client -->|Anthropic / OpenAI traffic| Proxy["services.proxy.token_proxy"]

    MCP --> Memory["Hot + cold memory"]
    MCP --> Graph["Code graph / notes"]
    MCP --> Web["Search / read"]
    REST --> LLM["llm.client"]
    REST --> Summ["Summarize"]
    Proxy --> Upstream["LLM providers"]

    Memory --> Qdrant[("Qdrant (local)")]
    Memory --> MemGraph[("SQLite memory graph")]
    Graph --> TS["tree-sitter"]
    Web --> Fetch["trafilatura / HTTP"]

Requirements

• Python 3.12 or newer.

• A local directory for Qdrant data and the embedding model cache.

• Optional: Node.js 18+ only if you use the JavaScript/Vue branch of the legacy AST scanner. The graph indexer is pure Python.

Install

From source:

git clone https://github.com/yuanyxh/agent-gateway.git
cd agent-gateway
python -m venv .venv
. .venv/bin/activate
pip install -e ".[proxy,memory,server]"

Extras are additive; install the ones for the features you use:

Build a wheel for distribution:

python -m build

The wheel includes prompts, locale files and the ast-js helper, so a host can install it into an isolated environment without copying source files.

Quick start

Create a configuration root:

mkdir -p ~/.config/agent-gateway
export GATEWAY_HOME="$HOME/.config/agent-gateway"

Write ~/.config/agent-gateway/config.yaml:

mcp:
  enabled: true
  qdrantPath: ~/.config/agent-gateway/data/qdrant
  embedderModel: sentence-transformers/all-MiniLM-L6-v2
  knowledgeBasePath: ~/notes

modelProxy:
  enabled: false

Check the environment and start the MCP service:

agent-gateway --check
agent-gateway --serve --service mcp

The first start downloads the embedding model into the cache. The service listens on http://127.0.0.1:9001; point an MCP client at the streamable HTTP endpoint:

{
  "mcpServers": {
    "agent-gateway": {
      "url": "http://127.0.0.1:9001/mcp"
    }
  }
}

/health reports liveness. If mcp.port is 0, the CLI probes for a free port and records it under GATEWAY_HOME/runtime/.

Enable the model gateway

Add one or more providers and start the proxy:

modelProxy:
  enabled: true
  providers:
    - name: anthropic
      url: https://api.anthropic.com
      apiFormat: anthropic
      apiKey: "your-key"
      models: [claude-sonnet-5-5]

agent-gateway --serve --service proxy

Point the client's base URL at http://127.0.0.1:8613. Provider URLs are base URLs without the protocol path; the proxy appends v1/messages or v1/chat/completions according to the client and provider formats.

MCP tools

Skills found under <notes>/MCP/Skills are also exposed as zero-argument tools.

HTTP API

The MCP service exposes generic REST capabilities on the same port:

The token proxy adds:

Configuration

agent-gateway reads $GATEWAY_HOME/config.yaml unless GATEWAY_CONFIG points to another file. An environment variable always wins over the YAML value; the YAML value wins over the built-in default.

Main config.yaml sections:

contracts/env.json is generated from the implementation and lists every environment variable the package reads. contracts/entries.json lists the module entry points a host can start. Consumers can validate against these files instead of scanning source across repositories.

Development

pip install -e ".[proxy,memory,server]"
pip install ruff build

ruff format --check src tests scripts
ruff check src tests scripts
PYTHONPATH=src python -m unittest discover -s tests -t .
python -m build
agent-gateway --check

The same commands run in GitHub Actions. Enable the local hooks once:

git config core.hooksPath .githooks

pre-commit runs formatting and lint checks; commit-msg enforces type(scope): summary.

Project layout

src/agent_gateway/     package source
  cli/                 command line and contract export
  core/                config, paths, logging, lifecycle constants
  llm/                 single model-call exit point
  services/            MCP, memory, retrieval, graph, search, read, proxy, summarize
  scanner/             AST symbol extractor and JS helper
  prompts/             prompt templates per language
  locales/             UI strings per language
contracts/             generated JSON contracts for host integration
scripts/               maintenance and evaluation scripts
tests/                 unittest suite

Contributing

Issues and pull requests are welcome. Keep changes focused, add tests for behavior changes, and run the development commands above before opening a pull request.

License

MIT. See LICENSE.