URL Agent
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@URL Agentsummarize the website at https://example.com"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
URL Agent
MCP server for intelligent web crawling using a ReAct agent. Adaptively explores websites and returns structured analysis.
Features
ReAct Loop: Agent reasons about which links to follow
Multiple Providers: OpenAI, Anthropic Claude, or local Ollama
Structured Output: Returns JSON with features, requirements, APIs, edge cases, and implementation notes
MCP Server: Single
summarize_urltool for IDE integration
Related MCP server: MCP WebAnalyzer
Quick Start
Using Docker (Recommended)
docker build -t url-agent .
docker run --rm -i -e OPENAI_API_KEY=your_key url-agentLocal Installation
# Install dependencies
uv sync
# Run the server
uv run urlagentConfiguration
Set via environment variables or .env file:
MODEL_PROVIDER=openai # openai, anthropic, or ollama
OPENAI_MODEL=gpt-4o-mini # Model name for any provider
OPENAI_API_KEY=your_key # For OpenAI
ANTHROPIC_API_KEY=your_key # For Anthropic
OLLAMA_BASE_URL=http://localhost:11434/v1 # For Ollama (optional)Using Ollama (local, free):
ollama pull llama3
export MODEL_PROVIDER=ollama
export OPENAI_MODEL=llama3
uv run urlagentUsing Anthropic Claude:
export MODEL_PROVIDER=anthropic
export OPENAI_MODEL=claude-3-5-sonnet-20241022
export ANTHROPIC_API_KEY=your_key
uv run urlagentMCP Integration
Option 1: Docker
{
"mcpServers": {
"url-agent": {
"command": "docker",
"args": ["run", "--rm", "-i", "-e", "OPENAI_API_KEY=YOUR_KEY", "url-agent"]
}
}
}Option 2: Local (after uv sync)
{
"mcpServers": {
"url-agent": {
"command": "urlagent",
"env": {
"OPENAI_API_KEY": "YOUR_KEY"
}
}
}
}Usage
Once registered with your MCP client:
Use summarize_url on https://example.comThe agent will:
Fetch the root page
Intelligently explore promising links
Stop when it has enough information or hits limits
Return structured JSON analysis
Tool Parameters
summarize_url(
url: str, # URL to analyze
max_depth: int = 2, # Maximum crawl depth
max_pages: int = 4 # Maximum pages to fetch
)Project Structure
url-agent/
src/url_agent/ # Main package
server.py # MCP server entry point
react_agent.py # ReAct loop logic
web_fetcher.py # Web scraping utilities
providers/ # LLM provider implementations
pyproject.toml
DockerfileLicense
Apache 2.0
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/rossja/url-agent'
If you have feedback or need assistance with the MCP directory API, please join our Discord server