Skip to main content
Glama
billallison

URL Text Fetcher MCP Server

by billallison
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
Python
Remote

URL Text Fetcher MCP Server

A modern Model Context Protocol (MCP) server that provides URL text fetching, web scraping, and web search capabilities using the FastMCP framework for use with LM Studio and other MCP-compatible clients.

The server is built using the modern FastMCP framework, which provides:

  • Clean decorator-based tool definitions

  • Automatic schema generation from type hints

  • Simplified server setup and configuration

  • Better error handling and logging

All security features and functionality have been preserved while modernizing to follow MCP best practices.

Features

This MCP server enables AI models to:

  • Fetch text content from any URL by extracting all visible text

  • Extract links from web pages to discover related resources

  • Search the web using Brave Search and automatically fetch content from top results

  • Handle errors gracefully with proper timeout and exception handling

Related MCP server: DuckDuckGo MCP Server

Security Features

Enterprise-grade security implementation:

  • SSRF Protection: Blocks requests to internal networks and metadata endpoints

  • Input Sanitization: Validates and cleans all URL and query inputs

  • Memory Protection: Content size limits prevent memory exhaustion

  • Rate Limiting: Thread-safe API rate limiting with configurable thresholds

  • Error Handling: Comprehensive exception handling without information leakage

Tools

The server provides three main tools:

fetch_url_text

  • Description: Downloads all visible text from a URL

  • Parameters:

    • url (string, required): The URL to fetch text from

  • Returns: Clean text content from the webpage

fetch_page_links

  • Description: Extracts all links from a web page

  • Parameters:

    • url (string, required): The URL to fetch links from

  • Returns: List of all href links found on the page

brave_search_and_fetch

  • Description: Search the web using Brave Search and automatically fetch content from the top results

  • Parameters:

    • query (string, required): The search query

    • max_results (integer, optional): Maximum number of results to fetch content for (default: 3, max: 10)

  • Returns: Search results with full text content from each result URL

Prerequisites

Brave Search API Key

To use the search functionality, you'll need a free Brave Search API key:

  1. Visit Brave Search API

  2. Sign up for a free account (2,000 queries/month, max 1 per second)

  3. Get your API key

  4. Copy .env.example to .env and add your API key:

    cp .env.example .env
# Edit .env and set: BRAVE_API_KEY=your_actual_api_key

Installation

  1. Clone this repository

  2. Install dependencies:

    uv sync --dev --all-extras

  3. Configure your environment:

    cp .env.example .env
# Edit .env file and set your BRAVE_API_KEY

Usage

With LM Studio

  1. Open LM Studio and navigate to the Integrations section

  2. Click "Install" then "Edit mcp.json"

  3. Option A: Use the configuration helper script

    ./configure_lmstudio.sh

    This will generate the correct configuration with the right paths for your system.

  4. Option B: Manual configuration - Add the server configuration:

{
  "mcpServers": {
    "url-text-fetcher": {
      "command": "uv",
      "args": [
        "run", 
        "url-text-fetcher"
      ],
      "cwd": "/path/to/mcp-server"
    }
  }
}

Note: The API key will be automatically loaded from your .env file in the project directory.

  1. Save the configuration and restart LM Studio

  2. The server will appear in the Integrations section

Standalone Usage

You can also run the server directly:

uv run url-text-fetcher

Examples

Once configured with LM Studio, you can ask the AI to:

  • "Fetch the text content from https://example.com"

  • "Get all the links from https://news.example.com"

  • "Search for 'Python web scraping' and show me the content from the top 3 results"

  • "What's the latest news about AI? Search and get the full articles"

  • "Find information about MCP servers and fetch the detailed content"

Dependencies

  • mcp>=1.12.3 - Model Context Protocol framework

  • requests>=2.31.0 - HTTP library for web requests and Brave Search API

  • beautifulsoup4>=4.12.0 - HTML parsing and text extraction

Configuration

The server can be configured via the .env file:

# Required: Brave Search API Key
BRAVE_API_KEY=your_api_key_here

# Brave Search API Rate Limit (requests per second)
# Free tier: 1 request per second (default)
# Paid tier: 20 requests per second  
# Higher tier: 50 requests per second
# Set this to match your subscription level
BRAVE_RATE_LIMIT_RPS=1

# Optional: Request timeout in seconds (default: 10)
REQUEST_TIMEOUT=10

# Optional: Content length limit in characters (default: 5000)
CONTENT_LENGTH_LIMIT=5000

# Optional: Maximum response size in bytes (default: 10MB)
MAX_RESPONSE_SIZE=10485760

Brave Search Subscription Tiers

The server automatically adjusts its rate limiting based on your Brave Search subscription:

  • Free Tier: 1 request per second (BRAVE_RATE_LIMIT_RPS=1)

  • Paid Tier: 20 requests per second (BRAVE_RATE_LIMIT_RPS=20)

  • Higher Tier: 50 requests per second (BRAVE_RATE_LIMIT_RPS=50)

The server will enforce the configured rate limit across all concurrent requests to ensure you stay within your API quota.

See .env.example for a template.

Development

This project uses:

  • Python 3.13+

  • uv for dependency management

  • MCP SDK for protocol implementation

To set up for development:

  1. Clone the repository

  2. Run uv sync --dev --all-extras

  3. Make your changes

  4. Test with MCP-compatible clients

Troubleshooting

LM Studio Configuration Issues

If you see errors like "Failed to spawn: url-text-fetcher" in LM Studio logs:

  1. Run the configuration helper:

    ./configure_lmstudio.sh

  2. Make sure you're using full paths:

    • Use the full path to uv (e.g., /Users/username/.local/bin/uv)

    • Include the cwd (current working directory) in your configuration

    • Set the BRAVE_API_KEY environment variable

  3. Test the server manually:

    uv run url-text-fetcher

    The server should start and wait for input (press Ctrl+C to exit).

  4. Check your API key:

    # Check if your .env file has the API key set
cat .env | grep BRAVE_API_KEY

    Or test manually:

    export BRAVE_API_KEY=your_actual_api_key
echo $BRAVE_API_KEY  # Should show your key

Common Issues

  • "BRAVE_API_KEY environment variable is required": Make sure your .env file contains BRAVE_API_KEY=your_actual_api_key

  • "Network error": Check your internet connection and API key validity

  • "Content truncated": Normal behavior for very long web pages (content is limited to 5000 characters by default)

Error Handling

The server includes robust error handling for:

  • Network timeouts (10-second default)

  • Invalid URLs

  • HTTP errors (4xx, 5xx responses)

  • Parsing failures

  • Missing API keys

  • General exceptions

All errors are returned as descriptive text messages to help users understand what went wrong.

Development

This project uses:

  • Python 3.13+

  • uv for dependency management

  • MCP SDK for protocol implementation

To set up for development:

  1. Clone the repository

  2. Run uv sync --dev --all-extras

  3. Make your changes

  4. Test with MCP-compatible clients

Debugging

Since MCP servers run over stdio, debugging can be challenging. For the best debugging experience, we strongly recommend using the MCP Inspector.

You can launch the MCP Inspector via npm with this command:

npx @modelcontextprotocol/inspector uv --directory /Users/wallison/TechProjects/mcp-server run url-text-fetcher

Upon launching, the Inspector will display a URL that you can access in your browser to begin debugging.

License

MIT License - see LICENSE file for details

Available Tools

5 tools
brave_search_and_fetchA

Search the web using Brave Search and automatically fetch content from the top results.

Args: query: The search query max_results: Maximum number of results to fetch content for (default: 3, max: 10)

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes
max_resultsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It mentions 'automatically fetch content from the top results,' which hints at behavior, but it lacks details on rate limits, authentication needs, error handling, or what 'fetch content' entails (e.g., full text, summaries). This is a significant gap for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded, with the core purpose in the first sentence and parameter details in a structured 'Args' section. Every sentence adds value without redundancy, making it efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations, 0% schema description coverage, but an output schema exists, the description is moderately complete. It covers the purpose and parameters but lacks behavioral details like permissions or rate limits. The output schema likely handles return values, so the description doesn't need to explain those, but it should address other contextual gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'query' is for search and 'max_results' controls how many top results to fetch content for, including default and max values. This clarifies beyond the basic schema, though it doesn't detail parameter formats or constraints fully.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('search the web using Brave Search' and 'fetch content from the top results'), and it distinguishes itself from siblings like 'fetch_page_links' and 'fetch_url_text' by combining search with content fetching in one operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for web search and content retrieval, but it does not explicitly state when to use this tool versus alternatives like 'test_brave_search' (which might be for testing) or 'fetch_url_text' (which requires a URL). No exclusions or specific contexts are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_page_linksB

Return a list of all links on the page.

Args: url: The URL to fetch links from

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns a list of links but doesn't cover critical aspects like whether it performs web scraping (implying network calls and potential rate limits), error handling (e.g., for invalid URLs), or output format details. This leaves significant gaps in understanding the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured, with a clear purpose statement followed by parameter details in a labeled 'Args' section. It avoids unnecessary words, though the formatting could be slightly more polished (e.g., bullet points). Every sentence adds value, making it efficient for quick comprehension.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (web scraping for links), no annotations, and an output schema (which handles return values), the description is minimally adequate. It covers the basic purpose and parameter but lacks details on behavioral traits like network dependencies or error scenarios. The output schema reduces the need for return value explanation, but more context would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds meaningful context for the single parameter 'url' by specifying 'The URL to fetch links from,' which clarifies its role beyond the schema's basic title 'Url.' Since schema description coverage is 0%, this compensates well, though it doesn't detail URL format requirements or validation rules. With only one parameter, the baseline is high.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Return a list of all links on the page.' It specifies the verb ('Return') and resource ('list of all links'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'fetch_url_text' (which might fetch text content rather than links), leaving room for minor ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools such as 'fetch_url_text' or 'brave_search_and_fetch', nor does it specify prerequisites, exclusions, or contextual cues for selection. This lack of comparative context limits its utility in guiding the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_url_textA

Download all visible text from a URL.

Args: url: The URL to fetch text from

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states what the tool does, not behavioral traits. It doesn't disclose rate limits, authentication needs, error handling, what 'visible text' means (e.g., excludes scripts/styles), or performance characteristics. The description is minimal beyond the core function.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with zero waste: a clear purpose statement followed by a brief parameter explanation. It's front-loaded with the main function and structured with a labeled 'Args' section, making it easy to parse efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (1 parameter, no annotations but has output schema), the description is reasonably complete for basic use. It covers purpose and parameter semantics adequately. However, it lacks details on behavioral aspects like text extraction scope or error cases, which could be important for an agent's decision-making.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds meaningful context for the single parameter 'url' by specifying it's 'The URL to fetch text from,' which clarifies its role beyond the schema's basic type definition. With 0% schema description coverage and only 1 parameter, this adequately compensates, though more detail on URL format expectations could improve it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Download all visible text') and resource ('from a URL'), distinguishing it from sibling tools like fetch_page_links (which gets links) or brave_search_and_fetch (which includes search). The verb 'download' and scope 'all visible text' provide precise differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for extracting text content from URLs, but provides no explicit guidance on when to use this tool versus alternatives like fetch_page_links or when not to use it (e.g., for non-text content). The context is clear but lacks sibling differentiation or exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_server_infoA

Get information about this MCP server including version, implementation, and capabilities.

Returns: Server information including version, implementation type, and available features

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden and does well by specifying what information is returned. It discloses the return format ('Server information including version, implementation type, and available features'), though it doesn't mention potential limitations like rate limits, authentication needs, or error conditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly front-loaded with the core purpose in the first sentence, followed by a clear returns section. Both sentences earn their place by providing essential information without any redundant or unnecessary content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (0 parameters, no annotations, but has output schema), the description is complete enough. It explains what the tool does and what it returns, and with an output schema present, it doesn't need to detail return value structure. The description covers all essential aspects for this straightforward informational tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on what the tool returns.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Get information') and resource ('this MCP server'), with explicit details about what information is retrieved ('version, implementation, and capabilities'). It distinguishes itself from sibling tools like search/fetch tools by focusing on server metadata rather than web operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving server metadata, but provides no explicit guidance on when to use this tool versus alternatives. There's no mention of prerequisites, timing considerations, or comparison with other tools that might provide similar information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 5 tool updates
    • First observedbrave_search_and_fetch
    • First observedfetch_page_links
    • First observedfetch_url_text
    • First observedget_server_info
    • First observedtest_brave_search
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A3.5/5.0
Disambiguation4/5

Most tools have distinct purposes: search-and-fetch, fetch links, fetch text, get server info, and test search. However, 'brave_search_and_fetch' and 'test_brave_search' could cause some confusion as both involve Brave Search queries, though their primary functions differ (content fetching vs. API testing).

Naming Consistency3/5

The naming is mixed with no clear pattern: 'brave_search_and_fetch' uses underscores and compound verbs, 'fetch_page_links' and 'fetch_url_text' follow a verb_noun style, while 'get_server_info' and 'test_brave_search' use different verb styles. This inconsistency makes the set less predictable but still readable.

Tool Count5/5

With 5 tools, this server is well-scoped for its URL text fetching purpose. Each tool earns its place by covering core functionalities like searching, fetching text/links, server info, and API testing, without being overly sparse or bloated.

Completeness4/5

The toolset covers the main workflows for a URL text fetcher: searching, fetching text and links, and server management. A minor gap is the lack of tools for advanced text processing or filtering, but agents can work around this with the provided fetch operations.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • DuckDuckGo MCP Server

    988664li-star
    A
    license
    B
    quality
    D
    maintenance
    Enables web search through DuckDuckGo and webpage content fetching with intelligent text extraction. Features built-in rate limiting and LLM-optimized result formatting for seamless integration with language models.
    2
    MIT

  • Web Search MCP

    pranavms13
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables web searching through Google, DuckDuckGo, and Bing using a headless Chrome browser, returning structured results with titles, URLs, and snippets. Also supports fetching and extracting text content from any webpage.
    13
    MIT

  • pure.md MCP Server

    ampcome-mcps
    A
    license
    D
    quality
    D
    maintenance
    Enables AI assistants to reliably fetch web content as markdown and search the web by bypassing bot detection and rendering JavaScript. Provides tools to unblock URLs and search the web with results converted to markdown format.
    2
    16
    MIT

Appeared in Searches

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/billallison/brsearch-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server