Skip to main content
Glama
Sonesh-2202

mcp-internet

by Sonesh-2202

๐ŸŒ MCP Internet Server v3.0

Python 3.10+ MCP License: MIT Multi-Platform LM Studio

Give your local LLMs the power of the internet โ€” with real data extraction, not just links! ๐Ÿš€

An open-source Model Context Protocol (MCP) server providing 25 internet tools for local Large Language Models. The v3.0 upgrade adds intelligent search with auto-extraction, multi-source aggregation, result caching, and structured output โ€” making your AI return actual data instead of raw links.

Seamlessly deployable across Windows, macOS, and Linux. No API keys required!


๐Ÿ†• What's New in v3.0

Feature

Before (v2.0)

After (v3.0)

Search Results

Links + snippets only

Full data extraction from pages

Tool Calls Needed

5-10+ per query

1-2 per query

Output Format

Plain text

Markdown tables, bullet points

Caching

None

SQLite with TTL (30min-24h)

Rate Limiting

None

Per-domain throttling

Multi-Source

Manual per source

Auto-aggregation from top results

Content Extraction

Generic text only

Domain-aware (Wikipedia, IMDb, LinkedIn)

New Tools

  • smart_search โ€” ๐ŸŒŸ Searches + scrapes + extracts + formats in one call

  • deep_search โ€” Like smart_search but examines more sources

  • clear_cache โ€” Manage the result cache


Related MCP server: Local-MCP-server

โœจ Key Features

  • ๐Ÿง  Smart Search: One tool call returns extracted data โ€” movie lists, profiles, technical docs โ€” not just links.

  • ๐Ÿ”„ Auto-Extraction: Automatically scrapes top results and extracts structured data (JSON-LD, tables, meta tags).

  • ๐Ÿ“Š Structured Output: Results come as markdown tables, bullet points, and organized sections.

  • โšก Cached Results: Repeated queries are served from SQLite cache in milliseconds.

  • ๐Ÿ›ก๏ธ Rate Limiting: Per-domain request throttling prevents IP bans.

  • ๐Ÿ”€ UA Rotation: Pool of 7 browser user-agents for stealth.

  • ๐ŸŒ Domain-Aware: Special extractors for Wikipedia, IMDb, LinkedIn, and more.

  • ๐Ÿ”’ Zero Config: No API keys, no external subscriptions, no setup hassle.


๐Ÿ› ๏ธ Available Tools (25 Total)

Category

Tools

๐ŸŒŸ Smart Search

smart_search, deep_search

๐Ÿ” Basic Search

search_web, quick_lookup, search_site

๐Ÿ“„ Reading

read_webpage, read_pdf

๐Ÿ“ฐ News/Social

get_news, search_reddit, search_twitter

๐ŸŽฌ YouTube

search_youtube, get_video_info

๐Ÿ’ป GitHub

search_github, get_repo_info

๐ŸŒค๏ธ Info / Utils

get_weather, get_current_time, translate_text, calculate

๐Ÿ”— URLs/IP

shorten_url, get_my_ip, geolocate_ip

๐Ÿ“ฑ Generation

generate_qr, generate_wifi_qr, send_email

๐Ÿ—„๏ธ Cache

clear_cache


๐Ÿš€ Quick Start

Prerequisites

Installation

# 1. Clone the repository
git clone https://github.com/Sonesh-2202/mcp-internet.git
cd mcp-internet

# 2. Install dependencies
uv sync

LM Studio Configuration

Add to your LM Studio mcp.json:

{
  "mcpServers": {
    "mcp-internet": {
      "command": "uv",
      "args": ["run", "mcp-internet"],
      "cwd": "/absolute/path/to/mcp-internet"
    }
  }
}

(Replace /absolute/path/to/mcp-internet with your actual project path)

๐Ÿ“– Detailed OS Guides

For complete, step-by-step instructions for Windows, macOS, and Linux, see the ๐Ÿ‘‰ Multi-Platform Usage Guide.


๐Ÿ’ก How It Works: smart_search vs search_web

Before (v2.0): Multiple tool calls, manual scraping

User: "What are the upcoming Bollywood movies in 2026?"

Tool call 1: search_web("upcoming Bollywood movies 2026")
โ†’ Returns 10 links with snippets

Tool call 2: read_webpage("https://www.imdb.com/...")
โ†’ Returns raw text

Tool call 3: read_webpage("https://www.wikipedia.org/...")
โ†’ Returns raw text

AI must manually piece together the answer from raw text.

After (v3.0): One tool call, structured data

User: "What are the upcoming Bollywood movies in 2026?"

Tool call 1: smart_search("upcoming Bollywood movies 2026")
โ†’ Returns:
  - Extracted movie list with titles, dates, genres
  - Markdown tables with ratings
  - Data from multiple sources combined
  - All in one response, ready to present

๐Ÿ—๏ธ Architecture

mcp-internet/
โ”œโ”€โ”€ pyproject.toml              # Dependencies & Config
โ”œโ”€โ”€ README.md                   # This file
โ”œโ”€โ”€ MULTI_PLATFORM_GUIDE.md     # OS-specific setup guide
โ””โ”€โ”€ src/mcp_internet/
    โ”œโ”€โ”€ server.py               # Entry point โ€” 25 MCP tools
    โ”œโ”€โ”€ tools/
    โ”‚   โ”œโ”€โ”€ smart_search.py     # ๐ŸŒŸ Intelligent search + extraction
    โ”‚   โ”œโ”€โ”€ search.py           # Basic DuckDuckGo search
    โ”‚   โ”œโ”€โ”€ webpage.py          # Page reader with structured extraction
    โ”‚   โ”œโ”€โ”€ news.py             # News headlines
    โ”‚   โ”œโ”€โ”€ youtube.py          # YouTube search & video info
    โ”‚   โ”œโ”€โ”€ github.py           # GitHub repos & users
    โ”‚   โ”œโ”€โ”€ reddit.py           # Reddit search
    โ”‚   โ”œโ”€โ”€ twitter.py          # Twitter/X via Nitter
    โ”‚   โ””โ”€โ”€ ...                 # weather, time, translate, math, etc.
    โ””โ”€โ”€ utils/
        โ”œโ”€โ”€ http_client.py      # Async HTTP with UA rotation & rate limiting
        โ”œโ”€โ”€ cache.py            # SQLite TTL cache
        โ””โ”€โ”€ extractors.py       # Domain-specific data extractors

Data Flow

User Query โ†’ smart_search
    โ”œโ”€โ”€ 1. Check cache (return if fresh)
    โ”œโ”€โ”€ 2. Optimize query & classify (person/movie/news/tech/general)
    โ”œโ”€โ”€ 3. Search DuckDuckGo (HTML scraping + ddgs fallback)
    โ”œโ”€โ”€ 4. Prioritize results by domain authority
    โ”œโ”€โ”€ 5. Scrape top 3 pages in parallel (asyncio.gather)
    โ”œโ”€โ”€ 6. Apply domain-specific extractors
    โ”‚       โ”œโ”€โ”€ Wikipedia โ†’ infobox, summary, key facts
    โ”‚       โ”œโ”€โ”€ IMDb โ†’ titles, ratings, cast, genres
    โ”‚       โ”œโ”€โ”€ LinkedIn โ†’ name, role, skills, experience
    โ”‚       โ””โ”€โ”€ Generic โ†’ JSON-LD, OpenGraph, tables
    โ”œโ”€โ”€ 7. Aggregate & format as structured markdown
    โ”œโ”€โ”€ 8. Cache result
    โ””โ”€โ”€ 9. Return comprehensive response

๐Ÿ”ง Supported Data Sources

Source

What's Extracted

Extractor Type

Wikipedia

Infobox, summary, key facts

Domain-specific

IMDb

Movie/show details, ratings, cast, genres

Domain-specific (JSON-LD)

LinkedIn

Name, role, skills, experience, education

Domain-specific (Googlebot UA)

GitHub

Repos, stars, forks, languages, topics

API-based

Reddit

Posts, scores, comments, subreddit info

JSON API

YouTube

Video titles, channels, views, duration

HTML parsing

Any website

JSON-LD, OpenGraph, HTML tables, main text

Generic fallback


โšก Performance

Metric

Value

Cache hit

< 50ms

Simple search

1-3 seconds

Smart search (3 sources)

3-8 seconds

Deep search (5+ sources)

5-15 seconds

Rate limiting

2 req/sec per domain

UA rotation

7 browser user-agents


๐Ÿงช Testing

# Run the integration test suite
uv run python test_v3.py
  1. "What are the upcoming Bollywood movies in 2026?" โ€” Should return movie lists

  2. "Search the web for Sundar Pichai" โ€” Should return profile data

  3. "Latest albums by Taylor Swift" โ€” Should return music data

  4. "Python asyncio tutorial" โ€” Should return technical content


๐Ÿ“„ License & Contributing

This project is licensed under the MIT License - See LICENSE for details.

Contributions, bug reports, and feature requests are highly welcome!


Made with โค๏ธ for the Local Developer & LLM community

A
license - permissive license
-
quality - not tested
C
maintenance

Maintenance

โ€“Maintainers
โ€“Response time
โ€“Release cycle
โ€“Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Sonesh-2202/mcp-internet'

If you have feedback or need assistance with the MCP directory API, please join our Discord server