What can you do with this server?

A content extraction and summarization server that pulls text from URLs, files, and media, then optionally summarizes it with an LLM. * extract_content — Extract text from a URL (web pages, YouTube videos, PDF links) or a local file path, with no API key needed for most sources. * Choose an extraction engine — Override per input type: auto, simple, firecrawl, jina, crawl4ai for URLs; auto, simple, docling for files. * Tune Docling parsing — Enable formula extraction, image/chart description (pictures), or disable OCR (no_ocr) when using engine=docling. * summarize_content — Summarize any text via an LLM, with an optional context prompt to steer the style (e.g. bullet points). * Requires an LLM key — Summarization needs OPENAI_API_KEY or another configured provider key; audio/video transcription also needs a key.

Which integrations are available for this server?

Exposes a set of compatible tools for Langchain framework, enabling extraction, cleaning, and summarization capabilities directly within Langchain agents and chains. Enables right-click integration with macOS Finder through Services, allowing content extraction and summarization from any supported file with options for clipboard or TextEdit output. Integrates with OpenAI services for transcription (Whisper) and content processing, allowing for AI-powered content extraction and summarization. Provides a Python library for programmatic access to content extraction, cleaning, and summarization capabilities, with asynchronous functionality and customizable options. Offers a Raycast extension with smart auto-detection commands for extracting and summarizing content from various sources, including URLs and files, with multiple output options and visual feedback.

How do I use content-core?

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@content-core extract the main points from this article: https://example.com/tech-news" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

de en es ja ko ru zh

content-core

by lfnovo

Overview Schema Source Related Servers Discussions Admin

Python

Hybrid

Content Core

License: MIT PyPI version Downloads GitHub stars GitHub forks GitHub issues Ruff

Extract, process, and summarize content from URLs, files, and text through a unified async Python API, CLI, or MCP server.

Supported Formats

Category	Formats
Web	URLs, HTML pages, YouTube videos, Reddit posts
Documents	PDF, DOCX, PPTX, XLSX, ODT, ODS, ODP, EPUB, HTML, Markdown, plain text
Media	MP3, WAV, M4A, FLAC, OGG (audio); MP4, AVI, MOV, MKV (video)

Related MCP server: FreeCrawl MCP Server

Quick Start

pip install content-core

import content_core

result = await content_core.extract_content(url="https://example.com")
print(result.content)

Or with zero install:

uvx content-core extract "https://example.com"

CLI Usage

Content Core provides a unified content-core command with subcommands for extraction, summarization, and MCP server.

Extract

# From a URL
content-core extract "https://example.com"

# From a file
content-core extract document.pdf

# With JSON output
content-core extract document.pdf --format json

# With a specific engine
content-core extract "https://example.com" --engine firecrawl

# From stdin
echo "some text" | content-core extract

Summarize

# Summarize text
content-core summarize "Long article text here..."

# With context
content-core summarize "Long text" --context "bullet points"

# From stdin
cat article.txt | content-core summarize --context "explain to a child"

MCP Server

content-core mcp

Configuration

# Set persistent config
content-core config set llm_provider anthropic
content-core config set llm_model claude-sonnet-5

# List current config
content-core config list

# Delete a config value
content-core config delete llm_provider

Config is stored in ~/.content-core/config.toml. Priority: command flags > env vars > config file > defaults.

Zero-Install with uvx

All commands work without installation using uvx:

uvx content-core extract "https://example.com"
uvx content-core summarize "text" --context "one sentence"
uvx content-core mcp

Python API

Extraction

import content_core

# From a URL
result = await content_core.extract_content(url="https://example.com")

# From a file
result = await content_core.extract_content(file_path="document.pdf")

# From text
result = await content_core.extract_content(content="some text")

# With engine override
from content_core import ContentCoreConfig
config = ContentCoreConfig(url_engine="firecrawl")
result = await content_core.extract_content(url="https://example.com", config=config)

Summarization

import content_core

summary = await content_core.summarize("long article text", context="bullet points")

Configuration

from content_core import ContentCoreConfig

config = ContentCoreConfig(
    url_engine="firecrawl",
    document_engine="docling",
    audio_concurrency=5,
)
result = await content_core.extract_content(url="https://example.com", config=config)

MCP Integration

Content Core includes a Model Context Protocol (MCP) server for use with Claude Desktop and other MCP-compatible applications.

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "content-core": {
      "command": "uvx",
      "args": ["content-core", "mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

The MCP server exposes two tools: extract_content and summarize_content. Both return plain text.

For detailed setup, see the MCP documentation.

Agent Skill (Claude Code & Codex)

Content Core ships an Agent Skill that teaches AI agents how to use it for extracting content from external sources. This repository is also a plugin marketplace, so the skill installs natively in both harnesses.

Claude Code — add the marketplace and install the plugin:

/plugin marketplace add lfnovo/content-core
/plugin install content-core@content-core

Codex — the repository carries a Codex plugin manifest (.codex-plugin/plugin.json) and marketplace catalog (.agents/plugins/marketplace.json) pointing at the same skill.

Manual fallback — copy the skill file directly into your project:

curl -o .claude/skills/content-core/SKILL.md --create-dirs \
  https://raw.githubusercontent.com/lfnovo/content-core/main/skills/content-core/SKILL.md

Once installed, the agent can use content-core to extract content from URLs, documents, and media files — either via CLI (uvx content-core) or MCP if configured.

AI Providers

Content Core uses Esperanto to support multiple LLM and STT providers. Switch providers by changing the config — no code changes needed:

# Use Anthropic for summarization
content-core config set llm_provider anthropic
content-core config set llm_model claude-sonnet-5

# Use Groq for transcription
content-core config set stt_provider groq
content-core config set stt_model whisper-large-v3

Supported providers include OpenAI, Anthropic, Google, Groq, DeepSeek, Ollama, and more. See the Esperanto documentation for the full list.

Configuration

Content Core uses ContentCoreConfig powered by pydantic-settings. Settings are resolved in priority order: constructor args > env vars (CCORE_*) > config file (~/.content-core/config.toml) > defaults.

Environment Variables

Variable	Description	Default
`CCORE_URL_ENGINE`	URL extraction engine (`auto`, `simple`, `firecrawl`, `jina`, `crawl4ai`)	`auto`
`CCORE_DOCUMENT_ENGINE`	Document extraction engine (`auto`, `simple`, `docling`) — `docling` raises `ConfigurationError` if the extra is not installed; `auto` falls back silently	`auto`
`CCORE_AUDIO_CONCURRENCY`	Concurrent audio transcriptions (1-10)	`3`
`CCORE_AUDIO_SEGMENT_MINUTES`	Length of the segments long audio is split into; `0` sends the file whole	`10`
`CRAWL4AI_API_URL`	Crawl4AI Docker API URL (omit for local browser mode)	-
`CRAWL4AI_API_TOKEN`	Bearer token for the Crawl4AI Docker API (required by Crawl4AI >= 0.9.0)	-
`FIRECRAWL_API_URL`	Custom Firecrawl API URL for self-hosted instances or Firecrawl-compatible backends (e.g. fastCRW)	-
`CCORE_FIRECRAWL_PROXY`	Firecrawl proxy mode (`auto`, `basic`, `stealth`)	`auto`
`CCORE_FIRECRAWL_WAIT_FOR`	Wait time in ms before extraction	`3000`
`CCORE_LLM_PROVIDER`	LLM provider for summarization	-
`CCORE_LLM_MODEL`	LLM model for summarization	-
`CCORE_STT_PROVIDER`	Speech-to-text provider	-
`CCORE_STT_MODEL`	Speech-to-text model	-
`CCORE_STT_TIMEOUT`	Speech-to-text timeout in seconds	-
`CCORE_YOUTUBE_LANGUAGES`	Preferred YouTube transcript languages	-
`CCORE_YOUTUBE_COOKIES_FILE`	Path to a Netscape `cookies.txt` for YouTube transcripts (missing file raises `ConfigurationError`)	-
`CCORE_YOUTUBE_PROXY`	Proxy URL for YouTube transcripts, e.g. `http://user:pass@host:port`	-

API keys for external services are set via their standard environment variables (e.g., OPENAI_API_KEY, FIRECRAWL_API_KEY, JINA_API_KEY).

Proxy Configuration

Content Core reads standard HTTP_PROXY / HTTPS_PROXY / NO_PROXY environment variables automatically. No additional configuration is needed.

YouTube on Blocked Networks

If YouTube extraction fails with IpBlocked/RequestBlocked on a flagged IP (cloud hosts, VPNs), either point CCORE_YOUTUBE_COOKIES_FILE at a cookies.txt exported from a logged-in browser (re-export it when the cookies expire), or set CCORE_YOUTUBE_PROXY to a residential proxy — datacenter proxies get blocked or CAPTCHA'd. See docs/usage.md.

Optional Dependencies

# Docling for advanced document parsing (PDF, DOCX, PPTX, XLSX)
# Required by document_engine="docling", which raises ConfigurationError without it.
# Use document_engine="auto" (default) or "simple" to proceed without Docling.
pip install content-core[docling]

# Crawl4AI for local browser-based URL extraction
pip install content-core[crawl4ai]
python -m playwright install --with-deps

# LangChain tool wrappers
pip install content-core[langchain]

# All optional features
pip install content-core[docling,crawl4ai,langchain]

Using with LangChain

When installed with the langchain extra, Content Core provides LangChain-compatible tool wrappers:

from content_core.tools import extract_content_tool, summarize_content_tool

tools = [extract_content_tool, summarize_content_tool]

Documentation

Usage Guide -- Python API details, configuration, and examples
Processors -- How content extraction works for each format
MCP Server -- Claude Desktop and MCP integration

Development

git clone https://github.com/lfnovo/content-core
cd content-core

uv sync --group dev

# Run tests
make test

# Lint
make ruff

License

This project is licensed under the MIT License.

Contributing

Contributions are welcome! Please see our Contributing Guide for details.

Available Tools

2 tools

extract_contentExtract ContentA

Extract content from a URL or file. Does not require an API key for most sources (web pages, PDFs, documents, YouTube transcripts). API key is only needed for audio/video transcription.

ParametersJSON Schema

Name	Required	Description
`url`	No	URL to extract content from (web page, YouTube video, PDF link, etc.)
`engine`	No	Optional extraction engine override, routed by input type. With `url`: auto, simple, firecrawl, jina, crawl4ai. With `file_path`: auto, simple, docling — `docling` requires `pip install "content-core[docling]"` and fails with a configuration error when the extra is missing, in which case use `auto` or `simple`. Any other value is rejected with an error naming the accepted ones.
`no_ocr`	No	Disable OCR in Docling (requires engine=docling)
`formulas`	No	Enable formula extraction via Docling (requires engine=docling)
`pictures`	No	Enable image description + chart data extraction via Docling (requires engine=docling)
`file_path`	No	Local file path to extract content from

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A3.5/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden; it usefully discloses the auth profile (no key needed for most sources, key needed for audio/video). It is silent on other behaviors: what happens if neither url nor file_path is supplied (0 required params), and whether unsupported sources error or degrade.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the core action, and the auth caveat placed immediately after. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, and the auth context is a genuine addition. Still missing is guidance on the unusual 0-required-parameter shape (must one of url/file_path be given?) and any routing versus the sibling tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents engine routing, docling requirements, and the flag semantics in detail. The description adds only a loose mapping of source types, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Extract') and resource ('content from a URL or file') plus the supported source types. It does not explicitly differentiate itself from the sibling summarize_content, so an agent must infer that this returns raw content rather than a summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives conditional guidance about when an API key is required (audio/video transcription only), which is useful. However, it never says when to choose this over summarize_content, nor does it explain the url-vs-file_path choice, leaving usage largely implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

summarize_contentSummarize ContentB

Summarize content using LLM with optional context. Requires OPENAI_API_KEY (or another LLM provider key) to be configured.

ParametersJSON Schema

Name	Required	Description	Default
`content`	Yes	The text content to summarize
`context`	No	Optional context to guide summarization (e.g., "summarize as bullet points")

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

B3.2/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It usefully discloses a real prerequisite — an LLM provider key such as OPENAI_API_KEY must be configured — which is not in the schema. However it says nothing about cost, latency, or truncation behavior on long input, leaving gaps for a tool that calls an external LLM.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with no filler, and the core purpose is front-loaded ahead of the setup requirement. Slightly terse, but every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists so return values need no explanation, and both parameters are schema-documented, making the definition largely adequate for a simple tool. The gaps are the absence of any routing guidance versus extract_content and the lack of cost/latency context for an external LLM call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already fully documented, and the description adds nothing beyond restating that context is optional. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('summarize content') plus the mechanism ('using LLM'), which distinguishes it from the sibling extract_content at a high level. It does not explicitly name or contrast with that sibling, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to choose this tool over extract_content, nor any exclusions or preconditions for use beyond the API-key note. The reader must infer that summarizing is the use case from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

2 tool updatesv2.1.1Oct 5, 2026

Changedextract_content6 fields changed

addedInput schema / properties / engine / description

Added value: +"Optional extraction engine override, routed by input type.\nWith `url`: auto, simple, firecrawl, jina, crawl4ai.\nWith `file_path`: auto, simple, docling — `docling` requires\n`pip install \"content-core[docling]\"` and fails with a\nconfiguration error when the extra is missing, in which case use\n`auto` or `simple`.\nAny other value is rejected with an error naming the accepted ones."

addedInput schema / properties / file_path / description

Added value: +"Local file path to extract content from"

addedInput schema / properties / formulas / description

Added value: +"Enable formula extraction via Docling (requires engine=docling)"

addedInput schema / properties / no_ocr / description

Added value: +"Disable OCR in Docling (requires engine=docling)"

addedInput schema / properties / pictures / description

Added value: +"Enable image description + chart data extraction via Docling (requires engine=docling)"

addedInput schema / properties / url / description

Added value: +"URL to extract content from (web page, YouTube video, PDF link, etc.)"

Changedsummarize_content2 fields changed
- addedInput schema / properties / content / description
```
Added value: +"The text content to summarize"
```
- addedInput schema / properties / context / description
```
Added value: +"Optional context to guide summarization (e.g., \"summarize as bullet points\")"
```

2 tool updatesv2.0.4Jul 13, 2026

Changedextract_content9 fields changed

addedInput schema / additionalProperties
```
Added value: +false
```

addedInput schema / properties / engine

Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null
+}

addedInput schema / properties / formulas

Added value: +{
+  "default": false,
+  "type": "boolean"
+}

addedInput schema / properties / no_ocr

Added value: +{
+  "default": false,
+  "type": "boolean"
+}

addedInput schema / properties / pictures

Added value: +{
+  "default": false,
+  "type": "boolean"
+}

removedOutput schema / additionalProperties
```
Removed value: -true
```

addedOutput schema / properties

Added value: +{
+  "result": {
+    "type": "string"
+  }
+}

addedOutput schema / required
```
Added value: +[
+  "result"
+]
```
addedOutput schema / x-fastmcp-wrap-result
```
Added value: +true
```

Addedsummarize_content

1 tool updatev1.0.0Jan 3, 2026
- Changedextract_content2 fields changed
  - removedInput schema / properties / file_path / title
```
Removed value: -"File Path"
```
  - removedInput schema / properties / url / title
```
Removed value: -"Url"
```
1 tool updateAug 15, 2025
- First observedextract_content

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.6/5.0

Scored 2026-10-05 10:20 across 2 tools

Disambiguation5/5

The two tools target clearly distinct actions: extract_content pulls raw content from a source, while summarize_content condenses it via an LLM. An agent can easily tell which to call based on whether it needs raw text or a summary.

Naming Consistency5/5

Both names follow the same verb_noun snake_case pattern (extract_content, summarize_content), with no deviations in convention or style.

Tool Count3/5

Two tools is on the thin side for even a focused server, though each earns its place in a natural extract-then-summarize pipeline. A couple of supporting operations (e.g. format conversion or output saving) would round out the surface.

Completeness4/5

The extract and summarize pair covers the core content-processing lifecycle, including multiple input sources (URLs, PDFs, YouTube). Minor gaps exist around batch processing, output formatting, or chunking, but agents can work around these.

Maintenance

ActivityMaintained

ResponsivenessWithin a week

Resources

GitHub Repository

Need Help?

Related Servers

Related MCP Connectors

CoreWise A
io.github.ryanportfolio
Extract structured insights from videos, podcasts, articles, and PDFs with multi-model AI
AI & Machine Learning Multimedia Processing Research & Data
42
arjunkmrm-scrapermcp_elOAuth
ai.smithery
Extract and parse web pages into clean HTML, links, or Markdown. Handle dynamic, complex, or block…
Web Scraping Browser Automation
2
PDF to Markdown & Document ParserOAuth
io.github.fernandoguiraud16-coder
Convert PDF, Word, Excel and scanned documents to Markdown, tables and RAG chunks. OCR.
RAG Systems Data Platforms
1
stagenth · 文档解析 A
com.stagenth
Parse PDF/Word/PPT/HTML to Markdown; tables as JSON, image extraction, RAG chunking, page ranges.
RAG Systems File Systems
2

Related MCP Servers

Web-curl MCP Server
Web Scraping Search Browser Automation
rayss868
A
license
C
quality
B
maintenance
A powerful tool for fetching and extracting text content from web pages and APIs, supporting web scraping, REST API requests, and Google Custom Search integration.
Updated 14 days ago (2026-09-29 00:40 UTC)
5
10
MIT
FreeCrawl MCP Server
Web Scraping Browser Automation Documentation Access
dylan-gluck
A
license
Not graded
quality
D
maintenance
Enables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.
Updated a year ago (2025-08-20 03:34 UTC)
4
MIT
ParseJet - Universal File &
Documentation Access Multimedia Processing Web Scraping
yooumuu
A
license
B
quality
C
maintenance
Parse any file or URL into structured text. Extract text from PDF, DOCX, YouTube, web pages, images, and 25+ formats via one API. Tools: parse_url, parse_file, get_youtube_transcript.
Updated 7 months ago (2026-03-21 15:18 UTC)
3
10 npm
MIT
readable-mcp
Browser Automation Web Scraping
tommypj
A
license
Not graded
quality
D
maintenance
Converts any URL into clean, LLM-ready Markdown, text, or HTML with production-grade features like SSRF protection, rate limiting, retries, caching, and structured error handling.
Updated 4 months ago (2026-06-23 11:40 UTC)
MIT

Appeared in Searches

A file opener for reading PDF files, images, and other file types

Content Core

Supported Formats

Quick Start

CLI Usage

Extract

Summarize

MCP Server

Configuration

Zero-Install with uvx

Python API

Extraction

Summarization

Configuration

MCP Integration

Agent Skill (Claude Code & Codex)

AI Providers

Configuration

Environment Variables

Proxy Configuration

YouTube on Blocked Networks

Optional Dependencies

Using with LangChain

Documentation

Development

License

Contributing

Available Tools

Output Schema

TDQS

Output Schema

TDQS

Tool Schema Changelog

TDQS

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

Web-curl MCP Server

FreeCrawl MCP Server

ParseJet - Universal File &

readable-mcp

Appeared in Searches